6

I have a code block

for (i=0;i<size;i++) { do job; } 

initially this job task was executed sequentially (as shown above) but later I made multithreaded with normal threads (inner runnable class implementation) like,

for (i=0;i<size;i++) { new threadingclass(some args) } runnable threadingclass { pub void run () { do job; } } 

this worked fine with some thread limit (till system resources were enough) so to avoid resource overloading I implemented same code with standard theadpool implementation (threadpool,executor service and worker thread implementation)

threadexecutor t=new threadexecutor(size) for (i=0 ; i<size ; i++) { t.execute(new threadingclass(some args)) } runnable threadingclass { pub void run () { do job; } } 

now scenario was like,

I wanted to run loop for 25 times (no. of threads), I tried with all 3 implementations

  1. sequential : takes 7 min approx
  2. normal multithreading : 40 sec
  3. multithreading with threadpool (size : 100) : 2 min approx

I am bit confused why normal threading and thredpool implementation timings differ so much and internally also threadpool does not involve much complex logic. any help is appreciated.

Thanks in advance

10
  • Threadpool overhead is very, very small. It seems likely there is a problem in your implementation, but without code can only guess. Commented Feb 4, 2014 at 10:03
  • You should try to profile the JVM while running your code and see where it spends all that time. Use the -Xprof option for the built-in profiler. Commented Feb 4, 2014 at 10:04
  • What is threadexecutor? Did you try using the JVM-Classes instead of your own? Commented Feb 4, 2014 at 10:05
  • Does this mean that you created 25 manual threads, but 100 in the Thread Pool? 100 Threads is quite a lot (unless you have 64 core machine....). Try using Runtime.getRuntime().availableProcessors() threads. Also: What kind of Executor did you create? Commented Feb 4, 2014 at 10:05
  • 5
    So can you show us the real code you executed and not just pseudo code? So we can try it ourselves and see what is wrong. Commented Feb 4, 2014 at 10:18

2 Answers 2

1

It mostly depends on which ExecutorService you chose. Here it seems you chose a FixedThreadPool, which is basically equivalent to launching your threads in parallel if its size is big enough to hold all the threads. You might even get some performance improvement since threads are not creating on the fly.

ExecutorService is usually the way to go since it is readable, maintainable and has almost no overhead. It has also been heavily tested over the past years.

Your results clearly reveal an implementation problem: you probably ran your tests with size = 100 for the ExecutorService example and with size = 25 for the other ones.

Sign up to request clarification or add additional context in comments.

Comments

0

Your processor most likely only has 4 or 8 cores. Every thread after that many is actually slowing your system down as it has to keep interrupting a running thread in order to give the next one some time.

Run the thread pool with 4 threads and the manual threads with 4 threads and you will see very little difference in performance.

ThreadPool sizes can be configured, as can the number of threads you spawn... set them both to the same number for a valid test.

10 Comments

It seems running the 25 jobs in separate threads would have precisely the same problem, though.
@Dolda2000 but 25 is a lot fewer than 100.
Yes I have servers with 8 cores and 24 GB RAM. actually I have to run this implementation against thread size ~(100-200) so 3-4 threads anyhow will not useful
Sure, but those extra 75 threads should just by lying idle, not disturbing anything anyway.
You are doing something wrong then. Sorry I can't be more detailed than that but if you spin up 1 thread per processor and then have it working hard you will see every core max unless there is something in your OS that is limiting resources allocated to that application.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.