• Post Reply Bookmark Topic Watch Topic
  • New Topic

threading program creating more threads than threadPoolSize at any moment  RSS feed

 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a multithreaded program and have specified threadPoolSize parameter as 10.Despite this I can see it creating a lot more than 10 threads in the beginning. I could not understand what is the mistake in below code which is causing more thread to be created than max thread count.

Below is my code:



I could not understand what I am doing wrong because of which it is creating so many threads instead of 10.

Thanks
 
Ulf Dittmer
Rancher
Posts: 42972
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How are you determining the number of threads being created? This code wouldn't seem to create more than a single one (in line 25/27). (Of course, it's not a SSCCE and seems to have been edited, so it's hard to tell what it actually does.)

In the future, please remove all unnecessary white space that makes it hard to read your code.
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How are you determining the number of threads being created?


If i have a line at starting of run method saying : System.out.println("thread starts"); (and also print the thread name)

and

If I have a line at end of run method saying System.out.println("thread ends"); (and also print the thread name)


And if the first line prints more than 20 times when the last line has got printed just 5 times than it means that it has started more than 10 threads at once.

This code wouldn't seem to create more than a single one (in line 25/27). (Of course, it's not a SSCCE and seems to have been edited, so it's hard to tell what it actually does.)


The line 25-27 is :

workers[i] = new MyThread(Integer.valueOf(i).toString(), file)

This line created a a worker thread so there are as many worker threads as number of files in the folder but maxpoolsize is only 10.


This code wouldn't seem to create more than a single one


I could not understand that why it would create just a single thread.Would it not create worker threads equal to the number of files in folder and than create max 10 threads at once?

 
Ulf Dittmer
Rancher
Posts: 42972
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I could not understand that why it would create just a single thread.Would it not create worker threads equal to the number of files in folder and than create max 10 threads at once?

That goes back to what I said:
Of course, it's not a SSCCE and seems to have been edited, so it's hard to tell what it actually does.

The code you posted creates exactly one thread, namely workers[0]. There are some leftovers that indicate it may be used as part of a loop (namely, "i++"), but since there is no loop it won't create more than that one thread. So please http://www.coderanch.com/how-to/java/PostRealCode. As it is, it is missing important pieces we need to see in order to understand what's happening.
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I apologize for that. I have pasted the code below. So it is creating worker threads inside a loop as below:




Here the the code I am pasting again:



 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:And if the first line prints more than 20 times when the last line has got printed just 5 times than it means that it has started more than 10 threads at once.

You still haven't posted all the code so it's hard to know exactly where you have put the print statements but I think you may have misunderstood what how the Executor works. Your loop won't wait at execute() once 10 threads are running, execute() immediately returns each time returning a Future object which you can use to find out if the task has completed, get a return value etc. There is no guarantee as to when the task will be run just that it will be run at some point in the future.
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think you may have misunderstood what how the Executor works. Your loop won't wait at execute() once 10 threads are running, execute() immediately returns each time returning a Future object which you can use to find out if the task has completed, get a return value etc. There is no guarantee as to when the task will be run just that it will be run at some point in the future.


My Understanding:

When using Executer there are 2 parameters and below is how they work:

1) number of worker threads: It is the total number of threads. But how many threads will be running at a time will be governed by the below parameter:

2) threadPoolSize : The number of threads running at a particular time.


So if the number of worker threads is 1000 and the threadPoolSize is 10 then not all 1000 threads would be running but only 10 at a time would be running.So at any point of time the number of threads actually running would be =10. Please correct me if I am wrong here

 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Can you show us the code for the MyThread class?
 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:
So if the number of worker threads is 1000 and the threadPoolSize is 10 then not all 1000 threads would be running but only 10 at a time would be running.So at any point of time the number of threads actually running would be =10. Please correct me if I am wrong here


Not sure what you mean by "the number of worker threads is 1000". With a thread pool, you don't create any worker threads. The thread pool creates the worker threads that it will use. You need to create runnable (or callable) instances, for the thread pool to execute.

Henry
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks all.

I had used the below example to create my thread pool.


Not sure what you mean by "the number of worker threads


In the below example is the code not creating 1000 worker threads out of which only 10 would be running at a time?
 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:In the below example is the code not creating 1000 worker threads out of which only 10 would be running at a time?

No, it's creating a 1000 instances of a class called WorkerThread, these aren't worker threads. This highlights why giving things good names is so important, calling your class WorkerThread is misleading because it isn't a worker thread it's a task to be run by a worker thread .
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
it's creating a 1000 instances of a class called WorkerThread


Please correct me if I am wrong here:


It s creating a 1000 instances of a class called WorkerThread. Now when these 1000 instances have been created it does not mean that 1000 threads are running. At any time only 10 threads will run?
 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
When you create 1000 instances of your WorkerThread class no threads will be running (other than the ones that are already running). Once you call execute on your Executor mutliple times then up to 10 extra threads will be running. I say up to because it depends on how quickly you repeatedly call execute and how long each task takes to run.
 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:
It s creating a 1000 instances of a class called WorkerThread. Now when these 1000 instances have been created it does not mean that 1000 threads are running. At any time only 10 threads will run?



Why don't you tell us? Your WorkerThread class, when the run() method is called, seems to print it's start and end. Do you see, at any time, more than 10 instances of you class started, but not ended yet?

Henry
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I realised that I had done a blunder. I had the line t.start(); in the constructer of the MyThread class. I removed this line and it worked as expected. But now I have a doubt.

Despite writing the t.start() in constructor the time taken for program to complete was different with different value provided for ThreadPoolSize.


E.g time taken with

was a lot different with time taken by program when below line as used:



or time taken when below was used:





If we are using Thread Pool and specifying t.start() it is a mistake so in that case it will create threads immediately.So why is there difference in time taken in above cases despite specifying t.start() ?
 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:I realised that I had done a blunder. I had the line t.start(); in the constructer of the MyThread class. I removed this line and it worked as expected.

In future please post the exact code you are running or we are trying to fix problems that are being caused by something we don't know about.

Monica. Shiralkar wrote:
But now I have a doubt.

Despite writing the t.start() in constructor the time taken for program to complete was different with different value provided for ThreadPoolSize.

You can't have just had t.start() in the constructor as that wouldn't have compiled, there must have been other differences as well.
Without seeing the exact code you were running I can't answer this without guessing.
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Below is the complete code where t.start might be creating and issue:

}
 
Henry Wong
author
Sheriff
Posts: 23295
125
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:
If we are using Thread Pool and specifying t.start() it is a mistake so in that case it will create threads immediately.So why is there difference in time taken in above cases despite specifying t.start() ?


If you are doing both, well, in the first case, you are creating 1010 threads. 1000 of those threads will do the task once, and the other 10 threads will process the 1000 tasks, 10 tasks at a time. In the second case, you are creating 1015 threads. 1000 of those threads will do the task once, and the other 15 threads will process the 1000 tasks, 15 tasks at a time. And in both cases, you are doing each task twice (once with its own thread, and once by the pool).

Can you guess from this description why one case will finish first?

Henry
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Can you guess from this description why one case will finish first?


I think by specifying thread.start it will take more time than just specifying the pool.

So is doing both a completely wrong approach.

One more thing which I could not understand is that why it is taking different time when specifying different maxpoolsize parameter as below if all it is taking is thread.start:

E.g time taken with




was a lot different with time taken by program when below line as used:




or time taken when below was used:


If we are using Thread Pool and specifying t.start() it is a mistake so in that case it will create threads immediately.So why is there difference in time taken in above cases despite specifying t.start()
 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:
One more thing which I could not understand is that why it is taking different time when specifying different maxpoolsize parameter as below if all it is taking is thread.start:

Threading is a complex subject and making things perform faster is not just a case of creating more threads, in fact in some circumstances making more threads can just slow things down. The optimum number of threads depends on a range of factors such as the task being computed on the thread, the number of processors in the computer etc.

In your case, because all the thread task does is wait for 5 seconds it uses very little processor time and so creating more threads in the thread pool will bring the overall run time down. The difference in time taken has nothing to do with using t.start(), you will see the same difference without using t.start().
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The difference in time taken has nothing to do with using t.start(), you will see the same difference without using t.start().


Is using t.start() not a mistake as I am thinking?
 
Tony Docherty
Bartender
Posts: 3271
82
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Monica. Shiralkar wrote:
The difference in time taken has nothing to do with using t.start(), you will see the same difference without using t.start().


Is using t.start() not a mistake as I am thinking?

Yes it's a mistake (unless you really want to run each task twice - once on their own thread and then again on a pool thread). The point I was making was having or not having t.start() isn't anything to do with why you see a difference in execution time when you change the thread pool size.
 
Monica Shiralkar
Ranch Hand
Posts: 922
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks all.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!