Michael Remijan wrote:...
This behavior is odd and I can't quite figure out why it works this way.
Yeah, when I think of how it works I tend to think of there being two Thread Pools. The first (from 0 to minimumThreadCount) is the main Thread Pool which represents your best-effort compromise between speed (more threads) and resources. The Queue and the threads between minimum and maximumThreadCount are fall-back behaviors. What happens when your best compromise between speed and resources leaves tasks unable to complete? Queue them up so they can be gotten to when a Thread has time. But even this may need a fall back mechanism - if tasks need to be done in a timely fashion, or just storing those tasks is memory intensive then you need some way to relieve the burden - and this is where the extra Thread Pool (count between minimum and maximum Threads) comes into play. Using more than the minimum number of Threads is non-optimal, so try to avoid it and if you can't. then go ahead and make more Threads but let those Threads timeout if they don't get used so we drop back down to the optimal situation when we can.
This mental concept of how it works is helped by the Java 1.5 java.util.concurrent implementation which calls those threads between 0 and minimumThreadCount the 'Core Pool.' I have no idea if this is how Doug Lea envisioned it when he developed the idea but it works for me.
When I think of a thread pool, I think about a minimum number of threads sitting there waiting to process commands. A command goes in and it goes to a thread, if there are no threads available, start adding new threads until a maximum is reached. If the maximum is reached and commands are still coming in, then start queuing the requests. If the max of the queue is then reached, then some sort of rejection policy is consulted on what to do next. I'd like a pool which does this.
So basically this is what you want to do, if I understand:
When a new task comes in ...
1) Either pre-start minimum threads, or prefer making a new thread when less than minimum threads exists
2) If minimum threads already exist send the task to an already present, idle thread
3) If there are no idle threads and there are less than maximum threads running, then add a new thread to execute the task
4) If there are already maximum threads running, then queue the task
5) If the queue is filled, then send it to a rejection policy
This is essentially what happens when you have minimum == maximum thread count, with the exception of step 2. If this is important to you then you might consider using a synchronous queue to send tasks to the pool. A synchronous queue is essentially a zero sized blocking queue (I think in the library you are using they are called SynchronousChannel) - so when the task is added to the queue it is either immediately taken by a Thread or Queueing fails - forcing a new non-core Thread to be made. If all threads are already made it would be sent to the rejection policy.
And here is where your custom code comes in. You implement a rejection policy which takes the failed task and re-queues it into a fixed sized queue, and if that fixed sized queue fills up, then passes it on to one of the pre-implemented rejection policies. The trick is then to get tasks from this queue back into the execution circuit which you could do either by using a 're-filling' thread which attempts to re-execute the task or by modifying the Worker implementation to check this back-up queue before using the 'new task' queue.