I have a server processing requests on a pool of some number of threads. It happens to be
EJB but that may not matter. Many different types of requests come in and go to threads first come first serve I guess.
We handle most requests in tenths of a second. But many requests make calls out to partner systems and every once in a while there is a problem that causes us to get no response. The request times out in the container defult 120 seconds.
One
thread blocking that long isn't much of a problem but it doesn't take long before the same kind of request to the same sick partner comes in on all the other threads, and the whole pool is blocked. The container makes more threads up to some limit but that only bogs the server down with resource issues and they all get stuck soon enough.
So I'd like to say "out of 50 threads in the pool, never use more than 15 on partner X" The 16th request can't block because that's the problem we're trying to eliminate. I think I want the 16th request to throw a "partner service busy" exception.
I'm thinking about a named pool of tokens. The "CallPartnerX" service would try to get a token from the "PartnerX" pool. So this feels like a map of some collection.
Any other ideas?