• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Tim Cooke
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Knute Snortum
  • paul wheaton
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Ron McLeod
  • Piet Souris
  • Ganesh Patekar
Bartenders:
  • Tim Holloway
  • Carey Brown
  • salvin francis

FixedThreadPool issue: more threads then expected seems to be working

 
Ranch Hand
Posts: 44
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich
 
author
Posts: 23838
141
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

hildich kilal wrote:Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich




The fixed thread pool is exactly that -- it doesn't create new thread pool threads. So, I would recommend looking at your callable again.


However, if you don't believe that this is so (ie. you disagree), then do a thread dump to check how many threads are in the thread pool.

Henry
 
Bartender
Posts: 1104
10
Netbeans IDE Java Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A FixedThreadPool will create a max of n threads at a given point. But, if you submit more tasks, once earlier threads are done and the current number of threads become less than n, then it will create further threads for those tasks.

This is what is said in the API:
"At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available."

So, if you haven't closed the connection properly, then there is a possibility many connections being open...
 
Sheriff
Posts: 3752
62
Netbeans IDE Oracle Chrome
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'd just mention that to effectively process 20 requests in your database concurrently, the database must be sized up to the task. If it isn't, the processing might actually take longer and squeeze out other users from the database.

That said, moving the processing to the database might be much, much more effective to shorten the processing time.
 
Trust God, but always tether your camel... to this tiny ad.
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!