Win a copy of React Cookbook: Recipes for Mastering the React Framework this week in the HTML Pages with CSS and JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Rob Spoor
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Junilu Lacar
  • Tim Cooke
Saloon Keepers:
  • Tim Holloway
  • Piet Souris
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Frits Walraven
  • Himai Minh

FixedThreadPool issue: more threads then expected seems to be working

 
Ranch Hand
Posts: 44
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich
 
author
Posts: 23906
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

hildich kilal wrote:Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich




The fixed thread pool is exactly that -- it doesn't create new thread pool threads. So, I would recommend looking at your callable again.


However, if you don't believe that this is so (ie. you disagree), then do a thread dump to check how many threads are in the thread pool.

Henry
 
Bartender
Posts: 1104
10
Netbeans IDE Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A FixedThreadPool will create a max of n threads at a given point. But, if you submit more tasks, once earlier threads are done and the current number of threads become less than n, then it will create further threads for those tasks.

This is what is said in the API:
"At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available."

So, if you haven't closed the connection properly, then there is a possibility many connections being open...
 
Sheriff
Posts: 3837
66
Netbeans IDE Oracle Firefox Browser
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'd just mention that to effectively process 20 requests in your database concurrently, the database must be sized up to the task. If it isn't, the processing might actually take longer and squeeze out other users from the database.

That said, moving the processing to the database might be much, much more effective to shorten the processing time.
 
We must storm this mad man's lab and destroy his villanous bomb! Are you with me tiny ad?
Thread Boost feature
https://coderanch.com/t/674455/Thread-Boost-feature
reply
    Bookmark Topic Watch Topic
  • New Topic