• Post Reply Bookmark Topic Watch Topic
  • New Topic

Threads update controller is it possible  RSS feed

 
Georgios Chatziefstratiou
Ranch Hand
Posts: 106
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all ,

i have code that i count the current threads running on a sping project that is updating some subscribers.

The time between these updates is enough for a thread to finish the task that this thread has.Say i have subscriber 1 subscriber 2 subscriber 3
If a thread for example 1 goes to task(subscriber 1 ) 1 and stays blocked because of long running task and the other tasks are finished(subscriber 2 subscriber 3 ) , when a new update comes one thread is still running and the thread pool size is minus one thread. Example 5 threadpoolsize and then 4 threadpoolsize ,Because 1 thread is running.
The update will use a new thread and then second thread will have to wait for the first to finish the 1 task(subscriber 1 ) so that it can start and the other tasks are finished(subscriber 2 subscriber 3 ) .
Then the new update will come and start the new update and still a new thread will start to do the task and this have to wait until the first finishes and the second thread finishes, o i have minus 3 threads from the threadpoolsize.
So then I have 3 thread in waiting to do something. If this happens eventualy I ' ill be running out of threads if I have a thread pool of 5 threads.

and my question is how can i make the new thread notice that this subscriber is busy and fail the task for later send it to db to store it for later update.
I have seen semaphores and locks. Is there any other way or something else ?
I actually want to create a Thread Controller.

thanks
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Georgios Chatziefstratiou wrote:how can i make the new thread notice that this subscriber is busy and fail the task for later send it to db to store it for later update.

Look into ThreadPoolExecutor if you aren't doing so already.

If you really want to fail and retry a task when threadpool is occupied, create the executor with a RejectedExecutionHandler (see the javadoc's "Rejected Tasks" section).

Probably a better option is not to fail any task but let it wait and get executed eventually. Use an unbounded queue for that (see the javadoc's "Queueing" section).
 
Georgios Chatziefstratiou
Ranch Hand
Posts: 106
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Karthik Shiraly wrote:
Georgios Chatziefstratiou wrote:how can i make the new thread notice that this subscriber is busy and fail the task for later send it to db to store it for later update.

Look into ThreadPoolExecutor if you aren't doing so already.

If you really want to fail and retry a task when threadpool is occupied, create the executor with a RejectedExecutionHandler (see the javadoc's "Rejected Tasks" section).

Probably a better option is not to fail any task but let it wait and get executed eventually. Use an unbounded queue for that (see the javadoc's "Queueing" section).

thanks for your reply i have create this code can you give some help.
I want to fail so that i can send it to the db to run later when the running thread is not running anymore.



Karthik Shiraly wrote:
Georgios Chatziefstratiou wrote:how can i make the new thread notice that this subscriber is busy and fail the task for later send it to db to store it for later update.

Look into ThreadPoolExecutor if you aren't doing so already.

If you really want to fail and retry a task when threadpool is occupied, create the executor with a RejectedExecutionHandler (see the javadoc's "Rejected Tasks" section).

Probably a better option is not to fail any task but let it wait and get executed eventually. Use an unbounded queue for that (see the javadoc's "Queueing" section).



is already made like this unbounded queue and if a task takes too long i am out of threads beacuse they are waitng for the one to finish.
thats why i have to change it i also thought to remove the queue but i need it.


thanks
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Georgios,

I'm trying to understand why you prefer a design in which DB operations are failed now (merely because of temporary threadpool exhaustion) and then retried at some future point of time.
This whole thing looks like premature optimization to me, but I might be misunderstanding the problem.
What is the problem you are really worried about that you are trying to solve here?
Why is failing a DB operation ok, but not waiting for it to complete?
 
Georgios Chatziefstratiou
Ranch Hand
Posts: 106
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Have you read the first post? There i explain why.Eventually threadpool will be with all threads in waiting i am trying to avoid that.Because of (merely because of temporary threadpool exhaustion) that is the task i have.

Or to say it better how can i avoid temporary thread pool exhaustion when using spring ThreadPoolTaskExecutor or just java.
The idea is to make choose if i have less threads than i need if i have equal threads than i need
So i made a thread to fail if i have less because i dont want to have it waiting .How can i do it in the code i have create.
I need to know that the current thread has a subscriber and who is the subscriber so that i compare it.
Any help?

thanks giorgo
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I read your first post in detail even before my first reply. You have described what you think is a problem, implemented what you think is a solution, and are now asking a correction for that solution.
But you have not explained why you think it's a problem in the first place.
The closest statement is "I don't want it waiting", but you have not explained why not?

What really is the difference between
1) submitting task to unbounded queue executor and allow it to execute it at some point in future, without you having to code any additional logic
and
2) avoid submitting task now, instead hold it somewhere else (presumably in your own queue implementation) and then execute those rejected tasks at some point in future?

I'm asking these questions because from your code it looks like the "long running tasks" are mongodb writes, which usually not be long at all because mongodb is designed for fast writes.
Even if all threads in pool are busy at some instant, if all they are doing is writing to mongodb, they will finish within few millisecs or seconds.
Your statement "eventually threadpool will be with all threads in waiting" gives the impression that they will wait for ever, but that's not true at all. That's why I called it "temporary exhaustion".

I've used threadpools but have never seen any reason to implement logic that uses core pool checks or stack trace information. From the details so far, I feel you are on the wrong track with this code.
 
Georgios Chatziefstratiou
Ranch Hand
Posts: 106
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Karthik Shiraly wrote:I read your first post in detail even before my first reply. You have described what you think is a problem, implemented what you think is a solution, and are now asking a correction for that solution.
But you have not explained why you think it's a problem in the first place.
The closest statement is "I don't want it waiting", but you have not explained why not?

What really is the difference between
1) submitting task to unbounded queue executor and allow it to execute it at some point in future, without you having to code any additional logic
and
2) avoid submitting task now, instead hold it somewhere else (presumably in your own queue implementation) and then execute those rejected tasks at some point in future?

I'm asking these questions because from your code it looks like the "long running tasks" are mongodb writes, which usually not be long at all because mongodb is designed for fast writes.
Even if all threads in pool are busy at some instant, if all they are doing is writing to mongodb, they will finish within few millisecs or seconds.
Your statement "eventually threadpool will be with all threads in waiting" gives the impression that they will wait for ever, but that's not true at all. That's why I called it "temporary exhaustion".

I've used threadpools but have never seen any reason to implement logic that uses core pool checks or stack trace information. From the details so far, I feel you are on the wrong track with this code.


Hi again the mongo writes are not real writes they are when the update failed( this is my idea save to mongo use nothing else and when everything is ok the retrieve the failed update) due a reason so they go to mongo for a later update.

you mean with Future ?
Karthik Shiraly wrote:
1) submitting task to unbounded queue executor and allow it to execute it at some point in future, without you having to code any additional logic
and


can you explain what you mean
Karthik Shiraly wrote:
2) avoid submitting task now, instead hold it somewhere else (presumably in your own queue implementation) and then execute those rejected tasks at some point in future?



I have also seen that Smart Thread pool Code project
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android C++ Java Linux PHP Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
the mongo writes are not real writes they are when the update failed

Ah, I see now. Sorry for the misunderstanding.

If what you want to do is something like this (in pseudo code):


then one way is to implement it using a fixed thread pool, a bounded queue, and a RejectedExecutionHandler. Basically, it uses the thread pool's feature
of rejecting tasks under certain conditions (such as bounded queue size) and then doing alternate processing for rejected tasks in the rejection handler.


Since I don't know your full application, I can't say if you need to retain all that 100ms polling of
thread pool active count, pool size, map lookup, etc.
 
Georgios Chatziefstratiou
Ranch Hand
Posts: 106
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Karthik Shiraly wrote:
the mongo writes are not real writes they are when the update failed

Ah, I see now. Sorry for the misunderstanding.

If what you want to do is something like this (in pseudo code):


then one way is to implement it using a fixed thread pool, a bounded queue, and a RejectedExecutionHandler. Basically, it uses the thread pool's feature
of rejecting tasks under certain conditions (such as bounded queue size) and then doing alternate processing for rejected tasks in the rejection handler.


Since I don't know your full application, I can't say if you need to retain all that 100ms polling of
thread pool active count, pool size, map lookup, etc.


thanks i will try it and come back i also have something new.
Create a trad pool for every subscriber and use it .

i will try them both and come back.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!