• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Jeanne Boyarsky
  • Liutauras Vilda
Sheriffs:
  • Rob Spoor
  • Bear Bibeault
  • Tim Cooke
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Piet Souris
Bartenders:
  • Frits Walraven
  • Himai Minh

Java multithreaded int manipulation doing right

 
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
the code here is a working prototype of a bigger project but just to show the idea.
i have actions i need to do on int a lot of actions on each int.
i decided that all the actions will be stored in a list and all the ints will be stored in the list
each thread will take 1 action and will run on all the list
but keep getting wrong numbers in the result integerList:

Here is the example code :


And here is the result looks like the numbers been accumulated:

 
Saloon Keeper
Posts: 4497
166
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The problem is in the line (in the method exec):
var  tempList = new ArrayList<Integer>();

Since two Threads are executing this method, each Thread gets its own tempList. I put some println's in the code, and you can see what happens:
input: [1, 2, 3, 4]
in A, parameter 1
tempList: [2]
in A, parameter 2
tempList: [2, 3]
in A, parameter 3
tempList: [2, 3, 4]
in A, parameter 4
tempList: [2, 3, 4, 5]
integerlist: [2, 3, 4, 5]
in B, paramter 1
tempList: [2]
in B, paramter 2
tempList: [2, 4]
in B, paramter 3
tempList: [2, 4, 6]
in B, paramter 4
tempList: [2, 4, 6, 8]
integerlist: [2, 4, 6, 8]
result: [2, 4, 6, 8]

I gave therefore the class ArithmeticManager a field tempList, so that both Threads added to only this List:

and to be sure I synchronized on this List, to prevent possible race conditions. Output:
input: [1, 2, 3, 4]
in A, parameter 1
tempList: [2]
in B, paramter 1
tempList: [2, 2]
in B, paramter 2
tempList: [2, 2, 4]
in B, paramter 3
tempList: [2, 2, 4, 6]
in B, paramter 4
tempList: [2, 2, 4, 6, 8]
integerlist: [2, 2, 4, 6, 8]
in A, parameter 2
tempList: [2, 2, 4, 6, 8, 3]
in A, parameter 3
tempList: [2, 2, 4, 6, 8, 3, 4]
in A, parameter 4
tempList: [2, 2, 4, 6, 8, 3, 4, 5]
integerlist: [2, 2, 4, 6, 8, 3, 4, 5]
result: [2, 2, 4, 6, 8, 3, 4, 5]
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
im getting :



which disappeared after im doing :
private List<Integer> integerList = new Vector<>();
but now i have 2 sync's on the same code what is the right way ?
 
Piet Souris
Saloon Keeper
Posts: 4497
166
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What line is line 44?
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
its the loop
i replaced it with :


And its working now as this error from what i read invoked when i im editing list that as bean looped
But i don't know if its affecting the thread safeness
Thanks
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
also the

as it point to the same data every time
 
Piet Souris
Saloon Keeper
Posts: 4497
166
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I thought that 'for(int i: list) was using an iterator in the background, giving effectively your code, but I am probably wrong about that.

What I could not fault was using this construct:

Problem is the synchronization. That is expensive. I am thinking about your first code, where each thread had its own tempList, and somehow, by using say a CyclicBarrier, put the two templists into the original list. Just a thought.
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
what if i set :

it will synchronized on the list

also i can do :



but i guess it the same as all the above-synchronized methods.
but yeah i still have the problem of the list
 
Piet Souris
Saloon Keeper
Posts: 4497
166
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The problem is still that expensive synchronization. I just thought of a way where synchronization is not necessary, perhaps it is applicable:
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
what is CompletableFuture().completeAsync ?
also I think you implement it with functional programming , where I can't
i must use threads
 
Piet Souris
Saloon Keeper
Posts: 4497
166
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
See the API of a CompletableFuture. If you have a Supplie<T> supplier, rhen you can invoke cf.completeAsync(supplier), and it gets executed on a different Thread. With cf.get(), the Thread that issues this is blocking until that cf is finished. And so, when both cf's have finished and I have my two result lists, then I know I can add the two lists together.

In my example I create a new List result, to collect the two lists. I cannot set the original list to that result, since in order to get the two Suppliers, I have to make the original list final.

The advantage is that both calculations can go full speed, no synchronization needed. You can alternatively define two Callables that do the same with the list, and run these in an Executor. The result is two Futures, and you can handle it in the same way. You can give the operations hard coded in the Runnables.

Anyway, give it some thought.
 
Master Rancher
Posts: 3889
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I read this kind of quickly, so I hope I didn't miss something major.  But it seems to me that the main intent of the original code was in its one public method:

which was basically trying to take that list of actions and apply that list of actions to each element in the list of integers.  Now when you do that, I believe it would be important to apply each operation in order, and not to skip any accidentally.  This is where multithreading can get you in trouble - if two threads are applying two different actions to the same element, the result will depend on what order they are applied - and you can't easily control that.  I think it's a mistake to let different threads handle different actions, for that reason.  Instead, you can write it in a way that each thread handles the full list of actions, in order... but only applies it to a subset of the list of ints.  That is, when you split the problem into different parts for multi-threading, don't split by action -- split by data instead.  The list of ints can be subdivided into different sections, and different threads can work on different sections, and they won't interfere.  Then you just have to figure out how to combine all the results into a single list.  Which may sound complicated... but most of the work has already been done for you by the authors of the Stream API:

Here, by successively applying the map() method, we are giving the stream a list of operations to perform.  These operations will be performed in sequence.  The work of parallelizing this is done by parallelStream(), which will figure out how to split the input data among multiple threads, and toList(), which will (for a parallel stream implementation) figure out how to combine the results of each thread into a single list.
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
@Mike Simmons
Thanks
so if i understand you right you suggest that each thread will execute all the actions on each element of the integerList ? right?
also why is the invokeAll is synchronized ? isn't it aproblem ?
 
Mike Simmons
Master Rancher
Posts: 3889
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Meir Yan wrote:so if i understand you right you suggest that each thread will execute all the actions on each element of the integerList ? right?


Not quite.  Each thread will execute all the actions on some of the elements on the integerList.  Together, the threads will handle all the elements on the list.  But each individual thread will only handle some of them.

Meir Yan wrote:also why is the invokeAll is synchronized ? isn't it aproblem ?


It could be.  But it's also necessary to ensure that the calculations are done correctly.  If we're busy executing one invokeAll() call, and it's not done yet, and another thread calls invokeAll with a different list of operations... we really don't want those new instructions started until we're done with the first set.  Otherwise, we get a lot of confusion, and we can't tell what order things happened in, and the answers become unpredictable.  We're basically saying, let all available threads in the thread pool focus on finishing the work for the first call, before we start another.  Maybe that means some calls will wait.  But we don't want to give wrong answers.  And we're still taking advantage of all available threads to do useful work - we're just focusing on finishing one job before starting another.

If we have 2 calls coming in, and each one individually would take 10 seconds to do the computation, using all available threads... if we make the second call wait, then the first call finishes after 10 seconds, and the second one finishes after 20 seconds.  If instead we find a way to process both simultaneously... then we're using half the available threads on each call, so if everything goes smoothly, the first call will finish after 20 seconds, and the second call will also finish after 20 seconds.   Which isn't any better, really.  And in practice, we need to be able to make guarantees about the order the operations happen in, or we get very different answers.
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the replay , how would you suggest to divide the integer list  to groups of elements, that will be worked by the actions?  
Although what i did in the end is :
to add new :

and add the elements to this results array





yet not sure if its the right sulotion .. as it works
 
Mike Simmons
Master Rancher
Posts: 3889
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Meir Yan wrote:Thanks for the replay , how would you suggest to divide the integer list  to groups of elements, that will be worked by the actions?


The code I showed already does that, behind the scenes.

 
Regarding your new code... well, I'm skeptical.  Have you tried it with your public invokerActions() code?  Do you get consistent results?

The idea here is if we have a list of 1000 numbers (0-999), and then apply 10 n+1 operations, and apply 10 n*2 operations, are the results consistent?  I suspect they will not be...
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for your great explanation.
I'm still looking into the stream method as an alternative method but from your explanation, its looks really much more simple then using threads.
but what about threads timeouts? what about a thread pool to control how many threads i can execute?
does stream support this?


 
Mike Simmons
Master Rancher
Posts: 3889
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Meir Yan wrote:but what about threads timeouts? what about a thread pool to control how many threads i can execute?
does stream support this?


Good questions.  As far as I recall, no it does not support this.  The parallelStream() method basically gives you a stream that relies on the standard Java fork/join pool, which I believe is not really configurable in the ways you're asking about.

If you do need more control over the threads, you may need to code it yourself.  I would still follow the basic strategy I outlined: Don't give different threads different tasks.  Give them the same list of tasks, but give them each a different set of ints to operate on.  If you have five threads and a list of numbers 1-1000, then let thread 1 process numbers 1-2000, let thread 2 process numbers 2001-4000, let thread 3 process 4001-6000, thread 4 6001-8000, and thread 5 8001-10000.  Then when they're all done, let one thread concatenate the five lists into one big list.  Maybe that's actually six threads in that example... but it's something like that.
 
Mike Simmons
Master Rancher
Posts: 3889
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There are some tricks that give you some more control of the thread pool, at least the number of threads.  Check out Custom Thread Pools In Java 8 Parallel Streams.


For doing a timeout, here's a quick idea I had that seems to work:

You can make a custom exception to catch, rather than a generic RuntimeException.  But it has to extend RuntimeException, or otherwise be an unchecked exception, in order to call it from within the peek() method.
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for your detailed answer .
what i did in the end ( in the thread version ) i took you idea of splitting the data to ranges for example :
if i have 5 threads i took the integerList
and did :


This way i divide the integerList to groups by range and sending to the worker thread this way i allso dont need to lock the return list as i know the exsact indexs :


by the way im using :

as it gives me timeout and thread pool out of the box .
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
i implemented your solution but the problem is that it is executing
the 2 actions on the string and it concatenate the string and not creating new element in the return list.
what i need is that the new list that is returned will contain the new elements
by the way the toList() not exist anymore in java 8
i had to do it with :



 
Mike Simmons
Master Rancher
Posts: 3889
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The toList() method is actually added in Java 16.  But yes, collect(Collectors.toList()) is the easiest way to achieve pre Java 16.
 
ben josh
Ranch Hand
Posts: 620
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks, i still face a problem as i try to implement your solution with small fix but the output is wrong see this :
if you can help here this will be great



currently, if you run this program you will see that the results are connected to the string.

 


what i need that each element will be in its own element e.g :

   
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic