• Post Reply Bookmark Topic Watch Topic
  • New Topic

Stream

 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why wouldn't you always use parallelStream?
 
Stephan van Hulst
Bartender
Posts: 6583
84
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Because for few elements, parallelization may add more overhead than it's worth. In general, it's only useful for large data sets.
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:Because for few elements, parallelization may add more overhead than it's worth. In general, it's only useful for large data sets.


Define "large data sets".
 
Stephan van Hulst
Bartender
Posts: 6583
84
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Large enough that parallelization is useful.

If you need a number, you need to benchmark.
 
Campbell Ritchie
Marshal
Posts: 52565
119
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The benefits of parallelisation may outweigh the overhead if the tasks are very time‑consuming. There is a chapter about that question in Maurice Naftalin's book.
 
Tim Cooke
Sheriff
Posts: 3294
153
Clojure IntelliJ IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Large enough that running in parallel is consistently faster on your target hardware and dataset. If speed is important to you for that operation then testing and profiling is essential for you to determine which is best.
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tim Cooke wrote:Large enough that running in parallel is consistently faster on your target hardware and dataset. If speed is important to you for that operation then testing and profiling is essential for you to determine which is best.

And so you are supposed to test on the production server? And what happens when the configuration changes as upgrades or whatever happen? Are you supposed to go back and check again.
It all seems so unscientific.
 
Pierre-Yves Saumont
Author
Ranch Hand
Posts: 96
17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tim Bee wrote:Why wouldn't you always use parallelStream?


Because we made benchmarks and found that:

1) On an idling machine, there was no gain past the first half of the available physical threads. Parallel streams use as many threads as there are physical threads available on the machine. (It is possible to change this, but it becomes a bit complicated. This is how we tested with a lower number of threads.)

2) Our applications run on 8 core machines with lot of processes running, among which from 8 to 12 Java servers. (Nothing close to an otherwise idling machine.)

3) Each of these servers is by nature already parallelized, serving a huge number of requests in parallel.

In the end, we remarked that parallel streams generally bring a performance increase in development environment, and a decrease in production. This makes them a good marketing tool, but nothing more. This is however only our experience.

When we really need parallel processing for a specific task, we use actors.
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Pierre-Yves Saumont wrote:
Tim Bee wrote:Why wouldn't you always use parallelStream?


Because we made benchmarks and found that:

1) On an idling machine, there was no gain past the first half of the available physical threads. Parallel streams use as many threads as there are physical threads available on the machine. (It is possible to change this, but it becomes a bit complicated. This is how we tested with a lower number of threads.)

2) Our applications run on 8 core machines with lot of processes running, among which from 8 to 12 Java servers. (Nothing close to an otherwise idling machine.)

3) Each of these servers is by nature already parallelized, serving a huge number of requests in parallel.

In the end, we remarked that parallel streams generally bring a performance increase in development environment, and a decrease in production. This makes them a good marketing tool, but nothing more. This is however only our experience.

When we really need parallel processing for a specific task, we use actors.


But if you were in a job interview and didn't parrot the company line, they would laugh at you.
 
Pierre-Yves Saumont
Author
Ranch Hand
Posts: 96
17
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tim Bee wrote:But if you were in a job interview and didn't parrot the company line, they would laugh at you.


So what? What are you trying to say? Do you really think yo have to parrot the company line so that the interviewer doesn't laugh at you? From my experience, this is totally wrong. Interviewers are not stupid. At least not the majority of them. If an interviewer laughs at you because you don't parrot the company line, just go away. You wouldn't want to work in such a company, would you?
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So what is the main benefit of streams over iteration? Is it the functions that can be called through Lambda statements or do iterations take up space in memory for each element?
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Pierre-Yves Saumont wrote:
Tim Bee wrote:But if you were in a job interview and didn't parrot the company line, they would laugh at you.


So what? What are you trying to say? Do you really think yo have to parrot the company line so that the interviewer doesn't laugh at you? From my experience, this is totally wrong. Interviewers are not stupid. At least not the majority of them. If an interviewer laughs at you because you don't parrot the company line, just go away. You wouldn't want to work in such a company, would you?

As a consultant I can tell you they all do it.
 
Campbell Ritchie
Marshal
Posts: 52565
119
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The first benefit is in writing simple code which you can read, rather than lots of boilerplate in a loop.
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:The first benefit is in writing simple code which you can read, rather than lots of boilerplate in a loop.


I find loops easier to read.
 
Stephan van Hulst
Bartender
Posts: 6583
84
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tim Bee wrote:So what is the main benefit of streams over iteration? Is it the functions that can be called through Lambda statements or do iterations take up space in memory for each element?

The main benefit of functional code is that it more clearly expresses what you want to do, not how you want to do it.

Let's say you have a list of countries, from which you want to retrieve the European capitals:

Using procedural code:

The first code example describes exactly what you want to achieve. The second example is more concerned with the technical details: instantiating an array list, adding items to the list, etc.

If you find the second example easier to read, it's probably because you're more used to procedural languages.
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:
Tim Bee wrote:So what is the main benefit of streams over iteration? Is it the functions that can be called through Lambda statements or do iterations take up space in memory for each element?

If you find the second example easier to read, it's probably because you're more used to procedural languages.


You mean like Java?
 
Stephan van Hulst
Bartender
Posts: 6583
84
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
With the introduction of streams, Java now has functional features. Java is not purely procedural.
 
Tim Bee
Ranch Hand
Posts: 91
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:With the introduction of streams, Java now has functional features. Java is not purely procedural.


Well, it allows a lot of procedural coding.
 
Winston Gutkowski
Bartender
Posts: 10571
64
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tim Bee wrote:Well, it allows a lot of procedural coding.

Arguably too much.

The reason I like Streams is that they remind me of script programming - take a stream, bang it through a few "additive" operations, and hey presto, you have what you want at the other end. I wrote an entire menu system based on it.

Procedure is fine - and necessary in some situations - but you don't need to be a slave to it. Streams and lambdas are just another string to the programming bow.

Winston
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!