How can I understand what actually a stream is?

[Monica Shiralkar]

I read the below about streams :

1) Streams are used to allow Java to do functional programming. For e.g convert list go stream and then apply filter.

2) Streams is data executed in sequence

3) Streams cannot be operated upon more than once .

But based on this how do I understand what a stream is and such a thing that it cannot be run more than once ?

Thanks .

[Mike Simmons]

Monika Shiralkar wrote:1) Streams are used to allow Java to do functional programming. For e.g convert list go stream and then apply filter.

Well, I would say that streams are the place, in the libraries, where they put all the functional methods for processing groups of data, like collections.

Monika Shiralkar wrote:2) Streams is data executed in sequence

That's the most basic definition, I think.  A sequence of data, combined with a rich API of methods to help you do the processing.

Monika Shiralkar wrote:3) Streams cannot be operated upon more than once .

This was an intentional design choice to encourage users to structure their processing in a way that does not require all the data to be held in memory at once.  Without streams, people typically use Lists and other collections for everything.  And that's usually fine... you loop through a list, then you loop through it again to do something else, and maybe once more for something else.  In order to do that, the list keeps references to everything in the list, to prevent it from being garbage collected.  But, some data is too big to comfortably fit it all in memory at once.  Or even if you can, it uses more memory than you really need to.  It's often much more efficient to structure your processing to do everything in one loop, if possible.  So Stream does that, empowering you to do all sorts of things concisely and efficiently... but all in one loop.

When I say loop here, you don't see the one loop as a loop, in your code.  But it's there, behind the scenes.  If you pay attention to the idea of a terminal operation on a stream... whenever you invoke a terminal operation on a stream, you call code that executes the loop.  All the operations you've defined so far in lambdas and method references... those will finally be called, all behind the one big loop, once you've invoked a terminal operation.  Once you do that, there's no more Stream for you to operate on.  You're done looping.

Another way to think of this is, a Stream is basically a modern, enhanced version of an Iterator.  Not an Iterable, which can be looped through repeatedly by calling iterator() multiple times... but a single simple Iterator, which lets you see everything in the sequence, just once.  A Stream is like that... but it has a bunch of other methods added, to make your processing more concise and efficient.

Note that in many cases, if you really need to, you can process everything more than once, by making sure there's a List or other Collection being used to save everything in memory.  Either you start with one, or you save your intermediate results midway through your processing using Collectors.asList() or something similar.  That's always possible.  But you have to choose to do that, rather than be forced to do that.  Streams encourage you to do everything in one loop, and if you can't, then save your results to a List, and loop as many more times as needed.

[Pierre-Yves Saumont]

A stream is actually a generator. It generates a series of data. For this, it can use:

- a collection. Each step will generate the next element By reading it from the collection until there is no more element.

- a function of the position in the stream. For example, a random number generator. Generally, however, the generator is not a seen as a function. Instead, it memorizes its state, so it does not actually take the position as an argument.

- a function of the previous element. In such a case, it needs a seed In addition to the function.

Java streams often use a collection to generate data, which is why many programmers see them as “lazy” collections.

Laziness is essential to streams, which allows composing operations without executing them. It also allows working with infinite streams.

Basically, streams are not different from loopa generating values. It is in fact the functional equivalent of a loop generating a value on each iteration. The main benefit is that there is an agreed upon idiom to compose streams with operations, instead of manually composing loops an operations.

Saying that streams can only be used once makes no more sense than saying that a loop can only be executed once.

Java streams add parallelism, but this is not something specific to streams. Or maybe, they were called “streams“ in order to allow for this specificity. The correct name is “sequence”.

[fred rosenberger]

i'll admit i have not used streams muhc...ok...at all.

But my interpretation is that they are much like piping together commands on a unix shell.  the output of each stage is passed as the input to the next...something like

ps -ef | grep MatchProcess |  cut -b 1-20  > myFile.txt

the output of the ps command is fed into grep, and that output is fed to cut, which is then written to a file.  there are several stages where the data is ephemeral as it works its way through, until at the end it's written to the file.  

Please correct me if i'm wrong.

[Tim Holloway]

The original java.io.Stream is an abstraction referring to a source or sink of serial data. As an abstraction, the format of the data could be anything: characters, binary data, java Serialized objects, XML, and on and on. The only constraint was that the data be 1-dimensional. No pushing stuff around in parallel bits or bytes.

There is such a concept as a seekable Stream. Some seekable Streams can be re-wound to the beginning and re-read, some can be forward- and/or backward-spaced. The streams employed in Lambdas, however are not seekable. You get one pass and one pass only over the stream contents.

[Ron McLeod]

I visualize a stream as being like a conveyor belt or production line, where you put stuff in at one end and get it out at the other end, and as the stuff moves along the way it can be processed, filtered, rearranged, etc.

Something like a bakery where a production line is supplied with raw ingredients, which are then mixed in to dough, formed in to bread loaves, baked in an oven, then the baked loaves are inspected and odd shaped loaves are removed, sorted according to size, put in packaging, and then made available to consumers.

[Monica Shiralkar]

Is it enough to know if all I understand is that Stream is something that is to give Java the ability to use lambdas on a collection ?

[Mike Simmons]

That's most of what you need, yes. It's also good to know about the limitation that you can only call one terminal operation on a stream - but then there are ways to work around that, either by reorganizing your processing, or by calling stream() again on a collection, to get a new stream.

[Campbell Ritchie]

P-YS: good to see you back
Fred, I agree with your example with pipes. But Streams have additional capabilities beyond that. They can filter the data and remove elements, or even sort elements as they go.