• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Subtle Question About Type Parameter in Stream's .map method

 
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Regarding:
<R> Stream<R> map​(Function<? super T,​? extends R> mapper)

I am getting way more confident on non-trivial generics usage, but this one stumped me.

In the Sybex 816 book by Jeanne and Scott, they say that the type of the Stream is determined by the lambda passed in.

I will take that as obviously meaning the lambda, method reference, or class implementing the required interface passed in, and not be pedantic there.

I also agree that R always matches ? extends R for all R, so I see how it is true....

But what I don't see is how it would ever infer an R different from the return type of that lambda, method reference or method of the class instance passed in, unless one used the rarely seen "type witness".

That is, when will the type parameter of the Stream returned ever be a super-class of that return type?
I think if we invoked map with a <TypeWitness> that would happen...but otherwise, won't <R> always to be taken to be exactly the exact type that the function returns?

It's a subtle point and I can live without knowing this, but I am trying to go from pretty good Generics and type inference knowledge to ... better than that I guess?
 
Marshal
Posts: 79177
377
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ref: Ken Kousen, Modern Java Recipes, Sebastopol CA: O'Reilly (2017), pages 279‑280.

As Joshua Bloch tells you in Efffective Java (not 1st edition), PECS. Producer extends and Consumer super.
Kousen says, “another simple example.” We already know about the T, which is defined by the Stream's declaration. The method adds an R type parameter, and I think (not certain) R is short for result. It produced a Stream<R> as its result. The R can usually be inferred from the type of the Function. As we know, Function takes two types, T which is the type of the argument to its functional method and R which is that method's return type. That method consumes its argument, which is why it is defined as, “unknown extends T,” and produces its returns value, so that is declared as, “unknown super R,”.
If you have an extensible type, it would be feasible for the method to accept an instance of its supertype as a parameter. It would also be possible for the method to return a subtype of the returned type, and in both cases still satisfy an IS‑A relationship. To quote Kousen as above:

. . . the input variable could have been treated as a method from Object rather than Employee, because of the super wildcard. The output type could, in principle, have been a List containing subclasses of String , but String is final so there aren’t any.

Here Kousen is using a Function<Object, String> to demonstrate the super keyword and exploit the fact that an Employee IS‑AN Object.
 
Saloon Keeper
Posts: 15510
363
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
While your answer is correct Campbell, it is the answer to a different question than the one that Jesse is asking.

Generic type arguments are inferred from assignment context as well as from functional interface implementations. For instance, let's say we have the following statement:

In the code above, R is inferred as Employee, so Student::getFavoriteTeacher returns a Function<Student, Employee>, even though the return type of getFavoriteTeacher() is Teacher, and not Employee. The method handle returns a function with a type that has adapted to the generic type argument that was inferred from the assignment context.

Is there a point to declaring the mapper parameter of the map() method as a Function<? super T, ? extends R> instead of just Function<? super T, R> when the return type of a method handle or lambda expression will always adapt itself to the inferred type argument for R anyway? If map() had declared its mapper parameter without using a wildcard for the upper type bound, the code above would have worked as well, right?

Well yes, but...

Higher order functions don't just take lambda expressions and method handles. They also take expressions that have a fixed type:

This code works because map() declared the mapper function with a wildcard on the upper type bound. If mapper was declared as Function<? super T, R>, the compiler would not be able to infer a type for R, because the type constraint imposed by the assignment context is incompatible with the type constraint imposed by the method argument.
 
Campbell Ritchie
Marshal
Posts: 79177
377
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I suppose it is better to give the right answer to the wrong question than the wrong answer to the right question.

Not the first time I have made that mistake (sorry) and I am sure it won't be the last time.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This is the second time in a couple of days that I am saying, "Whoa--I learned important stuff from both your answers, tho the second one was spot on what I was after."

I would say this is all very subtle and complicated, but I am a refugee from C++, from which I can cite the trivia of having FIVE DIFFERENT CATEGORIES OF TYPE INFERENCE THERE, with very different rules in some places compared to the others, and almost but not quite the same in some pairs of places.

I presume the JLS yields its secrets of type inference up if stared at long and hard enough.

For today, the take-home lesson to not forget is that BOTH SIDES are considered in inferring the types!!

The full details are probably helpful to know, and the example really drives home and fully illustrates the point, but the magical sentence here is indeed:

Generic type arguments are inferred from assignment context as well as from functional interface implementations.



It is interesting that both inferring right side from left side, and left side from right side works in different places in Java at this point.

In C++, since 2011 we have had:
auto typeInferredLocal = ValueFromWhichTypeIsInferred;

as we now have in Java 10+ with var.

Of course, since Java 7, we have had the single case that was most screamed for of the diamond operator, <> inferring, well, flat-out just lifting, the type parameters on the right side from those on the left side.

Of course, the two interact if one decides to write:
var myList = new ArrayList<>();
which drags us all the way back to an inferred type of <Object> for both sides.

In Stephan's example there is no LVTI, but he does show the answer to exactly precisely the answer to what I was wondering about, at least one great example of it, there may be others...

None of the examples of functional programming in Java (EDIT--THAT I HAD SEEN!!) had done something like:
static final Function<Student, Teacher> GET_FAVORITE_TEACHER = Student::getFavoriteTeacher;

But I am used to pretty similar idioms in other languages.  I'm still marveling at how clearly it shows an example of what I was failing to imagine.

There is a real difference here between Java and C++.  I'm not 100% on top of Java's type inference as I type this line, but I can clearly see that I will be with a little more work.

I can literally question whether ANYONE is fully on top of it in C++, including people who professionally lecture on it.  It is that complex.
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ha, I have an admission to make. I know Java like I know my back pocket, and if something's unclear I can usually find it in the JLS pretty quickly; but I have no idea how Java's type inference system works exactly.

One time I tried to find out. I really did. I think I wrote a huge post on this website with my findings. But I never managed to get to the end, that precious "click" moment. It's really one of those areas where I rely on my tools to tell me what I did wrong.
 
Campbell Ritchie
Marshal
Posts: 79177
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have tried implementing type inference myself, and it was by no means difficult. So I presumed that Java® type inference would be easy to understand and find out about.
Is there anything in the JVMS about type inference? [Ignore that question; if it is done at compile‑time, it won't be in the JVMS.]
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Did your type inference system account for generic higher order functions that accept generic functions that have wildcard bounds on their type parameters, Campbell?
 
Campbell Ritchie
Marshal
Posts: 79177
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
No, it was a lot simpler than that.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I took a quick gander at Chapter 18, Type Inference, of the Java SE 16 JLS.

* * * shudders * * *

I won't be going back there anytime soon, it is way out of scope for my current Java goals.

Suffice it to say, it is indeed somewhat more complex than I had expected, but still far, far simpler than the PhD-worthy treatise to be found in authoritative and comprehensive description of type inference in C++.

I've learned a lot even in this very thread, and have already used it in interview discussions, where it demonstrated an attention-to-detail that would make me either a very desirable or undesirable candidate depending on team preferences.

Thanks again to both of y'all!
 
Campbell Ritchie
Marshal
Posts: 79177
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jesse Silverman wrote:. . . Thanks . . .

That's a pleasure
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Campbell Ritchie wrote:The method adds an R type parameter, and I think (not certain) R is short for result. It produced a Stream<R> as its result. The R can usually be inferred from the type of the Function. As we know, Function takes two types, T which is the type of the argument to its functional method and R which is that method's return type. That method consumes its argument, which is why it is defined as, “unknown extends T,” and produces its returns value, so that is declared as, “unknown super R,”.



Wait, this wasn't exactly what I was asking, but does the last sentence there contain a "Strike That, Reverse That!" typo??
 
Campbell Ritchie
Marshal
Posts: 79177
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jesse Silverman wrote:. . . a "Strike That, Reverse That!" typo??

Yes, it does; well done finding my daft mistake And sorry.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A quick Acronym Note:
PECS confused me at first, because I was thinking in terms of Consumer and Supplier

There is no ambiguity with Consumer, it consumes whatever you give it and gives nothing back, like the worst dog in the world.

When first coming to Java's Single Abstract Method / Functional Interfaces, I kept calling Supplier Producer, probably due to exposure to terminology over the previous decades.

So Consumer is both the name of the specific interface (along with BiConsumer, IntConsumer, DoubleConsumer, LongConsumer and friends) and the general concept.

A Supplier takes NOTHING and gives you what you need or want.

(I just checked the JLS, it contains exactly two references to Consumer with a capital C, zero to Supplier)...

Producers, which I would prefer to see with a lowercase p, probably, because the name isn't part of the java.util.function namespace, include actual Supplier instances, as well as every different kind of Function imaginable.  This gets interesting when attempting to use constructors in method references, Supplier assignments get the no-args version of the same name, but if you assign it to the right functional interface you get to use the exact same name to  call a constructor with appropriate args...

Still working out some of the implications of that stuff, but the basics are when I saw PECS the first few times, my eyes kept seeing the S for Supplier and getting confused.
In my defense, the C actually stands for a functional interface type, the P does not, but as this discussion has shown, the PECS principle goes way beyond streams, way beyond functional interface use.

When discussing the general topic, and how differently languages as ostensibly similar as Java and C# approach some things, the physics terms contra-variance and co-variance often got used, which everyone not from a physics background hated.  People working solely (or vastly predominantly) in just one of them probably don't realize how many fundamental design decisions there are regarding contravariance and covariance in a typed language, I suspect.
PECS in larger (C#/Java/Scala) context:
https://www.codeproject.com/Articles/899319/Comparing-covariance-contravariance-rules

Add in two or three more strongly-typed languages and your brain will melt...

I'm focused on Only Java now, and I think I won't get confused about PECS until the next time I jump back and forth between languages...

Thanks!
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I've used PECS only very briefly. After a while it becomes second nature, much like how you don't have to resort to silly aphorisms to know which side your "left" hand is on.

Any time I need a reminder, I usually think of Comparator in Java. To me, Comparator is the classic consumer interface, and it's the first interface I was exposed to that is invariably used with the super keyword.

I think the designers of C# did well to use the "in" and "out" keywords to specify type bounds, rather than "super" and "extends". They seem to have realized it's much more intuitive that way.

C# messed up in another way though: C# uses "declaration site variance", and doesn't support "use site variance". This means that the variance is fixed per type: you can't declare that your method requires an ICollection<out Foo> to make use of covariance. ICollection is always invariant.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Interesting.  Any thoughts on Scala's approach, which I will only read briefly now, but would come back to if I am starting to work with Scala?
I have had interviews for Java/Scala positions (as well as Java/C# and Java/C++ and Java/C++/Python) but haven't landed any.
I was not required to know Scala at hiring, but would have been required to have been excited about learning it...
 
Campbell Ritchie
Marshal
Posts: 79177
377
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jesse Silverman wrote:. . .  Consumer, it consumes whatever you give it and gives nothing back, like the worst dog in the world. . . .

Many objects or λs act as consumer and producer simultaneously. In the Object::toString example I showed you from Ken Kousen's book, the parameters to the method “consume” the input and the method “produces” a return value. That means that Producers might produce things and Consumers might consume things, but a Function does both.
Remember that the acronym PECS pre‑dates the Streams API. I don't have my Effective Java to hand, but I think Bloch refers to an earlier but similar concept, called something like “put and get”.
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I can't make comments on Scala because I haven't really worked with it.

I'm not excited about learning languages that improve on a parent language without introducing some radical new concepts. Scala seems decent enough, but I won't be spending my free time on it until I need it for my professional life.

Kotlin is worse. In my (admittedly limited) experience, it's just a poorly documented clone of Java that doesn't really commit to OO and also doesn't really commit to functional programming, but is a half-hearted mix of both, with some terrible new features bolted on top.

Currently my favorite language is C#. It took Java, and REALLY improved on it. The only reason I don't use it more in hobby projects is because Microsoft is still bad at writing tools and documentation, and every time I try to invest time in learning the nasty details of MSBuild and NuGet, I just end up frustrated because Maven is SO much better.

I think the top candidate for my next language to learn is Rust. It seems to offer some exciting features that I haven't encountered in other languages yet.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:I can't make comments on Scala because I haven't really worked with it.

I'm not excited about learning languages that improve on a parent language without introducing some radical new concepts. Scala seems decent enough, but I won't be spending my free time on it until I need it for my professional life.


Same here.  Bruce Eckel's book "Atomic Scala" was 1300 pages and still growing last I checked, it doesn't seem to be a minor investment!

Stephan van Hulst wrote:
Kotlin is worse. In my (admittedly limited) experience, it's just a poorly documented clone of Java that doesn't really commit to OO and also doesn't really commit to functional programming, but is a half-hearted mix of both, with some terrible new features bolted on top.


It does seem to be terribly popular, but with both Google and JetBrains pushing it hard on Android that doesn't by itself imply it is very good, just that it is NOT JAVA.

Stephan van Hulst wrote:
Currently my favorite language is C#. It took Java, and REALLY improved on it. The only reason I don't use it more in hobby projects is because Microsoft is still bad at writing tools and documentation, and every time I try to invest time in learning the nasty details of MSBuild and NuGet, I just end up frustrated because Maven is SO much better.


Yeah, but an ever-increasing percentage of their docs are open-sourced, I think I have almost a dozen docs corrections for C#/.Net/PowerShell, versus 0 for Javadocs which I actually use more.  A few Javadocs errors I have noticed have remained open with "Yeah, that's bad, someone should fix this" up to 15+ years!!  Some of my rendering complaints are waiting for new docs-processing toolchains to go into production, but the others have all been addressed.  Python docs are also easier to contribute to, tho I have gotten some "Yeah, we aren't going to bother to do that" for fixes they considered too trivial.

Stephan van Hulst wrote:I think the top candidate for my next language to learn is Rust. It seems to offer some exciting features that I haven't encountered in other languages yet.


I have been hearing that Rust itself is fantastic, the main information sources for *learning* it, much less so.  Once you navigate the sub-standard learning process, it is great.
 
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jesse Silverman wrote:

Stephan van Hulst wrote:
Currently my favorite language is C#. It took Java, and REALLY improved on it. The only reason I don't use it more in hobby projects is because Microsoft is still bad at writing tools and documentation, and every time I try to invest time in learning the nasty details of MSBuild and NuGet, I just end up frustrated because Maven is SO much better.


Yeah, but an ever-increasing percentage of their docs are open-sourced, I think I have almost a dozen docs corrections for C#/.Net/PowerShell, versus 0 for Javadocs which I actually use more.  A few Javadocs errors I have noticed have remained open with "Yeah, that's bad, someone should fix this" up to 15+ years!!  Some of my rendering complaints are waiting for new docs-processing toolchains to go into production, but the others have all been addressed.  Python docs are also easier to contribute to, tho I have gotten some "Yeah, we aren't going to bother to do that" for fixes they considered too trivial.


OpenJDK is open source, and it's on GitHub these days: https://github.com/openjdk/jdk/. However, the process is still sluggish. Before Oracle even considers a pull request, you have to go through this entire slow procedure. I just don't think it's worth it anymore. I have a local branch that adds the getChars method from String / StringBuilder to CharSequence so that instanceof checks like this one will no longer be necessary. That code is probably never going to make its way into the JDK because of the cumbersome procedure involved.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rob Spoor wrote:...
OpenJDK is open source, and it's on GitHub these days: https://github.com/openjdk/jdk/. However, the process is still sluggish. Before Oracle even considers a pull request, you have to go through this entire slow procedure. I just don't think it's worth it anymore. I have a local branch that adds the getChars method from String / StringBuilder to CharSequence so that instanceof checks like this one will no longer be necessary. That code is probably never going to make its way into the JDK because of the cumbersome procedure involved.



Is this another thing that might be submitted to OpenJDK if it weren't such a hassle?
https://robtimus.github.io/io-functions/

It feels like that is dealing with one of the most common instances of an area that seems to Still Suck:
functional programming style in the face of exceptions
 
Rob Spoor
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Knowing Oracle, that example is too specific to be included in OpenJDK, especially now that it's already in a library.

What would be great is allowing checked exceptions to escape lambdas. That is very difficult to implement though.

The following seems possible enough:

Now comes the difficult part. Image that such a lambda is not used in a stream call, but passed to a method:
How does the method that invokes the function (someMethod1) know what exceptions the function can throw? And what if the function isn't invoked directly but instead stored in an instance field, and called at a later time? The compiler and runtime would need to know when a lambda is invoked, and only allow its checked exceptions to escape in specific cases.
 
Jesse Silverman
Bartender
Posts: 1737
63
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:...

Currently my favorite language is C#. It took Java, and REALLY improved on it. The only reason I don't use it more in hobby projects is because Microsoft is still bad at writing tools and documentation, and every time I try to invest time in learning the nasty details of MSBuild and NuGet, I just end up frustrated because Maven is SO much better.



19 styles of formatting for code blocks on here and C# is none of them.
How difficult would it be to add support for C# syntax?

Neither sarcastic nor rhetorical -- would that be easy or difficult?
 
An elephant? An actual elephant. Into the apartment. How is the floor still here. Hold this tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic