• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Can code running from CLASSPATH/in-the-unnamed-module access code that is only on the module path?

 
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am seeing directly conflicting information about whether in transitional projects using both --class-path and --module-path, code in the unnamed module (i.e. loaded from CLASSPATH) can actually access code that is on the module path.

The Sybex 816 book clearly says that they don't intend to tell you everything you need to know to convert a real world project to modularized form in the book, but only to know what is in the exam.  They recommend one great book on Java Modules (I know of two others).

I would be reading one or more of those before I endeavored to convert a Real World project to modular form, but even just for the exam, a clear notion of whether or not code running in the Unnamed Module (i.e. from classpath) can read code that is solely on the module path (or not) seems to be important to know.  It is on this point that I am seeing conflicting information in different sources that I would normally trust individually, were they not disagreeing.

If the answer is different for directly reading versus via. reflection somehow, I will take note of that (despite the fact that reflection is considered outside the scope of the 819).

Anyone who can help clarify this rather specific point will be be greatly appreciated.
 
Jesse Silverman
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The first experiment I came up with seems to indicate that those who say that "Stuff on the classpath can read all modules" are just dead wrong:

PS F:\Java\ModulesTests> javac --module-path mods/ -d fromClassPath puppy/puppy/adorable/CutePuppy.java
puppy\puppy\adorable\CutePuppy.java:2: error: package animal.behavior is not visible
import animal.behavior.Behave;
            ^
 (package animal.behavior is declared in module animal, which is not in the module graph)
1 error


I can't even compile the non-modular version of CutePuppy.java with everything on the module path that is needed...

Maybe they are confused because an automatic module, which is most decidedly running from the --module-path can do this?

Or I am very confused.

But it looks like those who say "non-modular stuff running from classpath can't see anything by virtue of it being on your --module-path, it can't see that at all, it is running like it only knows about classpath" are the ones who are correct.

PS F:\Java\ModulesTests> javac --class-path 'mods/animal.jar;.' -d fromClassPath puppy/puppy/adorable/CutePuppy.java
PS F:\Java\ModulesTests> java --class-path 'mods/animal.jar;fromClassPath' puppy.adorable/CutePuppy
Puppy being cute!
Oh, Behave!
Puppy being bad!
Oh, Behave!


A named module running from the classpath isn't a module at all.  It is something that could have been a module, and will be, whenever it is placed on the module-path, but otherwise is just a .jar that happens to have an ignored module-info.class file at its root.

Just having the dependency on the --module-path it totally useless:
PS F:\Java\ModulesTests> java --module-path 'mods/animal.jar;fromClassPath' puppy.adorable/CutePuppy
Error: Could not find or load main class puppy.adorable.CutePuppy
Caused by: java.lang.ClassNotFoundException: puppy.adorable.CutePuppy


I feel like people saying that code running from the --class-path is able to see modules on the --module-path are being confused by the fact that pointlessly and uselessly putting those module dependencies on the --module-path while useless and pointless doesn't stop you from running:

PS F:\Java\ModulesTests> java --module-path 'mods/animal.jar;fromClassPath' --class-path 'mods/animal.jar;fromClassPath' puppy.adorable/CutePuppy
Puppy being cute!
Oh, Behave!
Puppy being bad!
Oh, Behave!


but it is useless and pointless.  Unless you place the "jars which could be used as modules if placed on the module-path, but otherwise are just jars belonging to the unnamed module" on the --class-path, code running from the class-path can't get to them.  If you do, they aren't being run as named or automatic modules, but as legacy jars in the 'unnamed module'.

Maybe I am still confused?



 
Saloon Keeper
Posts: 13256
292
  • Likes 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hold on.

There is a difference between something being on the module path, and something being in the readability graph. You can put everything you want on the module path, but that doesn't automatically add it to the readability graph.

Again please refer to the documentation of the java.lang.module package, it's pretty authoritative.

Before the readability graph is constructed, all the nodes of the graph (i.e. all modules) must already be known. In the documentation I linked to, enumerating all known modules is described under "Step 1: Recursive enumeration". In short, it looks at all module descriptors of a set of starting modules, and recursively enumerates their dependencies.

The --module-path switch only tells the compiler where to find modules when it is looking for one, it doesn't tell the compiler which modules to look for. In the vocabulary of the documentation I linked to, --module-path makes modules "observable", but it doesn't enumerate them.

Only in "Step 2: Computing the readability graph" is it determined which module can read which other module. This step only works with modules enumerated in step 1. If your module has been enumerated in step 1, the unnamed module will be able to read it, because the unnamed module reads ALL other modules.

The crux of the problem is that your module is not being enumerated. The set of root modules is usually java.se + the module you are trying to compile or run. You are not compiling or running the animal module and it is not declared as a dependency in the module descriptor of any other module that you are compiling or running.

In short, your module will only get enumerated automatically if it is the starting point of compilation/execution, or a (transitive) dependency of another module that has already been enumerated. That means that for applications where the entry point of compilation/execution is not on the module path but on the class path, you must explicitly enumerate your module using a command line option:

If you are certain that you want to enumerate all modules that are on the module path, you can alternatively use the following command:
 
Stephan van Hulst
Saloon Keeper
Posts: 13256
292
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There's also a good case to use --add-modules when your entry point is a named module on the module path, and not the unnamed module on the class path.

Let's say your application supports plugins. Since plugins are not required directly by any of the modules that the main application consists of, they won't be enumerated automatically when the module graph is constructed. To make sure that they are added to the module graph, you must enable your plugins using --add-modules.
 
Jesse Silverman
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I may be a fervent advocate of RTFJD in general (in fact I am) but I didn't realize *that* page existed, clarifying (with room for implementation differences) so much that is murkily alluded to elsewhere.

I am still processing some of the ramifications coming out of what's on that page, but at least I can read and re-read that page iteratively while I have unresolved questions.

So, thanks!
 
Jesse Silverman
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I seriously already learned a lot from that link, and it answered a question or three which were bugging me that I hadn't even posted, and while there is still more for me to learn from that page I see you are correct (of course):

PS F:\Java\ModulesTests> javac --module-path mods/ --add-modules animal -d fromClassPath puppy/puppy/adorable/CutePuppy.java

(the sweet sound of zero errors)

Of course, you were right that --add-modules animal was also needed at runtime as well:

PS F:\Java\ModulesTests> java --module-path 'mods/animal.jar' --class-path puppy puppy.adorable/CutePuppy
Puppy being cute!
Exception in thread "main" java.lang.NoClassDefFoundError: animal/behavior/Behave
       at puppy.adorable.CutePuppy.main(CutePuppy.java:8)
Caused by: java.lang.ClassNotFoundException: animal.behavior.Behave
       at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
       at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
       at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
       ... 1 more


PS F:\Java\ModulesTests> java --module-path 'mods/animal.jar' --add-modules animal --class-path puppy puppy.adorable/CutePuppy
Puppy being cute!
Oh, Behave!
Puppy being bad!
Oh, Behave!


This was a hazard of reading a Certification Book on a topic where the number of things that "know that you can do this" in scope is significantly larger than the scope of "you must know how to do this".

i.e. for the 819, you must know that you can issue some unspecified commands at compile-time and run-time (which turns out to be the steps above) and that they will work, this knowledge is required.
Just how to do it, and in fact the existence of the --add-modules option for javac and java is out of scope for the exam!

In an otherwise near-perfect intro presentation to this topic by Venkat Subramaniam, he tosses out the little nugget that "Stuff running in the unnamed module can't access things in the module-path, they know nothing about it."  Other viewers were also confused in the precise manner I was.  Complicating this was the summary tables we were required to memorize for the exam which controverted what he said, but the surrounding exposition of course didn't show how to do it, because that was out-of-scope, and I remained confused (unsure because I had consumed much tutorial material, none of which showed how to actually succeed at this task) until rescued by yourself.

If I see questions about this in the OCJP forum, I will let people know "You are required to know that you can do this for the exam.  You are not expected to know how to do this for the exam."

Of course, if I should find myself needing to do it, or someone else is dangerously inquisitive, or actually needs to do it, I can point them here with the caveat that it isn't in scope for the OCJP.

Thanks again!
 
Jesse Silverman
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
To make sure I understand this, rather than to complain about things that are inherently going to be sub-optimal because the Java team isn't going to break the whole world as much as they and many others might like them to...

All the nice fancy locking down we can do for our modules in terms of having public classes being hidden from other modules by not exporting packages meant to be internal...

does nothing to protect that code from Reflection access or even direct import-and-call stuff coming from code running on the classpath, right?

I don't intend to learn everything about JPMS at this time, a lot of it is out of scope for me, but having seen so much conflicting information on these basics, I would like to be sure I got this right.

Basically two concepts that are coming to mind are "Shutting the Barn Door after..." and the oft-heard complaint "If we ban guns only criminals will have guns" in that we get nice safe encapsulation as seen by other nice, modern well-behaved code running from Named Modules on the module-path, but there was nothing they could do to protect that stuff from legacy-mode code running from the classpath accessing it because that would have broken lots and lots of stuff in this transition period that may never end?

Thanks!
 
Jesse Silverman
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Do cyclic dependencies going thru CLASSPATH count?

I know the jPMS does not tolerate cyclic dependencies, that is made as an unqualified statement, like, "full stop".

I think the fact that all JARs that happen to be on the CLASSPATH are not just a gaggle of Unnamed Modules, but all part of One Big, Happy, Sloppy Family can be very important in one way I haven't seen, but was just thinking about...

If Named Module A is dependent on Automatic Module B,  which is dependent on the Unnamed Module (which I thought all Automatic Modules are) and the Unnamed Module itself has some dependency on Named Module A -- is this an illegal cyclic dependency?  Is it a legal exception they didn't bother to mention?  Am I all confused because I started studying with music playing??

So many sources explain parts of JPMS but not all of it that it is a bit easy to get lost...

OK, looking at the link provided as authoritative reference:
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/module/package-summary.html

It looks like The Unnamed Module is not part of the Module Resolution machinery that detects illegal cycles.  So what I said there doesn't cause a problem...

I'm also thinking that, tho I can't tell it easily by reading that page, it CAN'T be that illegal cycle detection can involve automatic modules either...just logically because...

I believe every automatic module can read every other automatic module as well as all named modules.

In my mind, that meant it was like it had implicit requires statements for them.  I kept thinking about it like that.
Same thing with exports.  I keep thinking that every automatic module essentially has an exports packageName for each and every package it contains...

that second part would be important for determining if the promised uniqueness of each package (no two modules may ever export the same package name, whether they are partial pieces with different contents and no overlap or not) looked at the automatic modules...I don't know if THAT happens from a casual read of that...

but for cyclical dependency checking, it looks like that can't logically possibly take implicit requires in automatic modules into consideration.  If it did, you could never compile or run a program with two or more automatic modules on the module path...

So I think when we say that "JPMS will fail any attempts at Cyclic Dependency upon either compile or run" we only mean to be saying this for fully modularized programs, running solely from the module-path alone, with no automatic modules in the mix?  This is only one (or two if we count package uniqueness) way that we miss out on the full benefits of JPMS until we abandon both the classpath and automatic modules?

I know, some day I will read one of the two or three great JPMS books out there and this and so much more will all be clear, but somehow this notion was bugging me a lot, essentially, that I was repeating statements by rote that I didn't fully understand because I was unsure how automatic modules and unnamed modules affected the statements about cycles and uniqueness.






 
Stephan van Hulst
Saloon Keeper
Posts: 13256
292
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Note that the page I linked to earlier only mentions cycles in the step that enumerates module names. Module names are retrieved from module descriptors, so we can easily deduce that automatic modules and the unnamed module don't factor into cycle detection at all.

Remember that even though the unnamed module reads all other modules, it is not used to enumerate other modules. This was exactly what caused the problem that you encountered in your second post of this topic.

Regardless, it's very unlikely to introduce a cyclic dependency in non-modular dependencies because you'd have to go out of your way to create one.

One thing that IS a bit strange (and I suspect this is a mistake in the Javadoc) is the notion that enumerating the same module twice implies a cyclic dependency. This is not true. Module A can depend on modules B and C, and both of those can depend on module D. This means module D would be enumerated twice, but it's not a cycle and it's also a perfectly valid and common situation. I think what they meant to say is that enumerating the same module twice during a single path through the recursion tree implies a cyclic dependency.


Jesse Silverman wrote:does nothing to protect that code from Reflection access or even direct import-and-call stuff coming from code running on the classpath, right?


Yes, I think that's correct. I checked it by compiling and running an application from the class path that uses a named module from the module path. It could access a package that wasn't exported just fine. No warnings, no nothing.


Basically two concepts that are coming to mind are "Shutting the Barn Door after..." and the oft-heard complaint "If we ban guns only criminals will have guns" in that we get nice safe encapsulation as seen by other nice, modern well-behaved code running from Named Modules on the module-path, but there was nothing they could do to protect that stuff from legacy-mode code running from the classpath accessing it because that would have broken lots and lots of stuff in this transition period that may never end?


The difference being that you now have the choice to shut the barn before all criminals buy guns simply by requiring that all dependencies must run from the module path. Sure, it may be an arduous process to add module descriptors to all your dependencies (or even hack them into third-party libraries), but theoretically it's possible if you want it.
 
Jesse Silverman
Saloon Keeper
Posts: 1277
38
Eclipse IDE Postgres Database C++ Java
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Note that the page I linked to earlier only mentions cycles in the step that enumerates module names. Module names are retrieved from module descriptors, so we can easily deduce that automatic modules and the unnamed module don't factor into cycle detection at all.

Remember that even though the unnamed module reads all other modules, it is not used to enumerate other modules. This was exactly what caused the problem that you encountered in your second post of this topic.

Regardless, it's very unlikely to introduce a cyclic dependency in non-modular dependencies because you'd have to go out of your way to create one.


Thanks.  All my problems here stem from the fact that I was primarily studying from a source that intends solely to teach you the small parts of a large subject that are in scope for the exam, coupled with the fact that I had some "extra knowledge" from other sources that made statements contained therein seem illogical or impossible, those external sources themselves flatly contradicted each other on fairly basic points (e.g. can code on the module path execute code from the classpath and vice versa, etc.) and there were some things in my primary source that weren't just limited to the scope of the exam, but just incorrect...

I am not sure it is very unlikely to introduce a cyclic dependency in non-modular dependencies during a big migration of a set of "projects" to modular code, perhaps if one sticks straight to the recommended "Top-Down" or "Bottom-Up" approaches it would be hard to do accidentally.  Knowing that neither Automatic Modules nor the Unnamed Module can be said to cause cyclic dependencies goes a long way to calming my fears...still not sure about duplicate package exports from different modules in that regard...

Stephan van Hulst wrote:
One thing that IS a bit strange (and I suspect this is a mistake in the Javadoc) is the notion that enumerating the same module twice implies a cyclic dependency. This is not true. Module A can depend on modules B and C, and both of those can depend on module D. This means module D would be enumerated twice, but it's not a cycle and it's also a perfectly valid and common situation. I think what they meant to say is that enumerating the same module twice during a single path through the recursion tree implies a cyclic dependency.


Thank you so much for either determining from what I wrote that this was confusing me or guessing that it would, when I just read it, the illogical interpretation was indeed what I thought was being described there.  But that wouldn't make sense!  Clarifying which they meant was beyond the scope of the other source materials I was using, as well.

Stephan van Hulst wrote:

Jesse Silverman wrote:does nothing to protect that code from Reflection access or even direct import-and-call stuff coming from code running on the classpath, right?


Yes, I think that's correct. I checked it by compiling and running an application from the class path that uses a named module from the module path. It could access a package that wasn't exported just fine. No warnings, no nothing.


Thanks again.  Most material on these matters focuses on an all-classpath or all module-path build or run, treating any mixes as something you will be working out before checking in changes to the build/deployment, not a state of affairs that might persist for long enough to need to be able to reason about it...

Stephan van Hulst wrote:

Jesse Silverman wrote:Basically two concepts that are coming to mind are "Shutting the Barn Door after..." and the oft-heard complaint "If we ban guns only criminals will have guns" in that we get nice safe encapsulation as seen by other nice, modern well-behaved code running from Named Modules on the module-path, but there was nothing they could do to protect that stuff from legacy-mode code running from the classpath accessing it because that would have broken lots and lots of stuff in this transition period that may never end?


The difference being that you now have the choice to shut the barn before all criminals buy guns simply by requiring that all dependencies must run from the module path. Sure, it may be an arduous process to add module descriptors to all your dependencies (or even hack them into third-party libraries), but theoretically it's possible if you want it.


The larger problem that this ties in to is adoption.  Brief Oracle presentations covering this as an overview kept bringing up the current state of adoption of JPMS as a build/deploy/run unit in the larger world, and nearly always evolved into "Let's see recent estimates of how much Real World Code is even building and running with Java versions greater than 8..."  That is certainly going up, but always lower than one might think while focusing on keeping up with changes in new Java versions.  Some of the optimists inside and outside of Oracle might have been thinking we'd be in a Fully JPMS-ized world by now, with everyone moving their code from Java 11 support to Java 17 now...

So, yeah, there is a reasonably high likelihood that I may wind up reading one or more of the better books devoted to this topic, but that isn't happening now.  I tried to stick to what is covered for the 819, but that proved simply impossible as just had too many Unresolved Contradictions and Quite Basic "Does this mean that?" questions to feel like I had even a solid partial understanding.  I think you got me over that hump.  If I was being paid to convert a large code base to modularization, or doing the same to convert an important Open Source Project, I would certainly be buying and completely reading one of those books, but I should be good for now.

One fun fact, my first exposure to modules was OSGi stuff.  We did use that at work, but my attention was mostly focused on tens of millions of lines of C/C++/C# code, and trying to "learn just enough OSGi" was terrifying/discouraging.  This was all before Java 9.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic