• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Bear Bibeault
  • Jeanne Boyarsky
  • Tim Cooke
Sheriffs:
  • Knute Snortum
  • Junilu Lacar
  • Devaka Cooray
Saloon Keepers:
  • Ganesh Patekar
  • Tim Moores
  • Carey Brown
  • Stephan van Hulst
  • salvin francis
Bartenders:
  • Ron McLeod
  • Frits Walraven
  • Pete Letkeman

Loading extended classes with unique dependencies  RSS feed

 
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a main project that provides a super class for other classes to extend, it knows about the interface for the class but doesn't care about how the class is actually implemented. In several other Maven projects, there are classes that extends the super class in the main project, but these Maven projects allow for unique dependencies that the main project does not know about. I want to be able to instantiate these sub-classes from the main project and treat them as instances of the super-class (I am using Spring to get these class instances as Beans), but right now I can not get it to work without copying the sub-project's dependency jars to the classpath of the main project. This is not acceptable because there could be any number of potential Maven sub-projects and possible dependency collisions.

I am thinking of two ways this could be resolved:
1. Load the class somehow with Spring so that only the sub-project needs access to the dependencies.
2. Run the separate sub-projects in separate JVMs and have them communicate with the main project somehow. As of now the sub-projects can not be run on their own because they have no main method, they only really hold these classes.

However I do not have any experience with either of these methods. If anyone has experience with doing either of these or has a better way of decoupling this, I would appreciate the help!
 
Sheriff
Posts: 21503
96
Chrome Eclipse IDE Java Spring Ubuntu VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Zachary Gill wrote:I want to be able to instantiate these sub-classes from the main project


How were you planning on doing that? If you're using reflection, you only need to know the name of the class. You don't need the dependencies until during runtime (at which time you will always need them if you want to use a single JVM).
 
Bartender
Posts: 3321
86
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You could use a separate class loader for each sub project, that way you won't get dependency collisions as each sub project is loaded by it's own Class loader.
 
Zachary Gill
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Rob Spoor wrote:How were you planning on doing that? If you're using reflection, you only need to know the name of the class. You don't need the dependencies until during runtime (at which time you will always need them if you want to use a single JVM).



Yes, I was using reflection before but I don't think that I will be able to use it now that there are unique dependencies in the sub-project that aren't loaded in the main project. I may have to end up using multiple JVMs but I have never worked with that and I don't know how hard it would be to pass object references and call public methods on them from another JVM.


Tony Docherty wrote:You could use a separate class loader for each sub project, that way you won't get dependency collisions as each sub project is loaded by it's own Class loader.



Could elaborate a little more on the use of multiple class loaders? The way I am thinking of it is that there would be a factory class that instantiates the sub-project specific classes and return them to the main project as an instance of the super class. Would I still need to use multiple JVMs with this method though? Even though I would be calling the class loader instead of trying to directly instantiate the class, since it is still called from the main project I think it would still need to know about the sub-project dependencies.
 
Sheriff
Posts: 23714
50
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Zachary Gill wrote:Could elaborate a little more on the use of multiple class loaders? The way I am thinking of it is that there would be a factory class that instantiates the sub-project specific classes and return them to the main project as an instance of the super class. Would I still need to use multiple JVMs with this method though? Even though I would be calling the class loader instead of trying to directly instantiate the class, since it is still called from the main project I think it would still need to know about the sub-project dependencies.



Yes, each class loader would have to know  about its local dependencies. If those correspond to a set of jar files, then it's easy to deal with that... check out the API docs for URLClassLoader, it's specifically designed for that situation. To make it work, those local-dependency jars must be excluded from the application's classpath.

Also, the specific object you extract from those jars must be a subtype of a type which has been loaded from the system classpath, and then it can simply be cast to that type. It sounds like that corresponds fairly closely to what you have already thought of.
 
Zachary Gill
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Paul Clapham wrote:Yes, each class loader would have to know  about its local dependencies. If those correspond to a set of jar files, then it's easy to deal with that... check out the API docs for URLClassLoader, it's specifically designed for that situation. To make it work, those local-dependency jars must be excluded from the application's classpath.

Also, the specific object you extract from those jars must be a subtype of a type which has been loaded from the system classpath, and then it can simply be cast to that type. It sounds like that corresponds fairly closely to what you have already thought of.



I am trying to use the URLClassLoader method but I am still running into the same problem I was having before. The jar that I am trying to load the class from was jar'd with its dependencies, but when I try to load from the jar it is still complaining about dependencies.



It creates the class loader and finds the Class but when it tries to instantiate the class it throws 'Exception in thread "main" java.lang.NoClassDefFoundError: marytts/exceptions/MaryConfigurationException'. Module is a class defined in the main project.

I am including the dependency jars in the URL array for the URLClassLoader. Is there something else I need to do to specify that the dependencies it needs is in the lib/ folder inside the jar?
 
Paul Clapham
Sheriff
Posts: 23714
50
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Zachary Gill wrote:Is there something else I need to do to specify that the dependencies it needs is in the lib/ folder inside the jar?



I'm sorry, I'm a bit lost now. Could you explain what "it" means when you say "it needs", and what "the" means when you say "the jar"?
 
Zachary Gill
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Paul Clapham wrote:Could you explain what "it" means when you say "it needs", and what "the" means when you say "the jar"?



"It needs" meaning the class that I am attempting to load with the class loader. The class imports dependencies that are available within the jar that I am trying to load it from. I attempted to include these dependencies in the list of URLs for the class loader to scan in but I am still getting the error:

Exception in thread "main" java.lang.NoClassDefFoundError: marytts/exceptions/MaryConfigurationException
Caused by: java.lang.ClassNotFoundException: marytts.exceptions.MaryConfigurationException


I build the jar using Maven. Before Maven packages the jar, it copies all the dependencies to the lib/ folder that will end up inside the jar. I also have Maven "add classpath" to the manifest with a "classPath prefix" of lib/ so I think that should make the dependencies visible to the classes within the jar.

Here are my additional Maven build steps, in case it makes a difference:


 
Paul Clapham
Sheriff
Posts: 23714
50
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Okay... the class which you're loading via the URLClassLoader must be able to load all of its dependencies from its classpath, which consists of (1) all jars associated with the URLClassLoader, and (2) the classpath of whatever class created the URLClassLoader. That's if you create the URLClassLoader in the normal way; you can specify its parent classloader to be something else if you use the constructor which allows that, but I don't think you need to do that.

It sounds to me like you're loading Class A from jar X via the URLClassLoader and then it can't find Class B which is in jar Y, and the URLClassLoader should be looking in jar Y but it isn't. Am I understanding that right?

Sorry I can't comment on your Maven issues since I've never used Maven myself.
 
Sheriff
Posts: 5300
142
Chrome Eclipse IDE Java Postgres Database VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Have you tried looking in the jars that Maven made?  Can you see the resources you need in the jar?
 
Zachary Gill
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Paul Clapham wrote:It sounds to me like you're loading Class A from jar X via the URLClassLoader and then it can't find Class B which is in jar Y, and the URLClassLoader should be looking in jar Y but it isn't. Am I understanding that right?



I am trying to load Class A from jar X via the URLClassLoader created in the main project. Class A cannot find Class B, which is in jar Y, and jar Y is in jar X. Jar Y was manually included in the list of jars for the URLClassLoader as you can see in the code above. The main project has jar X as a library but due to the restrictions of my project, the main project cannot also have jar Y as a library. If I make jar Y a dependency of the main project as well then it is able to load Class A without error, but I need the dependencies of jar X to be decoupled from the main project.


Knute Snortum wrote:Have you tried looking in the jars that Maven made?  Can you see the resources you need in the jar?



Inside the jar there is a directory called lib/ that contains all the dependency jars. Do I need to specify somewhere that lib/ contains resources for the jar?
 
Knute Snortum
Sheriff
Posts: 5300
142
Chrome Eclipse IDE Java Postgres Database VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Zachary Gill wrote:

Knute Snortum wrote:Have you tried looking in the jars that Maven made?  Can you see the resources you need in the jar?



Inside the jar there is a directory called lib/ that contains all the dependency jars. Do I need to specify somewhere that lib/ contains resources for the jar?


Are all the resources jars?  If so, setting them in your custom ClassLoader should be enough.
 
Zachary Gill
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Knute Snortum wrote:Are all the resources jars?  If so, setting them in your custom ClassLoader should be enough.



They are jars, and I looked through the jar and found the class that it is throwing NoClassDefFoundError on, so it is definitely in there. Am I setting them in the URLClassLoader incorrectly?
 
Paul Clapham
Sheriff
Posts: 23714
50
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Zachary Gill wrote:Am I setting them in the URLClassLoader incorrectly?



That seems like the most practical theory to work on. When I look at your code there's two places where you use URLs:



Seems like you can access classes from the first URL but not from the others, right? From what I can see the first URL starts with "jar:file:"... wait a minute, I just noticed this comment:



You're saying that these other classes which you can't access are in a jar which is inside another jar, if I read that right. That's your problem right there.

 
Zachary Gill
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I did not realize that this isn't allowed. I tried looking into special classloaders that are able to reference nested jars, but I am going to just extract the jar into a library directory instead. Thank you for your help with this issue.
 
Paul Clapham
Sheriff
Posts: 23714
50
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Glad to hear you're making progress with your problem. I'm a bit surprised that URLClassLoader doesn't complain about URLs which it thinks are invalid or which don't point to a jar file, but I suppose the architects kicked that decision around a bit and decided to let it be permissive rather than restrictive. Although "permissive" makes it harder to find problems like the one you just had.
 
Consider Paul's rocket mass heater.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!