• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Tim Cooke
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Knute Snortum
  • paul wheaton
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Ron McLeod
  • Piet Souris
  • Ganesh Patekar
Bartenders:
  • Tim Holloway
  • Carey Brown
  • salvin francis

class unloading

 
Ranch Hand
Posts: 201
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
sry - couldn't find a sub-forum for better fit - so posting it in general

as far as I know - when loading a class it persists until vm terminates - am I right or is there a way to unload a loaded class?

as explain why I want to know this specific: as I've read something about modular developement (also called plugins) I've wondered: as I can load a plugin while runtime am I able to update it at runtime after loading?
it's easy for me to load something with URLClassLoader - get an object with some factory / reflections magic - and do the same again - but, unless there's something I'm not aware about, the original loaded old version still persists in memory - even if all references cleared, GC ran and URLClassLoader.close() was called to release the file handle
yes, I know GC can't be forced so it can't be predicted if an object and may its class really get's freed even if all references set to null - and I guess this is beyond API but VM implementation specific - if possible at all

ServiceLoader doesn't offer what I want/need - and OSGi - only a framework with different implementing frameworks - seems overkill
it's for learning rather than production - I guess if this goes into production anytime it will be re-fitted with something more production ready - but this depends on outcome of this question and other team members

thanks in advance

for those want to ask why? just want to know if it's possible - or if I'm wrong about this topic
 
Saloon Keeper
Posts: 2679
335
Android Eclipse IDE Angular Framework MySQL Database TypeScript Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I looked at something like this a while back - maybe this old thread may help: Unload or Reload a Dymnically Loaded Class
 
Matt Wong
Ranch Hand
Posts: 201
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thank you - I'll have a look ...

//edit

after reading the linked thread I have to admit: I know to use URLClassLoader.close() (AutoCloseable for try-with-resouces doesn't fit for a class loader as it would close it right after the initial plug in class was loaded with would make it unable to load any other data) and to use a new classloader and kill all references and stuff - but that's not the point of this topic

for me it's about if it's possible to unload classes at all
what I've read from other sources I found is that until J7 or J8 classes were loaded into permgen - but as this got removed with J8 now new classes loaded into heap make them unloadable like any other object free of all references

in fact I can write a small sample gui where I can load and unload different panels doin different things - but again, as this is what I will end up with it's about deep-dive into unloading in specific
 
Matt Wong
Ranch Hand
Posts: 201
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, I dug deeper into Java lang spec and VM spec - chapter 12 for JLS and 5 for VMS - where chapter 12 of JLS reference to chapter 5 of VMS.
About unloading, only JLS makes a few lines in 12.7. The VM spec doesn'T mention anything about unloading at all.
12.7 of JLS says that 1) anything loaded by the bootstrap loader doesn't get unloaded at all and 2) that anything loaded by so called user class loader only be eligable for unloading when the class loader itself isn't reachable anymore and therefore itself is eligable for unloading. It also says that unloading is only a runtime optimization and "not to be discussed by spec".

So, what I read from this: if I somehow for whatever reason want to unload a class, may to replace it by another version, I have to make sure to get rid of any references to any objects created by the classloader I created for loading it in the first place. But: I can't make sure that it will get really cleared out of memory as this depends on garbage collection.
Also: I have to be careful about how to load classes and create objects in the first place to be able to get rid of anything. As far as I'm into it, this will require use of reflections, more specific either using Class.getDeclaredConstructor or invoking a factory or builder method by Class.getDeclaredMethod. I don't know if simple casting will somehow block unloading - but may try to test it.

So, the question if and how class unloading is possible, I guess now I should focus on how to load classes dynamicly. First that comes to my mind would be ServiceLoader + URLClassLoader. ServiceLoader in its inner workings already has all that fancy reflection stuff going with security implemented - wich enables me to also restrict what dynamic loaded classes are allowed to do by load the main code with policy at start up (although that's another chapter I have to learn).
Sure, I could do some myself like: URLClassLoader.loadClass().getDeclaredMethod().invoke() to run a factory with an explicit cast to some interface - but I guess this, while possible, isn't the best way and may introduce security issues wich can be avoided by ServiceLoader.
Yes, also frameworks like OSGi (or rather its implementations) be an option in respect to dependency management and such - but as said, for now that would be overkill.

I may try to come up with a small example ...

Matt
 
Sheriff
Posts: 24632
56
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, your initial impression is correct. To have the special classloader unloaded, you need to not have any references to objects which it loaded. But I think your reflection idea is flawed: if you create objects that way and have them be the ones from your special class, then they will still have a reference to its class object. So you'd still have to deal with getting rid of them before the special class could be garbage collected.

I worked with a system (not written by me) which would automatically reload classes when a new version of a jar file appeared in a certain location, and it used the technique of creating a special classloader to do that. It's been a long time since I worked with it at that level but if I recall right, the technique was that you'd get one special object from the special class and your application code would only use that object. Of course it would produce more objects from the special class and use them for doing whatever it did, but your application wouldn't have references to those objects. That way when you stopped holding a reference to the special object, it and all of the objects it used would then be available for garbage collection. As would the special class (after all objects in that class were GC'ed) and presumably the special classloader.

It was a complex system and it had its memory issues, but it didn't use increasing amounts of memory as classes were reloaded. The base memory usage did stay constant over time.

 
Matt Wong
Ranch Hand
Posts: 201
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
About your mentioned reflections flaw: Aside from it may not be a good idea for production code - What would the issues be? Where would that still accessible refernce come from? Would it somehow cached when calling URLClassLoader.loadClass()? If so, shouldn't clearing all references to the class loader get rid of it as well?
I'm still working on a small example using ServiceLoader - but still didn't decided yet if I would do an initial run at start up or only after the "main application" it self is fully up. Also, as I read classes loaded by bootstrap class loader can't be unloaded so I have to be careful with classpath not to include extension jars at start up of VM.
 
Paul Clapham
Sheriff
Posts: 24632
56
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Now that I think about it more, it's not a flaw. In fact you basically have to use reflection -- or at least the loadClass method of ClassLoader -- to get an instance of a class which was loaded by that ClassLoader. So it's not that it doesn't buy you anything, which is what I originally said. It's that you have to do it that way anyway.

Also bear in mind that every object contains a reference to the class which it's a member of. And every class contains a reference to the classloader which loaded it. So to unload a classloader you need to get rid of references to all objects in classes which were loaded by that classloader.

And this special Class which is loaded by your special ClassLoader: it has to be a subclass of some class which your compiler knows about. (Or implement an interface which your compiler knows about.) That means a class or interface which can be loaded at run time by the standard classloader. Or maybe that's where you planned to use reflection? I suppose you could do that, it's very clumsy, but it doesn't solve any garbage collection problems.
 
Matt Wong
Ranch Hand
Posts: 201
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
About reflections: Yea, it's somewhat "needed" to get an instance. As a ClassLoader.loadClass() only returns a class object, one has to get an object of it. Class.newInstance() clearly state to not to use it but rather Class.getDeclaredConstructor().newInstance() - as about somewhat compile-time exception checking. But getting a constructor is already considered as using reflections.

~ cut out rant about OSGi ~

To move this topic further on:
My original question was about unloading of classes. As I already got from the language specifications, unloading of classes is only considered a possible memory optimization neither specified by language nor as implementation guidline for the runtime. The only thing that in fact is specified is that unloading of classes is only possible for classes loaded by a user defined class loader - and not by the bootstrap classloader (and by that I guess this restriction also includes the extension classloader).
As this topic is about determine wich framework fits my use case for dynamic loading classes at runtime - I dug through what I already know. This includes reflections and ServiceLoader.

About reflections I've once read: Whenever one think to do some with reflections there's almost always a better way, even if it's just a lib wich does the stuff. As a reason for this I got: Security! After I learned a lot about what reflections enable a programmer to do I also learned it's very difficult to do this stuff securely. So, try to do some with reflections myself would may get done, but it would be a security nightmare open up so many issues - I shouldn't do it. And if I think about I have to use reflections I should look for some lib done by someone know how to do it securely.
So, although I may not see what issues could be caused by just using what doc recommend or go a bit "my own way" by searching for a method rather a constructor and invoke it, someone else know about the possible risks would go full ham.

This lead me to using ServiceLoader - as it's done by the Java folks themselves - so I hope that's secure. As I dug through the source, there's a lot of "magic stuff" happen, some reflections, lot of security stuff - and, wich I didn't knew yet, since Java 9 there was another option added to get an instance. Up to Java 8 it was required the class to be loaded by ServiceLoader has to have a public no-arg constructor. How that's actual called is pretty much the line from the doc. The new option since Java 9 is a factory method instead, wich has to be public, static, no-args and returns an object of the class/interface used in load() method.
I guess that's the method I may go, although this would break backwards compat with Java 8 and earlier - but as I see it: Java 8 has reached it's end of live - we all should move on.

About refrences: From what I understand, to make a loaded "extension" eligable for "unloading", it should be enough to get rid of all references, then call .close() on the URLClassLoader used to free the jar, set the reference to it to null, and finally set the reference to the used ServiceLoader to null. This way, there shouldn't be any reachable reference from any alive thread wich should make it eligable for garbage collection. Could there maybe any possible leaks where hidden references could set them self? Something like a hidden counter or class list?

I try to build a test with a few extensions to play around. How could I see if something really gets unloaded? Is a profiler like "mission control" the only option? I don't use an IDE but only an editor and a terminal.

Matt
 
Ron McLeod
Saloon Keeper
Posts: 2679
335
Android Eclipse IDE Angular Framework MySQL Database TypeScript Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Would something like this work?  This example will create a new class loader, load a plugin with, run a plugin's method method, and then close the class loader.

Console
Plugin Interface
First Plugin
Second Plugin
 
Ron McLeod
Saloon Keeper
Posts: 2679
335
Android Eclipse IDE Angular Framework MySQL Database TypeScript Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Matt Wong wrote:How could I see if something really gets unloaded?


Try running your program with the JVM argument -verbose:class
Use -verbose:gc to get information about GC.
 
"How many licks ..." - I think all of this dog's research starts with these words. Tasty tiny ad:
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!