Forums Register Login

Perm Gen Issue

+Pie Number of slices to send: Send
Hi,

We have a IP Telephony server [ uses lot of reflection ] with below JVM parameters .

Eariler Parameters :

-server -Xnoclassgc -XX:+UseConcMarkSweepGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=50 -XX:NewRatio=2 -Xms768m -Xmx768m -Xloggc:/logs/fw/gc.out

With above paramters we faced the problem of OutOfMemory Error: Perm Gen. After searching we added " -XX:MaxPermSize=256m " to it but this will not solve the issue. It will just delay and sooner or later we can again hit the same problem. So we added few more parameters as below

-XX:MaxPermSize=256m
-XX:+CMSPermGenSweepingEnabled
-XX:+CMSClassUnloadingEnabled

and removed only -Xnoclassgc.

But above new parameters caused high load averages on the Solaris Servers and lot of logging started happening in the gc.out that might be the cause for the same. I am not sure about this.

Please guide me in getting the right combination of parameters.

Java version :

java -version
java version "1.5.0_30"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_30-b03)
Java HotSpot(TM) Server VM (build 1.5.0_30-b03, mixed mode)

Thanks,
Rawat
+Pie Number of slices to send: Send
Hi,

Read the GC white paper - it would help you a great deal.

http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf

Now here is my thought process with the problem at hand.

1. Lot of reflection usage means lots of class definitions are being loaded
2. Perm Gen is associated with storage of class/method definitions in old generation
3. Golden rule - let the JVM automatically set/change gc related params and algorithms than we specifying it. To do this give JVM the expected performance goals rather than configuring the algorithms by yourself
4. NewRatio=2 is suggested for client JVMs. For severs the recommended value is 8. Makes sense because your perm gen needs more space than new generations.
5. Give as much memory as possible, and specify performance(in terms of memory or throughput) goals using XX switch and let JVM roll. Such goal params are -XX:MaxGCPauseMillis and -XX:GCTimeRatio

+Pie Number of slices to send: Send
 

Himanshu Rawat wrote:but this will not solve the issue. It will just delay and sooner or later we can again hit the same problem...


Have you considered looking at the design to see if you can reduce the amount of reflection? It seems that you've already worked out that it's your basic problem, and you may be able to speed up throughput as well (reflection is notoriously slow).

Winston
+Pie Number of slices to send: Send
Hi Prabakar,

Thanks for the document. I will go through it .

I am thinking of changing the collector i.e. –XX:+UseConcMarkSweepGC to –XX:+UseParallelOldGC.

Perm Gen is associated with storage of class/method definitions in old generation



If am correct, Perm Gen is related to non heap area??

Thanks,
Rawat
+Pie Number of slices to send: Send
Hi Winston,

Design is THE main issue which can't be changed .

Yes, we analysed and diagnosed heap dumps through various tools and came to the conclusion i.e. "Reflection" issue and "String" literals. Heap dump was flooded with them!!

Earlier we had an option -Xnoclassgc which was preventing the unloading of classes but we now think that this option doesn't affect on Perm Gen area.

I am thinking of upgrading to "JDK 1.6 Update 29" with same options. May be it will solve at-least the high load issues if not Perm Gen


Thanks,
Rawat
1
+Pie Number of slices to send: Send
 

Himanshu Rawat wrote:Design is THE main issue which can't be changed .


I hate to say, but if design is the main issue, then it had better be changed; otherwise you're going to end up in a never-ending spiral of tweaking and band-aid fixes.

Winston
+Pie Number of slices to send: Send
Hi Himanshu,

Perm Gen is non heap area alright, but is used to store class definitions. So, may be you could introduce intelligence in the system to periodically and explicitly unload unwanted class definitions from JVM if you can't stop using reflections. Also, try replacing string manipulations with StringBuffer/StringBuilder. Even such trivial changes might give you surprising results.

But then you can tell whether this is gc settings problem or it is actually a genuine memory issue for the current design right? Even if GC works at its best it may not help your problem. If that is the case you have to start tweaking reflection/string-manipulation code areas and parallelly pushing your higher-ups for design change, eh?

As Winston says, it makes sense to put your efforts in the right area.
+Pie Number of slices to send: Send
256 is too small try 512.
I dont think class unloading is a wise because loading it again will cause lots of time?
try marking much of the method as final(utility methods) although not a good practise.Just for testing.
definietly logging in server side or client side is the cause of very poor perfromance.
try to figure out what you need to store in the memory and what you nee not to.
You will always be treated with dignity. Now, strip naked, get on the probulator and hold this tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com


reply
reply
This thread has been viewed 3350 times.
Similar Threads
Unable to take heap dump, when exceuting this command(jmap -heap:format=b <process id of jboss>)
JVM Initilisation failure - Could not reserve heap
"Timer-10" java.lang.OutOfMemoryError: Java heap space
Spring CGLIB perm gen heap memory issue
GC taking lots of time
More...

All times above are in ranch (not your local) time.
The current ranch time is
Mar 28, 2024 09:05:48.