• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Perm Gen Issue

 
Himanshu Rawat
Ranch Hand
Posts: 141
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

We have a IP Telephony server [ uses lot of reflection ] with below JVM parameters .

Eariler Parameters :

-server -Xnoclassgc -XX:+UseConcMarkSweepGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=50 -XX:NewRatio=2 -Xms768m -Xmx768m -Xloggc:/logs/fw/gc.out

With above paramters we faced the problem of OutOfMemory Error: Perm Gen. After searching we added " -XX:MaxPermSize=256m " to it but this will not solve the issue. It will just delay and sooner or later we can again hit the same problem. So we added few more parameters as below

-XX:MaxPermSize=256m
-XX:+CMSPermGenSweepingEnabled
-XX:+CMSClassUnloadingEnabled

and removed only -Xnoclassgc.

But above new parameters caused high load averages on the Solaris Servers and lot of logging started happening in the gc.out that might be the cause for the same. I am not sure about this.

Please guide me in getting the right combination of parameters.

Java version :

java -version
java version "1.5.0_30"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_30-b03)
Java HotSpot(TM) Server VM (build 1.5.0_30-b03, mixed mode)

Thanks,
Rawat
 
Prabakar Kalivaradan
Greenhorn
Posts: 20
Chrome Eclipse IDE Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

Read the GC white paper - it would help you a great deal.

http://java.sun.com/j2se/reference/whitepapers/memorymanagement_whitepaper.pdf

Now here is my thought process with the problem at hand.

1. Lot of reflection usage means lots of class definitions are being loaded
2. Perm Gen is associated with storage of class/method definitions in old generation
3. Golden rule - let the JVM automatically set/change gc related params and algorithms than we specifying it. To do this give JVM the expected performance goals rather than configuring the algorithms by yourself
4. NewRatio=2 is suggested for client JVMs. For severs the recommended value is 8. Makes sense because your perm gen needs more space than new generations.
5. Give as much memory as possible, and specify performance(in terms of memory or throughput) goals using XX switch and let JVM roll. Such goal params are -XX:MaxGCPauseMillis and -XX:GCTimeRatio

 
Winston Gutkowski
Bartender
Pie
Posts: 10507
64
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Himanshu Rawat wrote:but this will not solve the issue. It will just delay and sooner or later we can again hit the same problem...

Have you considered looking at the design to see if you can reduce the amount of reflection? It seems that you've already worked out that it's your basic problem, and you may be able to speed up throughput as well (reflection is notoriously slow).

Winston
 
Himanshu Rawat
Ranch Hand
Posts: 141
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Prabakar,

Thanks for the document. I will go through it .

I am thinking of changing the collector i.e. –XX:+UseConcMarkSweepGC to –XX:+UseParallelOldGC.

Perm Gen is associated with storage of class/method definitions in old generation


If am correct, Perm Gen is related to non heap area??

Thanks,
Rawat
 
Himanshu Rawat
Ranch Hand
Posts: 141
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Winston,

Design is THE main issue which can't be changed .

Yes, we analysed and diagnosed heap dumps through various tools and came to the conclusion i.e. "Reflection" issue and "String" literals. Heap dump was flooded with them!!

Earlier we had an option -Xnoclassgc which was preventing the unloading of classes but we now think that this option doesn't affect on Perm Gen area.

I am thinking of upgrading to "JDK 1.6 Update 29" with same options. May be it will solve at-least the high load issues if not Perm Gen


Thanks,
Rawat
 
Winston Gutkowski
Bartender
Pie
Posts: 10507
64
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Himanshu Rawat wrote:Design is THE main issue which can't be changed .

I hate to say, but if design is the main issue, then it had better be changed; otherwise you're going to end up in a never-ending spiral of tweaking and band-aid fixes.

Winston
 
Prabakar Kalivaradan
Greenhorn
Posts: 20
Chrome Eclipse IDE Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Himanshu,

Perm Gen is non heap area alright, but is used to store class definitions. So, may be you could introduce intelligence in the system to periodically and explicitly unload unwanted class definitions from JVM if you can't stop using reflections. Also, try replacing string manipulations with StringBuffer/StringBuilder. Even such trivial changes might give you surprising results.

But then you can tell whether this is gc settings problem or it is actually a genuine memory issue for the current design right? Even if GC works at its best it may not help your problem. If that is the case you have to start tweaking reflection/string-manipulation code areas and parallelly pushing your higher-ups for design change, eh?

As Winston says, it makes sense to put your efforts in the right area.
 
Rojan punn
Greenhorn
Posts: 17
Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
256 is too small try 512.
I dont think class unloading is a wise because loading it again will cause lots of time?
try marking much of the method as final(utility methods) although not a good practise.Just for testing.
definietly logging in server side or client side is the cause of very poor perfromance.
try to figure out what you need to store in the memory and what you nee not to.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic