• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Memory usage in JVM

 
Ranch Hand
Posts: 114
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi,

not sure if I've chosen right forum. I have some web applications running on JBoss AS. Now I have connected to it's process with YourKit. When I go to memory, I see:

Class name: char[]
number of objects: 10,000,000
Shallow size 1,400,000,000 (43%)

And minior collection every 1 minute reducing this values to 2,000,000 and 400,000,000 accordingly.

I use a lot StringBuilder to generate some complex HTML code it would be difficult to make in JSP and put it as value of h:outputText (with escape=false). Can it be the problem? Anyway right now minor collection happening every 1 minute but traffic is still very low. I have assigned 8gb to xmx and xms and I think it's too much (major collection takes around 8-10 seconds during heavy traffic hours every 20 mins).

Is the number of char[] classes a problem?

Regards,
Michal
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That does seem like an awful lot of char[] alright.

Exactly how are you creating a StringBuilder? According to the javadocs, the no args constructor gives only 16 characters - that initial char[] will be abandoned quickly if you are building big Strings. The String arg constructor starts with the string length + 16 - also likely to be abandoned early.

If this was my problem I would use the StringBuilder( someInt ) with a realistic estimate of the eventual length needed, thus avoiding all the intermediate char[] creation.

The automatic capacity increase just doubles the char[] size - see AbstractStringBuilder

 
Michal Glowacki
Ranch Hand
Posts: 114
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think I do it hard way, but it is difficult for me to change it as the size depends on the numbers of rows returned from database:

 
Rancher
Posts: 4803
7
Mac OS X VI Editor Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'd handle this using the Google Guava collections methods and objects. With them, you dont' have to read all the rows from the DBMS into memory, you can use an Iterable and access each row inside the loop where you need the values. (You can do it without Guava, but it has so many other features that I normally start thinking of using it.) Guava is free, open source, available from Google, well supported, etc.

Its never a good idea to read in thousands of rows before you process the first one.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If this was my problem I would use new StringBuilder( 10000 ) or maybe 20000 - more than needed probably but think of all the extra char[] that would not be created, and there would only be one char[] to be garbage collected.

A further GC consideration is that all those little char[] can leave your memory fragmented.

Just humor me, give it a try.

Bill
 
Sheriff
Posts: 3837
66
Netbeans IDE Oracle Firefox Browser
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

William Brogden wrote:A further GC consideration is that all those little char[] can leave your memory fragmented.


Not really. Modern garbage collectors actually compact the heap. (This makes memory allocation in Java lightning fast, no inspection of available memory linked-lists, just updates to the memory pointer. It can even make up for the performance overhead garbage collectors produce.)

If this was my problem I would use new StringBuilder( 10000 ) or maybe 20000 - more than needed probably but think of all the extra char[] that would not be created, and there would only be one char[] to be garbage collected.


I'd suggest to try to obtain a distribution of sizes your code produces in real environment. Then looking at that, you might come up with sensible initial capacity that would suffice in large number of cases.
 
Michal Glowacki
Ranch Hand
Posts: 114
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am still struggling with this problem. I have made some screenshots, when CPUs are working 50-90% and the traffic is low. JBoss process is more than 9GB at this moment. Here are my jvm settings:


set JAVA_OPTS=%JAVA_OPTS% -Xms8g -Xmx8g -XX:MaxPermSize=512m
set JAVA_OPTS=%JAVA_OPTS% -Dsun.rmi.dgc.client.gcInterval=691200000 -Dsun.rmi.dgc.server.gcInterval=691200000
set JAVA_OPTS=%JAVA_OPTS% -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:gc.log
set JAVA_OPTS=%JAVA_OPTS% -XX:LargePageSizeInBytes=5m -XX:ParallelGCThreads=20 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:ThreadStackSize=1024

I know you gave me some tips regarding StringBuilder initial capacity, however I just wanted to give you more input before I start over with major (in case of my project) changes.

Thanks,
Michal
1.jpg
[Thumbnail for 1.jpg]
2.jpg
[Thumbnail for 2.jpg]
3.jpg
[Thumbnail for 3.jpg]
 
Michal Glowacki
Ranch Hand
Posts: 114
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
and some more from memory section
4.jpg
[Thumbnail for 4.jpg]
5.jpg
[Thumbnail for 5.jpg]
6.jpg
[Thumbnail for 6.jpg]
 
Ranch Hand
Posts: 30
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Michal
I am not an expert with performance but here are my thouhghts.
The StringBuilder initialization should certainly help.
Apart from that the screen shots show lots of Object[] and String . Have you inspected the stack trace of those Objects ?
Also how about caching certain strings that are repeatedly used ? like the HTML elements .
 
Rancher
Posts: 2759
32
Eclipse IDE Spring Tomcat Server
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I second the suggestion to set the initialCapacity on the StringBuilder. That should give you quite a boost.

Another suggestion:-
You might want to try pooling the StringBuilder so you don't create an instance for each request. This will reduce the number of char[] that the GC has to manage. Basically, the only time a char[] will be collected is when the StringBuilder is evicted from the pool, and if you tune the pool right, that should happen only when the load is low. Also, another advantage is that having a pool allows you to degrade gracefully. If the pool is maxed out, you can either make the request wait till instances are available or show a "Server busy" error message. Although either way is not the most desirable, it's much better than overallocation which is going to adversely affect all requests. Most probably you have a limit on JBoss's thread pool anyways. Depending on how much memory is used per request, you might be able to set the maxsize of StringBuilder pool = maxsize of JBoss thread pool

When you passivatethe StringBuilder the pool, call setLength(0). That should clear the array without deallocating it. setLength isn't cheap. So, there will be an overhead. You'll need to benchmark it to measure the impact.

ETA: Are you sure you can't change your implementation to build the HTML in JSP. I get the feeling that you are running into problems that JBoss has already solved. You might be reinventing the wheel a bit here.
 
Greenhorn
Posts: 17
Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

I am not very sure about the all of the below ,

set JAVA_OPTS=%JAVA_OPTS% -Dsun.rmi.dgc.client.gcInterval=691200000 -Dsun.rmi.dgc.server.gcInterval=691200000
set JAVA_OPTS=%JAVA_OPTS% -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:gc.log
set JAVA_OPTS=%JAVA_OPTS% -XX:LargePageSizeInBytes=5m -XX:ParallelGCThreads=20 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:ThreadStackSize=1024

but try the blow java_opts
-XX:+UseParNewGC
-XX:+DisableExplicitGC

is the second JAVA_OPT is really required in your settings?

It is worth try romoving all your above java option and put the one i provided.

Are you using any loggers?
Are you updating db at any point in this process?

I think the large size of string can be a cause of the issue.Is there any way to build the string in smll chuncks?
 
For my next trick, I'll need the help of a tiny ad ...
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic