Granny's Programming Pearls
"inside of every large program is a small program struggling to get out"
JavaRanch.com/granny.jsp
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

java process memory and heap size

 
Ranch Hand
Posts: 61
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,

I have following problem on AIX. Running Jboss with �Mx512mb the heap size never goes over 64Mb and perm space has 16MB. This is telling me, that I do not have memory leak.
The problem is with java process: at the beginning it takes 128mb and this is continuously growing, it has reached 300mb!

Can somebody explain me, how this can happen?
Why java is taking more and more memory if the internal memory usage does not increase?


Thanks,
Maciej
 
Ranch Hand
Posts: 443
3
Eclipse IDE C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Is the missing memory not just having been freed by the garbage collector, i.e. if you suggest garbage collection does it drop down or am I missing your point.

It looks like your allowing to claim more memory so it does as it can get better preformance that way i.e. memory reuse is quicker than new allocation.

Does JBoss give a garbage collection interaction , like other web servers do or can you run JMX against it.

To detect memory leaks I'm usually looking at the troughs (min memory after garbage collection) against time not rising rather than interested in the peaks that's the JVM's problem (-ish) , off course if your troughs rise to your peaks ;-)
 
Ranch Hand
Posts: 1970
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How are you getting the process size?

There is a pitfall on Windows with Task Manager. By default, it shows physical memory use, which is mostly a useless statistic. You want VM Size, which is available in Task Manager as an optional column.
 
Maciej Miklas
Ranch Hand
Posts: 61
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi ranchos,

process size is 100% correct - this is unix.
GC works fine and gives memmory free, otherwise heap would be bigger.


Regards,
Maciej
 
Peter Chase
Ranch Hand
Posts: 1970
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I failed to spot that it was AIX, so my Windows comment was not helpful. All the same, it can be worth finding out in detail what the reported process size is. For instance, on Mac OS X, you often get massive process size reported, which I think is due to including some shared libraries in the figure.

If you're sure that your memory is not Java heap, then the other places I can think it might be going are: -

  • Java Permanent Generation. This is sort-of part of the Java heap, but not included in the reported stats or controlled by -Xmx.
  • Per-thread memory, like stacks. I believe Unix (well, Linux anyway) is rather hungry for per-thread memory. Do you have hundreds of threads?
  • JNI native code memory allocation. Either legitimate or due to leaks.
  • JDBC native code memory allocation - if it's that type of driver.
  • JVM native code memory allocation. Either legitimate or due to leaks. Presumably, it's not the Sun JVM, as it's AIX. How buggy is IBM's?

  •  
    Maciej Miklas
    Ranch Hand
    Posts: 61
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi,

    I know that java must allocate some additional RAM for ist own proposals.
    The question is, why so much! and why this is just growing and growing and never goes back?
     
    Peter Chase
    Ranch Hand
    Posts: 1970
    1
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    If you really do have a Java process that grows continuously, it sounds like you have a memory leak. Since you say that the Java heap is not growing, it cannot be an ordinary leak in Java code.

    Assuming it's not the Permanent Generation, then some native code is leaking. Do you have any native code of your own? Do you have a JDBC driver with native code? Does the IBM JVM leak (check bug database, if there is one)?

    Also, you still haven't told us exactly what this process size means. I know you're on IBM, so you don't have the Windows Task Manager snafu to deal with, but all the same you need to know, and tell us, exactly what you're looking at. VM Size? Physical memory? Including shared libraries, or not? Does AIX have any tools for breaking down the memory usage of a process into categories?

    These questions are the same as in my last post. I think you have to investigate and answer these questions. No-one is going to be able to magically tell you what's wrong without getting more information.
     
    Maciej Miklas
    Ranch Hand
    Posts: 61
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    process memmory is growing up - I see it using "top".
    Java Heap/permanent is not growing up - I see it using for example Jconsole.

    Regards,
    Maciej
     
    Ranch Hand
    Posts: 66
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    1. It may not be a memory leak.

    2. You can not compare Heap size with the process size. JVM process internally might be using some .so or .dll files which are not accounted in the heap size you give. So even you have set your -Xms = -Xmx, the JVM process size might be bigger than what you have set.

    3. Also, if you have JNI, its not accounted in heap. So all these factors have to be considered. May be you can try pmap or something like that in linux/solaris to find more appropriate results.

    All the best..
     
    Maciej Miklas
    Ranch Hand
    Posts: 61
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    this is correct, and I do not have JNI.
    The problem is: how big can this be (native heap)??
    I've heard about people having process of 3GB for -MX:512 !!

    Is there possibility to telljava: "hey, leave this RAM for another applications, you are not the only one!"

    We have really big servers, and there must be some free ram, because there are running like 20 application servers...
     
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I am also facing the similar issue. I have tried setting all possible JVM parameters and still teh Virtual & RSS memory keeps shooting up.

    Is there a way tofind out what native leak is eactly happening.

    I checked in many forums & posted the question and got a reply saying it could be a native leak. But what is the native leak is still hidden, we do not use any JNI in our programm also.

    Anyu help is much appreciated.
     
    rajesh bala
    Ranch Hand
    Posts: 66
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Harish,

    Which OS are you using. Windows or Linux or Solaris. It may not necessarily be a leak at all. People generally get a feeling that if rss increases, it could be a leak.

    Have you observed any time your JVM crashing due to this?.
    Have you tried -verbose:gc logs and observed heap saturating?

    ~Rajesh.B
     
    Greenhorn
    Posts: 1
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    I'm looking at a similar potential issue for a Linux OS. I only have a small amount of memory avaiable and I'm concerned that the Java process will grow beyond that (128MB) and then die.

    I have used JProfiler for looking at the heap and it is all fine. I don't exhuste the heap at any time and the application has been running for 2 weeks without any memory errors.

    You mention that it isn't necessarily a leak, but I don't understand why the growth in the java process happens. At the moment I'm trying create a picture of the growths over time to see is it slows down or what. I would be very interrested in finding out if there are any clues to it stopping before the entire avaiable memory is used up.

    Is this normal behaviour and is it maybe related to how the specific JVM implementation deals with memory mamagment internally? (I'm using the HotSpot 1.5 from Sun)

    Thanks

    Thomas
    [ August 19, 2007: Message edited by: Thomas Lagoni Olsen ]
     
    Harish Narayanan
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    HI Rajesh,

    We are on Linux. We have got one hint, where we are using J2C connection pool and plenty of prepared statements.

    There are 4 listing page, which is accessed in 8000 times in total per day by all the users. We are now converting these listing page using statement and through a read only connection obtained by DriverManager.getConnection() and making this connection to null.

    I read in one IBM atricle that more usage of prepared statemnt with Type -2 driver will have a memory leak in the native heap.

    Any thoughts on this approach ?
     
    rajesh bala
    Ranch Hand
    Posts: 66
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Harish,

    Yes. That would definitely affect the system's heap. Thatz the reason you might have observed this behavior.

    Creating statements, explicitly closing the resultset and statement might clear the native leaks. But will it not affect the system performance?. PreparedStatements basically reduce the parsing-factor and query-path computation to a great extent. If you are going ahead with statements, every statement has to be parsed again.

    Is it not possible to use type-4 driver?. I am not sure about your environment. But if possible, you could get rid of type-2 and switch over to type-4.

    ~Rajesh.B
     
    Ranch Hand
    Posts: 1491
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Maciej Miklas , can you post your JVM stack & heap size parameters ?
     
    Peter Chase
    Ranch Hand
    Posts: 1970
    1
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by rajesh bala:
    Thatz



    There seems to have been an outbreak of this non-word recently. Please UseRealWords, by which I mean "that's" or "that is".
    [ September 12, 2007: Message edited by: Peter Chase ]
     
    Harish Narayanan
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by rajesh bala:
    Hi Harish,

    Yes. That would definitely affect the system's heap. Thatz the reason you might have observed this behavior.

    Creating statements, explicitly closing the resultset and statement might clear the native leaks. But will it not affect the system performance?. PreparedStatements basically reduce the parsing-factor and query-path computation to a great extent. If you are going ahead with statements, every statement has to be parsed again.

    Is it not possible to use type-4 driver?. I am not sure about your environment. But if possible, you could get rid of type-2 and switch over to type-4.

    ~Rajesh.B

     
    Harish Narayanan
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Rajesh,

    Still memory increasing inspite of converting prepstatem to staments. :-(
     
    Greenhorn
    Posts: 21
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Most outOfMemeory issue are usually related to application.

    It is even more acute when the application include reiterative database operations.

    In such case it is a good idea to use Database connection pooling.

    Use Prepared statement (with batch update) for repetitive inserts/updates

    Limit retrieve ResultSet to a maximum number of rows that your JVM env can handle

    Close database metadata objects(Statements, prepared statement,ResultSet )when no longer used

    Return connection object to connection pool when no longer needed.
     
    rajesh bala
    Ranch Hand
    Posts: 66
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Is it possible for you to throw up a memory profiler with the evaluation version of Yourkit Profiler or JProfiler??

    ~Rajesh.B
     
    Harish Narayanan
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Rajesh,

    We have tried JProfiler and it shows app is working fine. I have enabled Verbose GC and performed analysis with GCAnalyser and GC is also fine.

    The issue is the native heap is increasing. I am also unable to reproduce this native memory increase in our test environment. I have tried pumping more user with JMeter and there is no increase at all in the memory. It is quite normal.

    The application is basically used for capturing user complaints and requests in a call center.

    One thiong what i have noticed is if 1 user login and enters 100 complaints, the increase is not much, where as if 7-8 users log in and enter the same 100 complaints the usage is high (ie the memory is increasing high).

    So we suspected the type-2 driver could be a bottleneck and also we logged a problem (PMR) with IBM and we both are struggling to analyse and find a solution now.

    There are abt 200 + users using the system on a daily basis and this runs on Linux RH 4.3 , WAS 5.1.12 , DB2 UDB Client with fix pack 8.

    The database is seperate opn a server with Db2 UDB8.1 fix pack 8.

    Please advise if you think any change can be made so as to avoid the leak.
     
    rajesh bala
    Ranch Hand
    Posts: 66
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    1. Do you see JVM crash after sometime when you observe this native heap increasing?.

    2. What is your observation on "pmap <pid>" in the system. Which module is eating up the memory?

    My approach to this problem, to prove that driver-2 is the culprit is to make use of driver-4 and perform the benchmarks. If you feel comfortable that driver-4 is stable, you can escalate it further to your management for necessary changes.

    I understand the fact that people won't change from driver-2 to driver-4 immediately in PROD systems. It would go through complete test cycle and so on. But you can check it out with driver-4 in your simulated environment.

    If you are in Bangalore, let me know. We can discuss over phone as well.

    ~Rajesh.B
     
    Harish Narayanan
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    hi rajesh,

    i am in chennai, still i can give a call to your mobile. Please let me know your number.
    [ October 28, 2007: Message edited by: Harish Narayanan ]
     
    Ranch Hand
    Posts: 68
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Maciej Miklas
    I woder how you solved your issue. I have same problem to goes up to 300mb until ended the processing.

    please show me.

    thanks,
    paul
     
    Greenhorn
    Posts: 3
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Paul Maeng:
    Maciej Miklas
    I woder how you solved your issue. I have same problem to goes up to 300mb until ended the processing.

    please show me.

    thanks,
    paul



    Hello. I am currently trying to track on a very similar issue, and we definitely fall into the category of max heap allocated to 2GB and the process growing to 6+GB on a 64bit Linux system.

    We have run YourKit to profile our application and are not seeing our heap usage push its limits. Any ideas? Or if you have resolved your issues, can you give me some ideas or pointers?

    Thanks much in advance!
     
    Peter Chase
    Ranch Hand
    Posts: 1970
    1
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    In general, we would prefer you not to restart months-old discussion threads: DontWakeTheZombies.
     
    Mario Felarca
    Greenhorn
    Posts: 3
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Originally posted by Peter Chase:
    In general, we would prefer you not to restart months-old discussion threads: DontWakeTheZombies.



    My apologies! I did not find more recent ones until after I had already posted. I'll move my question to another thread or post a new one.

    Thanks!
     
    Surfs up space ponies, I'm making gravy without this lumpy, tiny ad:
    New web page for Paul's Rocket Mass Heaters movies
    https://coderanch.com/t/785239/web-page-Paul-Rocket-Mass
    reply
      Bookmark Topic Watch Topic
    • New Topic