• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

How Do I Find a Memory Leak?

 
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am getting the following:
java.lang.OutOfMemoryError: Java heap space

What should I look for? The error is not specific.
 
lowercase baba
Posts: 13089
67
Chrome Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
you need to look for someplace where you are creating objects and never freeing them. For example, if you have a recursive algorithm, you might never be reaching your end-condition.

Are you processing an obscene amount of data?

Something you can try is putting println() statements in your code to see what it's doing. maybe there's a loop somewhere that's not exiting?

Or, there are memory profilers out there you can try. I've never used on, so can't give any specific advice, but it will monitor your memory usage and maybe clue you in to where all your memory is going.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am processing an obscene amount of data. "Yes it is in the range of pornographic amounts..."

I have not seen this problem up until I began processing this large amount of data. I am currently setting my variables to null at certain points of my code and removing object declarations from inside For-Loops. I will let you know how I do.
 
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You can try increasing the heap size with the -Xmx command-line argument described here.
If that doesn't do the trick, you may have to moderate how much data you are processing.
There is still a chance that you have a real memory leak, and in that case, things get very difficult to figure out. Try searching for "outofmemoryerror" in our Performance Forum. There are many posts there with articles and tools to help (like this one).
 
Marshal
Posts: 79153
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You have got some useful answers (thank you Fred and Joe) but that question is too difficult for us beginners Moving it.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Will it still be answered if it is moved? The last time my topic was moved, there was no more help.
 
Marshal
Posts: 28177
95
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rachel Kozlowski wrote:I am currently setting my variables to null at certain points of my code and removing object declarations from inside For-Loops.

But this sort of thing usually amounts to rearranging the deck chairs on the Titanic. More likely your problem is that you are creating a large number of objects and adding them to some kind of collection. Or generating enormous strings. Or something else which involves storing all your data in memory.

If you don't see anything like that in your code, then you can run a profiler on your code and see which objects you're storing a large number of.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok, you are right it did not work to set objects to null. (I am a beginner, so bear with me.)

How does one run a profiler? I am using eclipse as my development too. I need step by step instructions of how to determine what is holding on to the large amount of data.
 
fred rosenberger
lowercase baba
Posts: 13089
67
Chrome Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How long is your code? if it's not too long, you can post it and it may be obvious to someone - but don't expect anyone to go through hundreds of lines of code. If it's under a hundred or so lines, someone may look at it.

Can you give us an idea what exactly your doing? a high-level overview of your algorithm? The more you can tell us, the more focused advice folks may be able to give you.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Fred, you are very sweet, but unfortunately my code is long and rambling. I would never expect you to look through that. I will get back with you on the logic.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am attempting to populate a report using a classes from Jasper Reports (3rd party software class).
The reportParametersMap is a Hashmap that is populated with data returned from a database:
and the reportDesignObj is basically used to format the Report, by putting a Title on the page and organize the report data on the report and then put footers at the bottom of the page.

The last line of code that the debugger hits before it goes off into the memory leak is :


The area of code that it is found in below is small. Do you see anything obvious?

 
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
you know, i may be talking non-sense, but there's a "runtime" object, that represents the current runtime of your process, this object have the "gc()" method, gc stands for garbage collector.

i'm not sure if java collects garbage for itself or if you need to call the gc() method, but it'll be a good practice to take a deep look at this.

also, make sure your variables are on the lesser scope that you can let them, because you would end up with a variable defined since the code start and only used inside one try, what would make that variable to be garbage outside the try, so define it only on the try.

again, i'm not completely sure about this, its just tips, things for you to give a look around the internet


regards
 
fred rosenberger
lowercase baba
Posts: 13089
67
Chrome Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
you do not need to call the garbage collection method - and I believe doing so can actually slow down your program.

Have you tried increasing your heap size? On the command line, you can type

java -X

and get the non-standard flags. You'll see

-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size

Try making both of these large... I don't know what the default is, or how much memory your box has, but this may help. Maybe something like

java -Xmx512m -Xms512m

if you have that much memory. This may also just be a band aid... if your data size keeps increasing, you'll eventually hit this limit and be back at square 1. Of course, this may have repercussions on other applications on the box, so take that under advisement.

 
Campbell Ritchie
Marshal
Posts: 79153
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
There is usually no need to call the System.gc() or similar methods. Java will run garbage collection automatically.

And you still seem to be getting answers.
 
Lucas Franceschi
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
aha! you're working with jasper too!


look at this:
i am working with jasper reports,
today I got the same problem you got, out of memory.

solution?


maybe you'll need to import some of these:


virtualization is a jasper prevention for your (and mine) case, out of memory.
that's because jasper builds the report on memory, then puts it into a file.

with virtualization, you tell jasper to create some temporary files, that would hold parts of the report, 'till its completed, and merges into one only report.
the "50" in the virtualizer, is the max number of virtual files you want to be crated.

with this property, you can give goodbye for your problem

so, its jasper's problem, not your code's.

(if you specified that you work with jasper in the first post, i could have helped you since the start.)

 
Lucas Franceschi
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
you know, i'm actually testing this property now, and its taking times like 2, or 3 hours to generate my "memory breaking" report, so that if you can test this property there, it would be a great help.

so i insist to say that i know it works, but I never tested it, i'm testing and its taking long time for me to get any result, i'm generating giant, ultrageous big reports, one for each result on database, that means kinda 100000 giant reports being generated once, only to make sure it will not be out of memory, under any circunstances.

also, the last line you execute before the memory leak will be the line that calls the report filling, because the memory leak occurs inside jasper's engine, and only the property I've told you can be a hope of avoiding it.
 
fred rosenberger
lowercase baba
Posts: 13089
67
Chrome Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Lucas Franceschi wrote:...to make sure it will not be out of memory, under any circunstances.


That's a pretty bold statement. Reminds me of the time some guy allegedly said "Why would anyone EVER need more than 640k of memory?"
 
Lucas Franceschi
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

fred rosenberger wrote:

Lucas Franceschi wrote:...to make sure it will not be out of memory, under any circunstances.


That's a pretty bold statement. Reminds me of the time some guy allegedly said "Why would anyone EVER need more than 640k of memory?"


actually, the idea is:
i'm useng a method that prevents java to get out of memory, what should I do to test it?
push it to the limits! i mean, its clear that generating almost millions of reports, one bigger than another, will leak the memory, so if I dont get a "out of memory" exception, the property works fine.

its about testing, because the costumer will not have the conscience to know that the program "crashed" because of the "memory leak", no, what will it say? "your program crashed."

thats what i'm trying to prevent, i'm pushing jasper to its limits, so that i can make sure that anything the user does will not be mentioned as a "crash in my program."

we need to remember, when programming for sell, that our users may be an expert about computer, and may be a complete ignorant about computer, that's why we need a lot of testing.


in about 10 years, we will repeat your quote, but like this "Why would anyone EVER need more than 100Gb of memory?"
or "can you believe that some years ago, people needed to use a strange device, they called mouse, because they could not touch the screen? weird isn't it?"


I think my sons will get curious when i start to talk to them how computers needed to have keyboards in the "old" times.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

look at this:
i am working with jasper reports,
today I got the same problem you got, out of memory.

solution?
view plaincopy to clipboardprint?
JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);
JRFileVirtualizer virtualizador = new JRFileVirtualizer(50);
reportParametersMap.put(JRParameter.REPORT_VIRTUALIZER, virtualizador);


Lucas, I am still having the same problem. Where in the code would I put this?
 
Lucas Franceschi
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rachel Kozlowski wrote:Lucas, I am still having the same problem. Where in the code would I put this?



well, you need to put it on your parameters map, in the same place where you define the other parameters.

try making some tests, take a look at jasper's documentation and try to make it work, but dont forget i'm having the same problem, so, post here if you get any results.


see you tomorrow guys! i'm going home.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I did increase the heap size yesterday. It is at 1024.

If I run the complied code outside of eclipse it takes a number of minutes for the page to display, but when I run it in eclipse I get the error. Regardless, I need to solve this issue before I get even more data.
 
Lucas Franceschi
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
so, setting the property didn't work?
i dont know what to do, it does not work here too.

i'm totally lost now, increasing java heap space would not help, because i would need to change jhs in everyone that uses my program.

I dont know what to do, sorry rachel.
 
Rachel Kozlowski
Ranch Hand
Posts: 40
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I may be doing something incorrect. As another person from this forum noted Jasper Reports put out demo code and I looked at it and the syntax is slightly different from yours.
The snippet from the sample is different. Notice that they have it writing to a tmp file and also notice that the virtualizer is included in the fillReport syntax. I am going to try this next. You should also give it a shot.



The entire Sample is below:
 
Lucas Franceschi
Ranch Hand
Posts: 106
Mac Mac OS X Ruby
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rachel Kozlowski wrote:I may be doing something incorrect. As another person from this forum noted Jasper Reports put out demo code and I looked at it and the syntax is slightly different from yours.
The snippet from the sample is different. Notice that they have it writing to a tmp file and also notice that the virtualizer is included in the fillReport syntax. I am going to try this next. You should also give it a shot.



The entire Sample is below:
...



well, what happens is that the virtualizer will be passed, in this snipped that you have pasted, by the only parameter in the report, that means, the "fillreport" method needs filename, datasource, and parameters. cirtualizer will be a parameter in this case.

also, have you already tested it? if so, what was the results?

it have been some time since the last time I looked at this, so maybe you have already found a solution for this case, if so, please tell us.

try looking for JRFileVirtualizer on google, find the javadoc, and take a look at the constructors, then you'll note that the second argument is optional, and it will be on the local directory if nothing is passed.

hope you have already solved it, but anyway if you don't, try looking a bit more.
 
reply
    Bookmark Topic Watch Topic
  • New Topic