Win a copy of Java Concurrency Live Lessons this week in the Threads forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

memory leak ??!!  RSS feed

 
Kalpesh Soni
Ranch Hand
Posts: 312
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What will be a memory leak in java !
theoritically there should be none
practically, an unused static List object having lots of object inside it, referring to 1000 other objects, will be a candidate
And there is always a way to leak your memory through CPP code and calling it from java
what else ?
are there any tools to find out such big objects ?
 
Mark Herschberg
Sheriff
Posts: 6037
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Splash screens often fall into wasted memory, as people hide them but don't remove the reference, so a big graphic sits around in memory.
--Mark
 
Avi Abrami
Ranch Hand
Posts: 1141
1
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Kalpesh,
While I agree that, regarding memory leaks,
theoritically there should be none

but it is, nontheless, very easy to cause a "OutOfMemoryError" to be thrown.
(In fact, I remember reading an article that advised how to handle such errors, and -- if I remember correctly -- why it is not such a disaster if you do get them. Unfortunately I don't remember where I read it -- and I'm too lazy to search for it, now :-)
This code gave me an "OutOfMemoryError":

If the returned "ResultSet" had tens of thousands of rows, I would get an "OutOfMemory" error. However, if I moved the definition of variable "col1" to be outside of the "while" loop, I didn't get an "OutOfMemory" error.
The point is, you can write code that does not allow the garbage collector to work as you would like it to. (In fact, according to the java virtual machine specification, a JVM does not _have_ to implement a garbage collector!)
So yes, although it would be nice if we could be guranteed that no memory leaks would occur, unfortunately, relaity dictates differently.
Hope this has helped you.
Good Luck,
Avi.
 
Kalpesh Soni
Ranch Hand
Posts: 312
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Avi Abrami:
Kalpesh,
While I agree that, regarding memory leaks,


What difference does it make if we move col1 declaration outsidte the loop
the same number of objects will still be created
and gc will have to do the same work
just that inside the loop, the jvm will have to work on the stack too much, each time trying to make a reference on the stack
right ? :roll:
 
Stefan Maric
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
code:
--------------------------------------------------------------------------------
Connection dbConn = // whatever
String sql = "select * from very_big_table";
Statement stmt = dbConn.createStatement(sql);
ResultSet rs = stmt.executeQuery();
while (rs.next())
{
String col1 = rs.getString(1);
}
With the String coll inside the while loop - a NEW string is created every iteration
Garbage collection may/may NOT get a chance to recover obsolete Strings
String col1 = "";
while (rs.next())
{
col1 = rs.getString(1);
}
With String coll outside the loop - only ONE instance of a String is created its REFERENCE is re-used at every iteration of the while loop
(NOT completely true with Strings - but pretty much so)
 
Stepan Samarin
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Kalpesh, not exactly. When you write

String object is created each iteration of the loop, and that puts a lot of work on garbage collector. If loop is big, of course you can get OutOfMemory error, simply because you'd have thousands of objects waiting to be gc'ed. If you declare col1 before the loop body:

String object would be created only once, consuming memory required only for one String object.
Hope this'd help.
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
[Stefan]: With String coll outside the loop - only ONE instance of a String is created its REFERENCE is re-used at every iteration of the while loop
[Stepan]: String object would be created only once, consuming memory required only for one String object.
Stefan and Stepan - absolutely not. New String objects are being created on every iteration through this loop, regardless of where the reference is declared. The only difference between the two code fragments is that when you exit the loop, the last string created may still be held in memory if the reference variable was declared outside the loop. But I can't see how this would have any appreciable effect on GC, unless that final string we really, really big.
Avi, it looks like you don't remember your original code exactly; I strongly suspect you are misremembering it based on an incorrect assessment of the cause of your problems. Unless you had the misfortune of running on a really bad JVM implementation which did not GC correctly.
 
Kalpesh Soni
Ranch Hand
Posts: 312
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
again
any good tools to find out big objects ?
we have a large application
to give an idea , its a mail mgmt system
where 100 users can login at a time
15 mb of jars
75 mb of external jars
1500 jsps
runs on weblogic 7
when we run it on jdk1.3.1 hotspot it simply crashes
on classic mode it runs ok
if i want to find out the performance bottlenecks, what is my best bet ?
 
Peter den Haan
author
Ranch Hand
Posts: 3252
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Stepan Samarin:
Kalpesh, not exactly. When you write
String object is created each iteration of the loop, and that puts a lot of work on garbage collector. If loop is big, of course you can get OutOfMemory error, simply because you'd have thousands of objects waiting to be gc'ed. If you declare col1 before the loop body:String object would be created only once, consuming memory required only for one String object.
Whoa. Stepan, you're wrong on three counts here.
1. The "col1" variable is merely a String reference. The actual String objects are created within rs.getString() on every iteration through the loop. This is completely independent of how you declare "col1".
2. The "col1" variable is created only once in both cases anyway. When you start executing a method, place will be allocated in the stack frame for all local variables within the method, regardless of their scope. There is no performance difference between declaring the String inside or outside the loop. I would strongly encourage you to use the declaration inside the loop, because code becomes more readable and maintainable if you keep variable scope as small as possible.
3. You can never get an OutOfMemoryError because there are many objects waiting to be garbage collected. In fact, one of the few guarantees the Java language gives you with respect to garbage collection is that the garbage collector will be run before an OutOfMemoryError is thrown. If you see this error, it is because you have too many live (reachable) objects.
- Peter
 
Kalpesh Soni
Ranch Hand
Posts: 312
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Peter den Haan:
3. You can never get an OutOfMemoryError because there are many objects waiting to be garbage collected. In fact, one of the few guarantees the Java language gives you with respect to garbage collection is that the garbage collector will be run before an OutOfMemoryError is thrown. If you see this error, it is because you have too many live (reachable) objects.
- Peter

well
the fact is that gc behaves differently in classic mode and hotspot mode (the hotspot one has totally different algorithms )
read
http://java.sun.com/docs/performance/index.html
and
http://java.sun.com/docs/hotspot/gc/index.html
&
http://www.javaworld.com/javaworld/jw-01-2002/jw-0111-hotspotgc_p.html
but what I found is
hotspot also has bugs
so u CAN get an OutOfMemoryError if there is a bug in jdk (As I said earlier We used 1.3.1_02 , so not sure about 1.4 and + )
[Jack, Dr. Heinz, someone listening ? ]
Kalps
[ April 29, 2003: Message edited by: Kalpesh Soni ]
 
Peter den Haan
author
Ranch Hand
Posts: 3252
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Jim Yingst:
Avi, it looks like you don't remember your original code exactly; [...]
That much is certain.Compiles toWhereas the code you get when you move the String declaration outside the loopCompiles toSpot the difference
- Peter
 
john guthrie
Ranch Hand
Posts: 124
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
not following the thread, but the example reminds me that some OutOfMemory issues when using jdbc are down to the implementation of the jdbc driver that you are using.
with postgresql, the jdbc driver tries to read and store the entire result set in the jvm, so yes, you can run out of memory with a query that has the potential to return large results.
other drivers, e.g. oracle's, use cursors by default so you don't have this issue (note that posgresql have the capability to use cursors in queries too, it's just not the default behavior)
 
Avi Abrami
Ranch Hand
Posts: 1141
1
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Jim,
You said

Avi, it looks like you don't remember your original code exactly; I strongly suspect you are misremembering it based on an incorrect assessment of the cause of your problems. Unless you had the misfortune of running on a really bad JVM implementation which did not GC correctly.

Yes, I was only going from memory, but, for the record, the test was carried out in the following environment:
java version "1.4.1_02"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_02-b06)
Java HotSpot(TM) Client VM (build 1.4.1_02-b06, mixed mode)
Microsoft Windows XP Professional
Version 2002 Service Pack 1
But I definitely remember that when the variable declarations (yes, there was more than one) where inside the loop, I got an "OutOfMemoryError" and when I moved the decalarations outside of the loop, I didn't -- for the exact same database query (that returned over 100,000 rows).
In any case, it appears that I have started a (healthy?) argument!
Cheers,
Avi.
 
Peter den Haan
author
Ranch Hand
Posts: 3252
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Avi Abrami:
[...] I definitely remember that when the variable declarations (yes, there was more than one) where inside the loop, I got an "OutOfMemoryError" and when I moved the decalarations outside of the loop, I didn't
It would be good if you could find the full code exhibiting this problem, and the before/after situations exactly as you tested them, because in the code listed above there definitely is no difference whatsoever (as demonstrated by decompiling it).
I'm not at all saying you're talking nonsense. To the contrary, you clearly have seen memory leakage that was affected by some seemingly innocuous moving around of declarations. I'm just saying that there must be more to it than we've looked at so far.
- Peter
 
Uday Kiran
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Would it make a difference, if it was StringBuffer, or some other user class such as Employee?
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks, Peter. I was tempted to use javap for my post too, but it was getting late, and I figured someone else would probably bring it up if necessary.
[Avi]: I definitely remember that when the variable declarations (yes, there was more than one) where inside the loop, I got an "OutOfMemoryError" and when I moved the decalarations outside of the loop, I didn't
Sounds very weird. Is it possible that one of the local variable names was shadowing a class or instance variable, and you were somehow changing other interations with that field? It seems like there's got to be some other issues with the surrounding code here, though I have a hard time imagining what they are...
[ April 29, 2003: Message edited by: Jim Yingst ]
 
Jack Shirazi
Author
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Haven't had time to run through the whole discussion, but one thing jumped out at me. Peter, you shouldn't assume that because your compiler compiles the two same, that every compiler will. I've been hit by that before
Compile these two with 1.1.8 and you will get two different bytecode runs
Variable declared outside loop

Variable declared inside loop

Of course if he compiled it under 1.4, then he probably got the same bytecode.
--Jack Shirazi
JavaPerformanceTuning.com
 
Avi Abrami
Ranch Hand
Posts: 1141
1
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Peter,
Regarding your request:

It would be good if you could find the full code exhibiting this problem

I'm sorry, but after a quick search, I couldn't find it -- I may have deleted it!
However, since John mentioned that

some OutOfMemory issues when using jdbc are down to the implementation of the jdbc driver that you are using

I feel -- for completeness' sake -- I should mention that I ran my test against an Oracle 8i (8.1.7.4) database with the [thin] JDBC driver (i.e. "classes12.zip" file) that is included in that distribution. Also, the database server runs on a SUN [sparc] with Solaris 7.
Hope this helps you.
Good Luck,
Avi.
 
Stepan Samarin
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Peter, thanks for clarifying.
 
Bruno Arantes
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
(In fact, I remember reading an article that advised how to handle such errors, and -- if I remember correctly -- why it is not such a disaster if you do get them. Unfortunately I don't remember where I read it -- and I'm too lazy to search for it, now :-)

You might be talking about Jack's article at http://www.onjava.com/pub/a/onjava/2001/08/22/optimization.html
 
Peter den Haan
author
Ranch Hand
Posts: 3252
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Jack Shirazi:
[...]you shouldn't assume that because your compiler compiles the two same, that every compiler will.
That is, of course, absolutely true. Thanks Jack. It's interesting to see that the older compiler optimizes the assignment to "col1" away where 1.4 doesn't: did you have optimization enabled?
- Peter
[ April 30, 2003: Message edited by: Peter den Haan ]
 
Jack Shirazi
Author
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Peter:
I didn't have optimization enabled for the 1.1.8 compile.
Kalpesh:
Any good memory profiler should tell which are the big objects in your application.
--Jack Shirazi
JavaPerformanceTuning.com
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!