Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

java.lang.outofmemory  RSS feed

 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

I am dealing with an out of memory exception while calling an Oracle stored procedure using Callable statement.

I have called 20000 objects with this procedure, procedure doesnt complain. Then when the objects are 24000 I got the error.

So what I did is, I created sublists with 1000 objects each and then for those 1000 objects, I call the stored procedure.

Still I deal with the same problem..

Any ideas will be appreciated.

Right now when I generate sublists, I am creating connection objects and then closing them also for each 1000 sets.

so its like one connection object will work for 1000 objects.
Callable statement is also registered once for every 1000 objects

csValues = conn.prepareCall("{ ? = call getvalues_proc(?,?,?,?,?,?,?) }");
csValues .registerOutParameter(1, oracle.jdbc.OracleTypes.CURSOR);

then before i stop working on these 1000 objects, I dispose the connection object and the csValues callablestatement also.

But still it fails saying Out of memory exception.

This is what it says exactly

Exception in thread "main" java.lang.OutOfMemoryError
at oracle.jdbc.driver.OracleStatement.prepareAccessors(OracleStatement.java:879)
at oracle.jdbc.driver.T4CResultSetAccessor.getCursor(T4CResultSetAccessor.java:315)
at oracle.jdbc.driver.OracleCallableStatement.getCursor(OracleCallableStatement.java:1659)

Please provide any thoughts

Now here is how I generate sublists.. I am thinking that I keep on creating list objects.. so even though Its working on 1000 objects at a time, the previously creating objects are still existing in memory.

Am I correct ?

this is a sample method I have which I use to create sublist



Thanks
Atul


[NK: Added code tags]
 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Could any one please provide inputs ?

Thanks
 
Satya Maheshwari
Ranch Hand
Posts: 368
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Please use code tags for posting code.
Is it possible to profile your code to see, what is exactly eating up the memory?
Also, can you post the code in processTestObj?
 
Peter Johnson
author
Bartender
Posts: 5856
7
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Add the JVM option -XX:+DumpHeapOnOutOfMemoryError to your app's command line. The JVM will then create a *.hprof file in the current directory when it runs out of heap. Load that file into VisualVM (it comes with JDK6) and it will tell you what is in memory.

If you are running JDK 6, you can attach VisualVM to the JVM running your app and take intermittent memory snapshot and have VisualVM compare them to see what changed in the heap.
 
kri shan
Ranch Hand
Posts: 1487
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Check your Stack and Heap sizes.
 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

First-thanks for the replies:

1. satya - sorry I used "quotes".. will use code tags going ahead.
What I am thinking is since I am storing all objects in the collection, Collections[Lists] eat up memory.

Hence I am trying to see whether I can get chunks of resultset and processs the chunks rather than loading all 20K [or even 50K] to collection at once..

Do you agree ?

Thanks Peter and Kri Shan
Peter-we dont have 1.6 yet, still in 1.5
and also i dont think the company will allow modifying prod file to do it.
I will give a try

Thanks
 
Peter Johnson
author
Bartender
Posts: 5856
7
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
we dont have 1.6 yet, still in 1.5

That is not a problem. I worked an OOME issue with one of our customers a few weeks back - they are using JDK 5. I had them set -XX:+DumpHeapOnOutOfMemoryError and then load the hprof file into the VisualVM installed on one of their laptops. You do not have to change the JDK for your production environment to make use of the features in VisualVM.
 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Peter,

I should just get VisualVM from Jdk 1.6 ?
Trying to download 1.6 and install it on my machine.
Then I will change the jvm option in the AIX machine and get the file.

Will let you know what I find out.

I did it, and added this to the shell script which calls the program
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=file:///oom.hprof

but I dont see a file. Am i missing any ?


Thanks
Atul
 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I did the above and am getting three files only

heapdump.20091105.003746.1380464.phd
Snap0001.20091105.003746.1380464.trc
javacore.20091105.003746.1380464.txt

the other file didnt produce.

Please suggest as of what am I missing

Can i use the .phd file against Visual VM ?

Thanks
 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Tried using .phd file, didnt work
Somehow the .hprof file is not getting created

Here is my java home settings and classpath entries

-Dcom.ibm.oti.vm.bootstrap.library.path=/usr/java5_64/jre/bin
-Dsun.boot.library.path=/usr/java5_64/jre/bin
-Djava.library.path=/usr/java5_64/jre/bin:/usr/java5_64/jre/bin:/usr/java5_64/jre/bin/classic:/usr/java5_64/jre/bin:/appl/local/nsp/lib/oracle:/usr/java5_64/jre/bin/j9vm:/appl/local/nsp/lib/oracle:/usr/lib
-Djava.home=/usr/java5_64/jre
 
Peter Johnson
author
Bartender
Posts: 5856
7
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Oh, you are using AIX. You should always say that you are not using the Sun JDK when asking questions because that is what is assumed. The AIX JVM does thing differently than the Sun JVM. But look at the AIX JVM docs to see if there is any tool for analyzing the files that it produced.
 
Atul Mishra
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Oh sorry..
Could anyone please suggest any ways to analyze this in AIX environment ?

Thanks
 
kri shan
Ranch Hand
Posts: 1487
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Better contact IBM guys. We had similar kind of issue in AIX, IBM send the patch and AIX specific tuning parameters to us.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!