OutOfMemory Error

[PraveenVV Veluthakkal]

App Server : JBoss 4.2.2GA
DB : Mysql 5.0.22
O/S : Cent OS 5.


We are using Hibernate / EJB 3. I have set the heap size as 1GB (-XMS & -XMX both 1024M). Recently on our production server, we are experiencing a problem with memory. We have some 70 entities which are associated with each other. So when we fetch an entity it loades all the related entities, which in turn consumes more memory and time. Our current problem is OUT-OF-MEMORY-ERROR. We have got only very less data in our database. Major tables having data only some 500 rows in one table, 2200 in other tables etc.

the Stack trace is given below. Any help in this regard would be greatly appreciated.

Thanks in Advance.
Praveen V


02:18:03,572 ERROR [JDBCExceptionReporter] Error; - nested throwable: (java.lang.OutOfMemoryError: Java heap space)
02:18:03,579 ERROR [JDBCExceptionReporter] Error; - nested throwable: (java.lang.OutOfMemoryError: Java heap space)
02:18:05,413 ERROR [JDBCExceptionReporter] No operations allowed after connection closed.Connection was implicitly closed due to underlying exception/error:


** BEGIN NESTED EXCEPTION **

com.mysql.jdbc.CommunicationsException
MESSAGE: Communications link failure due to underlying exception:

** BEGIN NESTED EXCEPTION **

java.io.EOFException
MESSAGE: Can not read response from server. Expected to read 2,528 bytes, read 2,445 bytes before connection was unexpectedly lost.

STACKTRACE:

java.io.EOFException: Can not read response from server. Expected to read 2,528 bytes, read 2,445 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1997)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2464)

[PraveenVV Veluthakkal]

more info ...

analysed the heap dump. out of 950 MB. 835 MB is consumed by 5 objects of JDBCResultSet (each having a size of 167 mb).

[steve souza]

do you know what those queries are? you should see how many rows they return and also more about the context in which they are called.

[Tim Holloway]

ORMs such as Hibernate support 2 types of data retrieval: Lazy Fetch and Eager Fetch. Usually a direct object reference is Eager Fetched, but its child objects are Lazy Fetched.

If you aren't careful when you turn on Eager Fetch for object cross-references, you can suck the entire database into RAM. This is especially likely to happen when you are working with complex apps where there's lots of cross-references and logic that views relationships in more than one way.

One of the things you can do besides directly controlling the Lazy versus Eager fetch on your model definitions is to use Fetch Groups. I think a basic Fetch Group concept is common to all the ORM systems I've seen, but some also support alternative (named) Fetch Groups to permit even more precise fetching control.

[Subrata Biswas]

Hi,
Just today I have signed as a member here and got attarcted with this Q. Does it happen for the every request after your application server startup? or it happen after some specific load? If second one is right then your issue is not really on the place what you are getting in your LOG. It may differ each time upon your request. I am not sure if i am able to make you understand the point.

Here I am breifing one of my experiencing on one of our application. After launching the application first 6 months there was no issue as planned we needed to restart it every night. but after 6 months when more loads started to come it used get terminated 5 to 6 times a day after Out-of-memory error. And mostly we used to get error message from our LOG file regarding one specific funtion point. So people started to dig into this area. everybody tried the same in myDevelopment Box but could not reproduce it. So one day I used JMeter(open source to do load testing) to capture the script. Then tried to put load through it and was able to reproduce it. After lot of inspection I find out main issue with session. In that application a lot of data is being stored into session. So more user loged in more memory is used. And if user close the browser without logging out session used to be there as it was having 1 hour session time out as per business recuirement.So we did tuning with session and issue got resolved.It may give you some idea.

Anyway can you ans me which one is right out of my two Q at the begining. Depending on your reply I may try to give you the solution.

[Patel Chintan]

Originally posted by Subrata Biswas:
Hi,
Just today I have signed as a member here and got attarcted with this Q. Does it happen for the every request after your application server startup? or it happen after some specific load? If second one is right then your issue is not really on the place what you are getting in your LOG. It may differ each time upon your request. I am not sure if i am able to make you understand the point.

Here I am breifing one of my experiencing on one of our application. After launching the application first 6 months there was no issue as planned we needed to restart it every night. but after 6 months when more loads started to come it used get terminated 5 to 6 times a day after Out-of-memory error. And mostly we used to get error message from our LOG file regarding one specific funtion point. So people started to dig into this area. everybody tried the same in myDevelopment Box but could not reproduce it. So one day I used JMeter(open source to do load testing) to capture the script. Then tried to put load through it and was able to reproduce it. After lot of inspection I find out main issue with session. In that application a lot of data is being stored into session. So more user loged in more memory is used. And if user close the browser without logging out session used to be there as it was having 1 hour session time out as per business recuirement.So we did tuning with session and issue got resolved.It may give you some idea.

Anyway can you ans me which one is right out of my two Q at the begining. Depending on your reply I may try to give you the solution.

Hi Subrata,
The problem is of heap with JVM. You can set heap size JVm by Xms and Xmx params.e.g, -Xms256m -Xmx256m ..

U can also refer for more info about JVM tuning:
http://edocs.bea.com/wls/docs61/perform/JVMTuning.html
http://www.caucho.com/resin-3.0/performance/jvm-tuning.xtp

[PraveenVV Veluthakkal]

Hi,

Thank you all for the suggestions. Actually the cross-relationships among the entities is eating up the memory. Today we found one and fixed it. We are going to deploy the changes in live server tomorrow. I will keep you updated about the progress.

Once again thank you for the solutions. First we had the issue with the session, since we used to keep a lot of data in session. We identified the issue and solved it. Now its the turn of Entities / Hibernate.

Best Regards,
Praveen V