• Post Reply Bookmark Topic Watch Topic
  • New Topic

Issue: OutofMemory error with simple application  RSS feed

 
Pravin Jagan
Greenhorn
Posts: 27
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, i have a java application that reads a log file, parses the each log and inserts into DB.(mysql)
the issue happens when we try to process more than 3.5Lakh records. It exits.
I have tried nullifying all the applicable objects (String, ArrayList etc) once its used. and also used explicitly System.gc()
Appreciate if some one can help me on that.
 
Mark E Hansen
Ranch Hand
Posts: 650
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Pravin Jagan wrote:Hi, i have a java application that reads a log file, parses the each log and inserts into DB.(mysql)
the issue happens when we try to process more than 3.5Lakh records. It exits.
I have tried nullifying all the applicable objects (String, ArrayList etc) once its used. and also used explicitly System.gc()
Appreciate if some one can help me on that.


What does "3.5Lakh" mean? I'll assume it means a lot of records.

You're not showing any code, so it's real hard to see what you might be doing wrong.

Gazing into my crystal ball, I can offer this:

Have a look at the methods available on each of your JDBC objects (like statements, result sets, etc.). If they have close() methods, then you probably need to call close() when you're done with them. Start with the ones inside your "for each record" loop, but don't stop there. You should learn good object management early and keep using it forever.

 
Rob Spoor
Sheriff
Posts: 21092
85
Chrome Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Mark E Hansen wrote:What does "3.5Lakh" mean? I'll assume it means a lot of records.

It sure does!

I agree on the closing of ResultSet, (Prepared)Statement and Connection objects once they are no longer needed. Furthermore, are you storing each line in memory before processing them all? If so, don't. Process them one by one, possibly using (Prepared)Statement's batch methods. Keep your batches relatively small (<1000); if they are too large you will again get memory problems.

In pseudo code:
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!