Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

How to avoid OutOfMemoryError ?  RSS feed

 
Jorge Pinho
Greenhorn
Posts: 25
MySQL Database PHP Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all

I am using an ArrayList to store A LOT of objects (14.000.000 minimum... that�s right

This ArrayList is going to be scanned several times, so, if it could be in memory would be great...

Obviously, I get an OutOfMemoryError... How can I avoid this ? can it be develop so that it uses the Paging File ?

Thankx
Jorge
 
Logan Owen
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Do you have to load all 14,000,000 at the same time? The easiest way would be to break it down into much smaller chunks, and operate only on those chunks, getting rid of them, and then loading up a new section of your 14,000,000.

EDIT:

Or get a computer with a hella lot more RAM... ;-)
[ December 03, 2004: Message edited by: Logan Owen ]
 
Raj Chila
Ranch Hand
Posts: 128
Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

some suggestions from me...

you could use a "lazy caching" approach...Look up an object the first time through some "selection logic" and then cache in a Map with a key.

Logan is right...,It is an exceptional case that all the 1.4 million Objects would be needed to be scanned at a time, and also imagine the time you might need to just get the application to start?...You could divide the Objects into some kind of groups by using a "Value Analisys"
( a term used to manage materials in a supply chain ) where you divide items based on their value and then keep an optimal buffer depending on the value to save resources.

Another passive approach is to maintain a history...so that over a period of time we can know which objects are accessed the most and you could keep those Objects in memory at the applcation start up time...

In case these Objects you are referring to are not some Data already stored in a Database, then you could as well use a Object Serialization where you can create temporary files and use it as a "database" to read and write these objects...

you can also think of "reducing" the Object to some "key" attributes that can be used for scanning purposes only and then load the "full" Object after "matching", for further processing.

well yes, having a lot of memory could solve the problem.
[ December 03, 2004: Message edited by: RajaniKanth Bhargava ]
 
Dirk Schreckmann
Sheriff
Posts: 7023
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm moving this to the Performance forum...
 
Jorge Pinho
Greenhorn
Posts: 25
MySQL Database PHP Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thankx for all your sugestions... I�ve also found some other identical posts here..

We are going for the database solution...

--
Jorge
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What is the pattern of use of these objects? Do you just scan through them in order or do you need random access?
Bill
 
Don't get me started about those stupid light bulbs.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!