Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Memory management.  RSS feed

 
Schuyler Goting
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm working on a project that may require a very large amount of memory... We're talking on the order of a gig, or more... Obviously, whenever I approach anything that uses a decently large amount of memory, I start getting out of memory problems. Is there a way to set the memory limit to, essentially, unlimited? (I'm happy to have it go slow if the memory exceeds available RAM, I just need the program to continue working even if the user has 2 gigs of graphics loaded, and then some.)
Thanks for the help.
--Sky
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, have you tried the -Xmx option when you invoke java? I don't think you can speicify "no limit" but you can set it to something really, really big. I suspect you'd be much better off finding a way to have the JVM use less memory, perhaps by keeping things in files rather than in memory. Hard to say; depends on the application; maybe this isn't possible for you. Good luck...
 
Vinod Chandana
Ranch Hand
Posts: 59
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Schuyler Goting:
I'm working on a project that may require a very large amount of memory... We're talking on the order of a gig, or more... Obviously, whenever I approach anything that uses a decently large amount of memory, I start getting out of memory problems. Is there a way to set the memory limit to, essentially, unlimited? (I'm happy to have it go slow if the memory exceeds available RAM, I just need the program to continue working even if the user has 2 gigs of graphics loaded, and then some.)
Thanks for the help.
--Sky

I know this may sound a little silly. But did you try calling garbage collection explictly?
- Vinod.
 
Chris De Vries
Ranch Hand
Posts: 65
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One nifty trick for accessing large amounts of memory in java is to use the java.nio.MappedByteBuffer class. a MappedByteBuffer is actually a file mapped into system memory outside of the java heap. It gets paged in and out by the operating system (usually fairly efficiently) and can be thought of as a virtual memory cache.
The down side is that you can't just create objects on a MappedByteBuffer like you can in the heap, and serialization/deserialization to a bytebuffer can be a bit tricky. But I find it very useful to map large multidimensional binary data sets to MappedByteBuffers before performing operations on them in java.
Chris
 
Idly Vada
Ranch Hand
Posts: 135
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Vinod Chandana:

I know this may sound a little silly. But did you try calling garbage collection explictly?
- Vinod.

Even if you call garbage collector explicitely, there is no guarantee that JVM will do garbage collection. And it recommended not to call garbage collector explicitely. Calling gc explicitly may cause performance problems.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!