• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Rob Spoor
  • Junilu Lacar
  • paul wheaton
Saloon Keepers:
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
  • Scott Selikoff
Bartenders:
  • Piet Souris
  • Jj Roberts
  • fred rosenberger

Memory issue with Strings/StringBuffers

 
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,
I have a simple java method dealing with Stringbuffers and Strings that eats up much more Memory than it should. The problem is in a loop, which inserts the same String (insertString) in multiple places into a StringBuffer (docStringBuffer):




insertString is a String which is about 30kb, docStringBuffer is initially 15Mb.
I insert the String 5000 times, which means the docStringBuffer gets to about 170Mb.

When I run the code, however, the JVM takes up much memory than I'd expect. At the first execution of the loop, it's about 100Mb, which I find very normal. But after 2000 iterations, this is up to 800Mb! And with 4000 iterations, I get an OOM.

I can understand that the JVM needs 30Mb more after 1000 iterations, but can anyone explain why after each 1000 iterations the JVM needs an additional 450MB?

My Java version is
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build pwi32devifx-20070608 (SR5+IY99712))
IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223-20070426 (JIT enabled)
J9VM - 20070420_12448_lHdSMR
JIT - 20070419_1806_r8
GC - 200704_19)
JCL - 20070608


I have also tried to do the same thing with different combinations of Strings/Stringbuffer (i.e. decraring docString as a Stringbuffer), but it made virtually no difference.

thanks for any help,
Ben
 
Bartender
Posts: 11497
19
Android Google Web Toolkit Mac Eclipse IDE Ubuntu Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Moving to a more suitable forum
 
Bartender
Posts: 1638
IntelliJ IDE MySQL Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Is this the only thing happening in the application when you profiled? If you used a profiler, did you see who is consuming the memory?

StringBuffer will copy arrays when the capacity is increased, so that might be a reason as to why you are seeing the additional memory consumption.
However, before OOM gc would have run and all the unreferenced memory would have been cleared.
 
Ben Zahler
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
in my test case, nothing else happens in the loop, and if comment the line

the memory doesn't increase.

I did profile it with a simple profiler and saw a lot of data in char[] .
 
Master Rancher
Posts: 4209
57
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Ben wrote:I can understand that the JVM needs 30Mb more after 1000 iterations, but can anyone explain why after each 1000 iterations the JVM needs an additional 450MB?


Well, just because it's using an additional 450 MB doesn't mean it needs 450 MB. Some or much of that memory may be occupied by objects that can be garbage collected, but haven't been, yet. Your code is actually creating a lot of extra objects.

Now, by the time the program throws an OOME, the JVM is pretty sure that it really needs more memory than is available. How much memory is that? Are you running with the -Xmx option, or setting the max memory using an IDE?

If you want to find out how much memory is really needed at intermediate steps along the way, you need to "force" garbage collection. Yeah, purists will tell you you can't force it, in the sense that you can never guarantee the results. But you can still get fairly reliable results on standard Sun JVMs, at least if you call the method repeatedly. I use something like this:

Running it ten times is probably overkill, but running it at least four times is often necessary.

Taking another approach though: have you tried using String's replaceAll() method? I think that would be much more efficient than what you're doing here.
 
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Mike Simmons wrote:
Taking another approach though: have you tried using String's replaceAll() method? I think that would be much more efficient than what you're doing here.



I totally agree - it looks like you're just trying to do a replaceAll here. I would go one step further and just compile the regex in advance, though, since otherwise replaceAll() will internally compile the regex over and over again (replace() does the same thing, but escapes the String so it can be understood as a regex). So in short, I think this code should be functionally equivalent but much cleaner:



Try running that and see if you still get the OOME.
 
pie. tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic