• Post Reply Bookmark Topic Watch Topic
  • New Topic

Memory Efficiency List -> String  RSS feed

 
Jason Griebeler
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm trying to take a List of Strings, consisting of an in memory CSV file, one element per row, and convert it to one giant String (to send over JMS). The ultimate String is a few megabytes, but in memory (using a profiler) it's taking up well over 100mb. How can I reduce this overhead? Thanks in advance for any help!

Everything happens in one very small loop.

 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 37181
515
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jason,
Welcome to JavaRanch!

I would expect 2-3 times the size of the String being used in memory. (You really have two copies of it - once in the list and once in the result. Plus some extra for the extra object state.) Where does the profiler say the memory is going? What objects are using the most?

You might want to try writing it out into a real file and then read in the file. That would give you a feel for the potential memory savings of not looping through the list.

A few comments:
1) Note that StringBuilder is faster than StringBuffer. Since you don't have multiple threads accessing the builder, you can use StringBuilder. I would expect both to use similar amounts of memory although the later will give the garbage collector more cycles to run .
2) Out of curiosity, is there any reason you are using an iterator rather than the enhanced for loop introduced in Java 5? I don't expect the Iterator is using large chunks for memory, but you never know.
3)I'm not sure if new String(buff) and buff.toString() use the same amount of memory. That's something else to try.

 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Isnt there a message creation method that takes a Stream? You could write the component Strings to that Stream and thus avoid duplication.

Bill
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!