• Post Reply Bookmark Topic Watch Topic
  • New Topic

String[] or ArrayList  RSS feed

 
Anonymous
Ranch Hand
Posts: 18944
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We have a maximum of 20 mio entities and want to store them in an array,(to build charts)
now we recognized with a String[] we have immense performance problems
with that huge amount of data. But the ArrayList crashes after about 7 mio datas.

(we tested on a Sun Blade with 1GB memory and Solaris 8)
Thx in advance
Oliver
 
Jessica Sant
Sheriff
Posts: 4313
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What heap size are you using with your JVM?

Does modifying the heap size help at all?
How big is a 'mio' object?
[ June 20, 2002: Message edited by: Jessica Sant ]
 
Ilja Preuss
author
Sheriff
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Oliver Refle:
We have a maximum of 20 mio entities and want to store them in an array,(to build charts)
now we recognized with a String[] we have immense performance problems

Where do these problems come from? What are you doing with that array?
 
Younes Essouabni
Ranch Hand
Posts: 479
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi jessica,
is the size specfied in Mo?
 
Ivan Tamayo
Ranch Hand
Posts: 49
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think you need to resize the heap. The objects are crated in the heap... you need more space there
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi jessica,
is the size specfied in Mo?

If she's anything like me, she's still wondering what the heck an "mio" is, and now you're asking about "Mo"? Some sort of shared language would be really useful here - like, say, English for example.
 
Ilja Preuss
author
Sheriff
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Perhaps "mio" = "million"?
 
Frank Carver
Sheriff
Posts: 6920
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
By constantly trying one ends up succeeding. Thus: the more one fails the more one has a chance to succeed
May I add that this works only if a different strategy is tried each time. Otherwise you'll just keep walking into the same old wall, and never find the doorway.
In this case, there seems to be a lot of puzzlement about the way the question was phrased. Would it be possible to re-ask it in some other words? I'm sure there are people here who could help.
 
Anonymous
Ranch Hand
Posts: 18944
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What i meant with mio was million data entries.
What we did exactly was to try to copy a result set from a database query into an array list, cause we needed only one row of this result set.
But we came to the point that it would be best to handle it directly with the result set. This meant to transport it into the view layer which i didn't like very much, but we saw no other solution.
To define the HEAP size ? I thought it's dynamicly allocated by the JVM, isn't? This would mean we run out of memory, but this can't be true cause we had 1GB RAM on the test system
 
Jessica Sant
Sheriff
Posts: 4313
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Oliver Refle:
To define the HEAP size ? I thought it's dynamicly allocated by the JVM, isn't? This would mean we run out of memory, but this can't be true cause we had 1GB RAM on the test system

The Maximum size of your heap is not dynamically defined -- you do that by the parameters I listed above. Try playing with those and see if it makes a difference. You can alo use the -verbosegc to see how the garbage collector is doing with your available memory
 
Thomas Paul
mister krabs
Ranch Hand
Posts: 13974
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Oliver Refle:
What i meant with mio was million data entries.
What we did exactly was to try to copy a result set from a database query into an array list, cause we needed only one row of this result set.
Should I bother to ask why you just didn't toss all the rows you didn't need and just keep the one you did need? Or will I just be more confused?
 
Ajith Kallambella
Sheriff
Posts: 5782
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
ResultSet objects are quite fat. What objects are you storing in the ArrayList. I hope you are not simply sticking in the ResultSet objects. At the least, you need a plain vanilla Java object that has just the attributes you need.
 
SJ Adnams
Ranch Hand
Posts: 925
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why the hell do you need 20 million rows of data to draw a chart?
 
Rudy Dakota
Ranch Hand
Posts: 54
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi there,
Given this amount of data, some sort of database seems like a useful tool. If you use one that supports its own data-processing, you could even use it for pre-processing of your data as you retrieve them for display.javascript: x()
Cool
I know it sounds like a detour, but it would take a lot of things from your mind: memory management for one thing.javascript: x()
Wink
Cheers,
Rudy.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!