• Post Reply Bookmark Topic Watch Topic
  • New Topic

OutOfMemoryError long before Integer.Max_Value keys are created by the Map.  RSS feed

 
Chan Ag
Rancher
Posts: 1090
14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

I need to create a TreeMap<Integer, SomeType>.

The number of elements in the TreeMap is taken at runtime from the user. For example if the user enters 6, the TreeMap will have 6 keys starting from new Integer(1) to new Integer(6). Similarly there will be 6 SomeType object created with some initial state information.
This is fine so far. So looking at it, I'd assume that I can have Integer.MAX_VALUE number of keys. But the problem is that I would get an OutOfMemoryError long before I am done creating Integer.MAX_VALUE number of SomeType objects. And yes, I need to have the entries pre created cause the entries have a state and the entries are supposed to be there before processing starts ( so that's a given ).

Ideally the capacity is going to be a small number compared to Integer.MAX_SIZE. But my program should handle ( and/or let the user know ) of all possible error prone situations.
Now, how many SomeType objects can be created along with other objects my program will need to create is dependent on the where the program is running.

What should be ideal way to handle this case?
I feel I shouldn't really handle an OutOfMemoryError. This is because in the catch block I may nullify the references and reset the program state and ask the user to specify a lower capacity again, but I don't know when the garbage collector will free up the memory, so my user does not get an OutOfmemoryError again.

So do you think, the ideal way is to add this point in the class documentation and not do anything about it in the code? Or is there another better approach to handle this case?

Please advice.
Thanks,
Chan.
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Chan Ag wrote:So do you think, the ideal way is to add this point in the class documentation and not do anything about it in the code? Or is there another better approach to handle this case?

Certainly it's worth documenting, but unless your objects can start out life in different states, you could probably manage it with a TreeMap allied with a "max Index" field (which is the value supplied by your user). Then you should be able to instantiate elements lazily.

Winston
 
Chan Ag
Rancher
Posts: 1090
14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Winston,

Thanks for your response. I also thought of instantiating them later but SomeClass is a BookSlot in a library. The library can be of regular shape, irregular shape, shapes that can be viewed as a single continuous row, shapes that be viewed as multi dimensional regular structure of some sort. So it becomes necessary to have the BookSlot contain some information about whether they are continuous with a previous BookSlot or not. My implementation is assigning book slot numbers to the slots of a library, adding some state information, so later I can determine if a book ( they may require multiple book slots ) can fit in a slot. Or if a slot can be used. I am also supposed to maintain information like when a slot got occupied and when it got released, if it is available and such things. Since the design is to be scalable for any shape of library I have to rely on adding serial slot numbers to bookslot and adding a state info like if they are continuous with a previous slot.

So with the above constraints, would you say that the API documentation is the only go?

Thanks,
Chan.
 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would think you could use persistent storage (a database of file storage) for the data, then have a reasonable sized 'window' of objects that are kept in memory. The window would slide back and forth around the area that is being viewed. So the amount of total memory required isn't important, only the amount of memory that is required for your window size.

There are several cache implementations that can do this for you, or cache strategies published if you want to build your own.
 
Chan Ag
Rancher
Posts: 1090
14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Steve. And I feel so happy that you've said a thing that I also thought of ( at least partially ). I thought that if the application crashed, I'd have no way to get the data. Yeah, but I really didn't think in terms of using caching options to help with getting the flexibility of instantiating a subset of slots. That's a good idea.

But unfortunately, I would have to read up things to implement it that way and I have only one more day ( actually 5-6 hours to work on it to be precise ) to complete this design- everything inclusive - conceptualizing, designing, coding, testing, and documentation. And most of my work ( except the documentation ) is done, and I am at the stage of unit testing my code and handling these error cases, so I can't really go back and change the design. Not sure if I'd be able to change it even if I had one more day- would need to study more than just a bit we require to fix things here and there.

So I think I'd better just create good documentation, mentioning this point and allow my application to fail ( I hate this part ) perhaps a little gracefully?

Chan.



 
Nikita Salnikov-Tarnovski
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You are just running out of the heap - a textbook case for increasing the size of the heap area by specifying a -Xmx parameter in your startup, just as the following:



If you are interested in why specific OutOfMemoryError's are created, read my recent post in http://plumbr.eu/blog/understanding-java-lang-outofmemoryerror about the topic.
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Chan Ag wrote:Thanks for your response. I also thought of instantiating them later but SomeClass is a BookSlot in a library. The library can be of regular shape, irregular shape, shapes that can be viewed as a single continuous row, shapes that be viewed as multi dimensional regular structure of some sort. So it becomes necessary to have the BookSlot contain some information about whether they are continuous with a previous BookSlot or not.

But that, surely, is something you determine on a case by case basis? If all the user enters is the number of shelves required, and/or some determining pattern, then surely you can use that information when you need to create the actual object. Alternatively (which might take a bit less room), use a BitSet to designate which shelves are "connected". It won't be quite as quick, but I doubt if you'll notice the difference, and you're unlikely to get an OOME since even info for a maxed-out array can be held in 250Mb.

BitSet's are absolutely wonderful things.

Alternatively again, don't try to cope with 2^31 shelves. Even the Library of Congress doesn't have that many "BookSlots", I'll bet.

Mind you, I suspect it's all too late now; but for next time...

Winston
 
Chan Ag
Rancher
Posts: 1090
14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for your response, Winston.

If all the user enters is the number of shelves required, and/or some determining pattern, then surely you can use that information when you need to create the actual object.


The problem, in this case, was that the creation of the Map that contains BookSlot objects as values ( they'd be the values of a Map<Integer, BookSlot>) was done via a factory method. So my interface looked something like this.



Of course I could have created a pattern by asking the user the number of dimensions in the structure of the Library ( considering it's a regular shape library ) and have this value as the return type ( some aggregate of ints). But the problem is not all users are intelligent enough to enter the capacity in terms of dimensions and stuff like that. Some of them would just be happy with - enter the number of bookslots you want in your library and the book slots would be continuous. So I created a few implementations based on the type of user who'd be creating the library ( creating is just one small part of the requirement ). Of course this is also a pattern with the number of dimensions =1 and the aggregate return type would probably be an Array/ArrayList<Integer> of size 2.

But wouldn't this make the implementation of LibraryProvider quite big and complex? I had multiple implementations. And each of these implementations would also call implementations of another interface for creating the library-----> I was going to continue writing this bit ....

Ok while writing this post, just now I have realized that ok, it is possible that all these subclasses return a pattern so instantiating objects on need is possible. What a method returns does not depend on the user, it depends on the programmer. Funny. The idea didn't even strike me. I should have done this.

I will explore BitSet and caching techniques next.

Thanks for your suggestions.

Chan.

  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!