• Post Reply Bookmark Topic Watch Topic
  • New Topic

Using Soft references for cache

 
Andreas Johansson
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,
I'm trying to implement a cache using Soft references, but they are cleared too early.
See this one for more about the problem please: http://www.javaranch.com/ubb/Forum34/HTML/000528.html
Could someone help me here?
Thank you. Andreas
 
Peter Tran
Bartender
Posts: 783
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Andreas,
Your problem is difficult to solve. (You probably already realized this and didn't need me to remind you.)
If you don't want to use the Reference API, have you thought about using your caching scheme. Doing that would take us back to your original question. How do you know how much memory the JVM is using to determine if you should clear out the cache or not. Hmm....
Any other gurus out there that can help us out?
Peter Haggar?
Jack Shirazi?
Anyone?
-Peter
 
Jack Shirazi
Author
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ouch, three threads, two forums. Well I'm only replying here.
Q1 how do I get the amount of free memory in the computer?
A1 Firstly, getting the system memory to see how much you could grow the JVM to: there is no Java API. You need to make a system call or call a utility. What is more, on some systems, it is not even clear what is available with system calls. Run vmstat on a modern Unix, and you'll think the memory is full with very little extra available, but this is because of the way the system allocates memory and is only an illusion. Sometimes. Actually its pretty much a pain, and I often have to work out what is available by determining RAM and subtracting allocated process sizes and shared memory (assuming you want RAM and not virtual memory size). It can be similar with windows: look at the task manager and ask if you are really using that much memory.
In any case, the size of the JVM is usually contrained to be smaller than the system memory (see next answer), so knowing what could be allocated can be useless.
Q2 When I tried this [cache] with the SoftReferences it seemed that they where cleared almost directly when the JVM needed more memory.
A2 If you want to maintain a large memory, and delay GC, start the JVM with a large initial memory using -Xms (-ms if using 1.1.x). You'll also need to set -Xmx. This could solve your first problem too: keep trying with bigger parameters until the system will no longer start the JVM, and you will know how much memory it's possible to allocate the JVM.
Q3 When the gc decided to clear the soft references it sweapt all of them, not just the ones necessary to get enough free memory.
A3 Yes, this is how it works, all or nothing. You could try using the other type of weak references too, but I don't really see the point unless you truly have two classes of cachable objects and you don't mind one class being cleared before the other.
If you don't want them garbage collected, don't use weak references. You might be better off with an object database that would swap the objects to and from disk.

Lets take another tack. Either
1. you can get the JVM big enough to handle all you want using -Xms, in which case don't use weak references at all; or
2. you cannot get the JVM big enough, so you need some mechanism for regaining the objects and minimizing garbage collection. In this case, you need to determine what the high water mark is (Runtime.totalmemory). Then create a runtime policy which keeps account of objects created, and deliberately dereference objects for garbage collection when you are near the high-water mark and
want to create more. Use a centralized object creation mechanism, it helps keep account. (also try recycling more objects).
Most applications that need this kind of manipulation of large amounts of data end up needing to use a disk based store of some kind - file storage, RDB or ODB. You can roll your own with serialization, but it won't be particularly efficient.
--Jack Shirazi http://www.JavaPerformanceTuning.com/
 
Andreas Johansson
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Jack Shirazi:
Ouch, three threads, two forums. Well I'm only replying here.

Sorry for the scattering! But I didn't get enough help from the other threads and was recommended to try this forum. Thank you for your answer!

Q3 When the gc decided to clear the soft references it sweapt all of them, not just the ones necessary to get enough free memory.
A3 Yes, this is how it works, all or nothing.

But this means that Soft reference are almost useless for making caches, even if the API for it mentions caching as its prime usage.

I read that for java 1.2beta3 someone reported a bug report (#4124447) that the gc wasn't aggressive enough when cleaning soft references. And now it is TOO agressive for them to be useful.

If you don't want them garbage collected, don't use weak references. You might be better off with an object database that would swap the objects to and from disk.

Lets take another tack. Either
1. you can get the JVM big enough to handle all you want using -Xms, in which case don't use weak references at all; or
2. you cannot get the JVM big enough, so you need some mechanism for regaining the objects and minimizing garbage collection. In this case, you need to determine what the high water mark is (Runtime.totalmemory). Then create a runtime policy which keeps account of objects created, and deliberately dereference objects for garbage collection when you are near the high-water mark and
want to create more. Use a centralized object creation mechanism, it helps keep account. (also try recycling more objects).

We will try doing a combination of setting -Xms & -Xms and to handle our own cahce system. But it will be less flexible for our customers when they have to static decide the amount of memory that the server will use.

I have one more question: Is it possible in some way (by counting bytes or somehow) to get to know how many bytes an object takes up in memory (in the cahce)? Each cached object is an array of quite simple objects and if I could know its memory size much would be gained!

Most applications that need this kind of manipulation of large amounts of data end up needing to use a disk based store of some kind - file storage, RDB or ODB. You can roll your own with serialization, but it won't be particularly efficient.

We use a RDB, but wants to store the most frequently used languages to gain performance.


Thanks again for your reply! Bought your book by the way. I haven't had time to read it through yet, but what I read was interesting!

Best regards,
Andreas

[This message has been edited by Andreas Johansson (edited February 08, 2001).]
 
Peter Tran
Bartender
Posts: 783
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Andreas - read this topic relating to finding the size of JAVA object: http://www.javaranch.com/ubb/Forum15/HTML/000098.html
Jack - thanks for the reply.
-Peter
 
Jack Shirazi
Author
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Okay, SoftReference GC is not actually all or nothing: there is no actual guarantee of anything other than that at some point you may find any one SoftReference cleared. The actual implementation is JVM dependent. But in practice I think it is easier to implement it all or nothing.
I don't really think this makes them useless for caching. I don't think they are intended for such large amounts of data to be cached as you want.
As for your customers' decisions, bear in mind that heap size was set previously - only you were using the default values (2 megs initial and 64 megs max). So you're only making them make a conscious choice instead of the default, and you are just setting your own more appropriate default to be nice.
--Jack Shirazi http://www.JavaPerformanceTuning.com/
 
Andreas Johansson
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Peter Tran:
Andreas - read this topic relating to finding the size of JAVA object: http://www.javaranch.com/ubb/Forum15/HTML/000098.html
Jack - thanks for the reply.
-Peter

Ahh. Excellent! I missed that thread.
Thanks for your support!
/Andreas
 
Andreas Johansson
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Jack Shirazi:
Okay, SoftReference GC is not actually all or nothing: there is no actual guarantee of anything other than that at some point you may find any one SoftReference cleared. The actual implementation is JVM dependent. But in practice I think it is easier to implement it all or nothing.
I don't really think this makes them useless for caching. I don't think they are intended for such large amounts of data to be cached as you want.

Maybe you're right, our data is "too large". But still I think that Soft references are bad implemented when they only stick around a few seconds before being gc:ed. Or rather bad specified, when the JVM is allowed to throw everyting out at once. :-)
Some kind of enhancement here, maybe with priorities, would be nice. Do you know (or someone else) if the Soft references in Java1.4 will be changed in any way? If so, where can I find out information about Java1.4?

As for your customers' decisions, bear in mind that heap size was set previously - only you were using the default values (2 megs initial and 64 megs max). So you're only making them make a conscious choice instead of the default, and you are just setting your own more appropriate default to be nice.
--Jack Shirazi http://www.JavaPerformanceTuning.com/

Ah, right! If you look at it that way it even sounds positive! And that's always good.
Thanks for your reply!
/Andreas
 
Jack Shirazi
Author
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Andreas,
The following method might help out to determine what is available to the JVM, but note that it expands the JVM to the maximum available, i.e. either the system or JVM upper bound
<PRE>
public static int testmemory(int megabytes)
{
if (megabytes > 1000)
{
System.out.println("forget it");
return -1;
}
Object[] memoryHolder = new Object[megabytes];
int count = 0;
try
{
for (; count < memoryHolder.length; count++)
{
memoryHolder[count] = new byte[1048576];
}
}
catch(OutOfMemoryError bounded){}
long highWater = Runtime.getRuntime().totalMemory();
// System.out.println("High water in bytes: " + highWater);
// System.out.println("Megabytes allocatable in megabytes: " + count);
memoryHolder = null; //release for GC
//might be a good idea to call System.gc() and Thread.sleep(2000)
//to give the GC time to happen.
//We know we could allocate "count" megabytes and have a
//high water mark of "highWater". Return whichever you prefer.
return count;
}
</PRE>
--Jack Shirazi http://www.JavaPerformanceTuning.com/

[This message has been edited by Jack Shirazi (edited February 09, 2001).]
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!