GC clears all SoftReference's under memory pressure

[Scott W Gifford]

Another question about SoftReference objects and the garbage
collector. As I mentioned in my previous post, I'm experimenting with
implementing a cache using SoftReference objects.

I noticed that the strategy Sun's JVM uses seems to be "when there is
any memory pressure, clear all SoftReference objects". For example,
with the appended code I see this:

Allocated 1 blocks; 1 still alive.
[...]
Allocated 64 blocks; 64 still alive.
Allocated 65 blocks; 1 still alive.
[...]
Allocated 128 blocks; 64 still alive.
Allocated 129 blocks; 1 still alive.
[...]
Allocated 192 blocks; 64 still alive.
Allocated 193 blocks; 1 still alive.
[...]
Allocated 256 blocks; 64 still alive.

I was surprised by this; I expected to see only some of the
SoftReference's freed, perhaps the least-recently used ones or some
approximation of that.

This seems to make a SoftReference inappropriate for a cache, since
when the cache grows too large (which will inevitably happen), the
whole thing will be discarded at once. If the items in the cache were
expensive to calculate, that means there will be a surge in CPU use to
re-calculate everything still needed in the cache, and eventually it
will be entirely thrown away again.

Is the behavior I'm seeing normal and expected, or is it an artifact
of the microbenchmark I'm playing with? Is there any way to affect
this or tune it?

What I like about using SoftReference objects for my cache is that the
cache will automatically be sized based on available memory; I will
have quite a few of these caches (hundreds, one for each client) in my
application, and giving each one an appropriately sized cache will be
quite a balancing act. Is there any way to get similar memory-
conscious behavior in my own cache implementation?

Thanks!

---ScottG.

import java.util.*;
import java.lang.ref.*;

public class SoftRef {
final static int NUM_BLOCKS = 256;
final static int BLOCK_SIZE = 1000000;

private final static class MemoryHog {
byte[] arr;
public MemoryHog(int size) {
arr = new byte[size];
Arrays.fill(arr,(byte) 3);
}
}

public static int liveCount(List<? extends Reference<?>> list) {
int count = 0;
for(Reference<?> e: list) {
if (e.get() != null)
count++;
}
return count;
}

public static void main(String[] args) throws Exception {
ArrayList<SoftReference<MemoryHog>> hogPen = new ArrayList<SoftReference<MemoryHog>>(NUM_BLOCKS);
for(int i=1;i<=NUM_BLOCKS;i++) {
hogPen.add(new SoftReference<MemoryHog>(new MemoryHog(BLOCK_SIZE)));
System.out.println("Allocated " + i + " blocks; " + liveCount(hogPen) + " still alive.");
}
}
}

 

[Thomas Hawtin]

This seems to make a SoftReference inappropriate for a cache, since
when the cache grows too large (which will inevitably happen), the
whole thing will be discarded at once. If the items in the cache were
expensive to calculate, that means there will be a surge in CPU use to
re-calculate everything still needed in the cache, and eventually it
will be entirely thrown away again.

Is the behavior I'm seeing normal and expected, or is it an artifact
of the microbenchmark I'm playing with? Is there any way to affect
this or tune it?

IIRC, Sun's default algorithm (since about 1.4 or 1.3) is to clear a
soft-reference if the number of seconds since last read is greater the
number of megabytes of memory free. It's not done in an exact
thread-safe manner for performance reasons. The Client VM calculates
free memory using the current memory size. The Server VM use the
configured maximum memory size.

Tom Hawtin

 

[Scott W Gifford]

IIRC, Sun's default algorithm (since about 1.4 or 1.3)

Ah, interesting. Any idea where I might be able to find documentation
about this? Google doesn't seem to be my friend on this...

is to clear a soft-reference if the number of seconds since last
read is greater the number of megabytes of memory free.

When I change my sample code to wait 1.5 seconds between allocations
and read the reference right after cerating it, still no references
are cleared until I'm very close to my 128MB memory limit (131 objects
at 1,000,000 bytes each). By that time over 3 minutes have passed,
and about 15 GC cycles, according to -verbose:gc.

So it seems that's not the algorithm it's using in this case, unless
I've misunderstood something.

Thanks again for your help Tom!

----ScottG.

 

[Chris Uppal]

What I like about using SoftReference objects for my cache is that the
cache will automatically be sized based on available memory; I will
have quite a few of these caches (hundreds, one for each client) in my
application, and giving each one an appropriately sized cache will be
quite a balancing act.

Given your description of how you want to use this, I'm sort of sceptical that
the JVM will be able to manage these caches for you adequately. The only
information it has is when the object has been referenced, and how big it is --
both of which are useful, but do not allow it to rank the objects by /value/ in
the way that perhaps you would wish.

I would be tempted to use some kind of explicit LRU scheme, plus my own
evaluation of the worth of each object, plus an explicit eviction process
(probably a Thread). The "eviction" might take the form of converting a ref to
a SoftReference, which would allow you to use the JVM's management feature
without being so dependent on it working properly.

BTW. It is extremely irritating -- and entirely typical of Sun's design
standards -- that java.lang.ref.Reference does not have the obvious (IMO)
fourth pre-defined subclass, StrongReference. That would make managing such
schemes much simpler.

-- chris

 

[Thomas Hawtin]

(1.3.1 it appears, I was in the right area.)

Ah, interesting. Any idea where I might be able to find documentation
about this? Google doesn't seem to be my friend on this...

The trick with google is already knowing the answer first. (This works
particularly well if there are differing opinions on a subject, and you
want your world view reinforced.)

Anyway, the top secret, big list of mostly undocumented Sun HotSpot
options (subject to random changes):

"-XX:SoftRefLRUPolicyMSPerMB=<value> (type intx).
"Starting with Java HotSpot VM implementations in J2SE 1.3.1, softly
reachable objects will remain alive for some amount of time after the
last time they were referenced. The default value is one second of
lifetime per free megabyte in the heap. This value can be adjusted using
the -XX:SoftRefLRUPolicyMSPerMB flag, which accepts integer values
representing milliseconds per MB of free memory."

http://blogs.sun.com/roller/resources/watt/jvm-options-list.html

(So the default value is 1000, not 1.)

Tom Hawtin

 

[Thomas Hawtin]

BTW. It is extremely irritating -- and entirely typical of Sun's design
standards -- that java.lang.ref.Reference does not have the obvious (IMO)
fourth pre-defined subclass, StrongReference. That would make managing such
schemes much simpler.

IIRC, the RFE evaluation for "StrongReference" states that such a class
is trivial and so needn't be part of the Java library. Not entirely sure
I agree with that. I want a NullReference too.

Tom Hawtin

 

[Scott W Gifford]

[...]

IIRC, Sun's default algorithm (since about 1.4 or 1.3)

Ah, interesting. Any idea where I might be able to find documentation
about this? Google doesn't seem to be my friend on this...

is to clear a soft-reference if the number of seconds since last
read is greater the number of megabytes of memory free.

When I change my sample code to wait 1.5 seconds between allocations
and read the reference right after cerating it, still no references
are cleared until I'm very close to my 128MB memory limit (131 objects
at 1,000,000 bytes each). By that time over 3 minutes have passed,
and about 15 GC cycles, according to -verbose:gc.

So it seems that's not the algorithm it's using in this case, unless
I've misunderstood something.

Turns out I was misunderstanding something.

Upon closer inspection (prompted by a comment from Thomas in another
thread), my microbenchmark was suffering from a bit of a Heisenberg
effect: in the process of counting how many references were still
uncollected, I was touching all of them, making the JVM's LRU data
useless. If I count things a different way (with a ReferenceQueue), I
get basically sane behavior.

Thanks Thomas!

----Scott.

 

[Mike Schilling]

BTW. It is extremely irritating -- and entirely typical of Sun's design
standards -- that java.lang.ref.Reference does not have the obvious (IMO)
fourth pre-defined subclass, StrongReference. That would make managing
such
schemes much simpler.

Speaking of which, has anyone found a use for PhantomReferences? They seem
to me to be perfectly useless.

 

[Chris Uppal]

Speaking of which, has anyone found a use for PhantomReferences? They
seem to me to be perfectly useless.

I've never used them myself, but I think they are intended for the case where
you just want to be notified that something /related/ to an object has to be
cleared up. For instance, if you had Java objects representing C structures
created, via JNI, with malloc(), then you might use phantom refs and a
reference queue to ensure that the C data was free()ed by storing the memory
address (as an int or long) in your custom subclass of PhantomReference. This
particular example could be handled by finalisation or by using weak references
plus a queue, but in both cases, the GC is being asked to do more work than is
necessary for the purpose at hand.

You might say that phantom references are intended specifically for setting up
an Observer pattern for watching objects' lifetimes.

-- chris

 

[Ian Pilcher]

IIRC, the RFE evaluation for "StrongReference" states that such a class
is trivial and so needn't be part of the Java library. Not entirely sure
I agree with that. I want a NullReference too.

The Javadoc for java.lang.ref.Reference says, "Because reference objects
are implemented in close cooperation with the garbage collector, this
class may not be subclassed directly." And, indeed, it does not have
any public constructors.

How is implementing a StrongReference (or NullReference) class supposed
to be trivial?

BTW, does anyone else think that Sun messed up when they "generified"
ReferenceQueue? ReferenceQueue.poll (et al) return a
Reference<? extends T>, but if one has subclassed a reference type, it
can still be necessary to cast the Reference to the specific type.

For example, ReferenceQueue<Foo>.poll() will return a Reference<Foo>,
which may need to be cast to a MyWeakReference<Foo> (which will also
cause the compiler to issue a totally ridiculous unchecked cast
warning).

IMO, it would have been better to define it as
ReferenceQueue<R extends Reference<T>>. poll, etc., could then simply
return an R.

 

[Thomas Hawtin]

How is implementing a StrongReference (or NullReference) class supposed
to be trivial?

public class StrongReference<T> extends WeakReference<T> {
private volatile T value;
public StrongReference(T value) {
super(value);
this.value = value;
}
public T get() {
return value;
}
// NB: Not called by GC.
public void clear() {
value = null;
}
}

There are points in the library classes where it would be useful to have
a NullReference without starting all the reference queue handling.
Unfortunately that is started by intialising the Reference class.

BTW, does anyone else think that Sun messed up when they "generified"
ReferenceQueue? ReferenceQueue.poll (et al) return a
Reference<? extends T>, but if one has subclassed a reference type, it
can still be necessary to cast the Reference to the specific type.

Yes, I'd like a better interface. However, it looks impossible to have
parameters for both referent and referrer, without introducing unchecked
casts. I'd prefer a parameter for the referrer and forget the referent.

Tom Hawtin

 

[Ian Pilcher]

public class StrongReference<T> extends WeakReference<T>

Hmm. Do you really feal that a StrongReference is a type of
WeakReference?

Yes, I'd like a better interface. However, it looks impossible to have
parameters for both referent and referrer, without introducing unchecked
casts.

The following code compiles just fine, although it doesn't actually do
anything:

package temp;

import java.lang.ref.*;

public class RefQ<T,R extends Reference<? extends T>>
extends ReferenceQueue<T>
{
public R poll()
{
return null;
}

public R remove()
{
return null;
}

public R remove(long timeout)
{
return null;
}
}

So it looks to me as if Sun could have provided a completely typesafe
ReferenceQueue.

 

[Thomas Hawtin]

Hmm. Do you really feal that a StrongReference is a type of
WeakReference?

No, but it gets the job done.

The following code compiles just fine, although it doesn't actually do
anything:

package temp;

import java.lang.ref.*;

public class RefQ<T,R extends Reference<? extends T>>
extends ReferenceQueue<T>
{
public R poll()
{
return null;
}

public R remove()
{
return null;
}

public R remove(long timeout)
{
return null;
}
}

So it looks to me as if Sun could have provided a completely typesafe
ReferenceQueue.

That's all very well, but if you try to put an implementation behind
that you should come across the need for an unchecked cast that cannot
be checked. You might get away with passing in a Class object and
requiring a subclass without generic parameters.

Tom Hawtin

 

[Mike Schilling]

I've never used them myself, but I think they are intended for the case
where
you just want to be notified that something /related/ to an object has to
be
cleared up. For instance, if you had Java objects representing C
structures
created, via JNI, with malloc(), then you might use phantom refs and a
reference queue to ensure that the C data was free()ed by storing the
memory
address (as an int or long) in your custom subclass of PhantomReference.
This
particular example could be handled by finalisation or by using weak
references
plus a queue, but in both cases, the GC is being asked to do more work
than is
necessary for the purpose at hand.

I don't understand that last bit. How is a weak reference more GC work than
a phantom one?

The unique thing about phantom references is that the object hasn't been
collected (and finalized) when the reference is queued, and won't be until
the reference is manually cleared. This allows cleanup to be done after the
object becomes accessible only by phantom references but before the object
is collected (what the javadoc calls "pre-mortem" cleanup.) I cannot come
up with an example of that being useful. (If I needed to get in before the
finalizer ran, I suppose, but I can't come up with a good example of
*that*.) In your example, it's fine to free the malloc'd memory after
collection, so WeakReferences suffice.

 

[Chris Uppal]

I don't understand that last bit. How is a weak reference more GC work
than a phantom one?

Unless my understanding of phantom references is off, the GC can clear the
reference as soon as it realises that the referent is not otherwise reachable.
It can, therefore, reclaim it immediately since a phantom reference cannot
"resurect" an object in the way that a finaliser or a weak reference can.

The unique thing about phantom references is that the object hasn't been
collected (and finalized) when the reference is queued, and won't be until
the reference is manually cleared.

I don't believe that's true. I could be wrong -- I've never used phantom
references (as I said before) -- but my understanding is that a phantom
reference is queued /after/ finalisation (if any) and that the reference is
cleared before it is queued. The last paragraph of the class javadoc for
PhantomReference is, IMO, just plain misleading -- in fact it makes no sense at
all -- the doc for get() makes it clearer that a PhantomReference doesn't keep
an object alive.

-- chris

 

[Chris Uppal]

public class StrongReference<T> extends WeakReference<T> {
private volatile T value;

Poor, poor, garbage-collector. Doing all that work to track those nasty weak
references, never realising that it is wasting it's time...

-- chris

 

[Mike Schilling]

Unless my understanding of phantom references is off, the GC can clear the
reference as soon as it realises that the referent is not otherwise
reachable.
It can, therefore, reclaim it immediately since a phantom reference cannot
"resurect" an object in the way that a finaliser or a weak reference can.


I don't believe that's true. I could be wrong -- I've never used phantom
references (as I said before) -- but my understanding is that a phantom
reference is queued /after/ finalisation (if any) and that the reference
is
cleared before it is queued. The last paragraph of the class javadoc for
PhantomReference is, IMO, just plain misleading -- in fact it makes no
sense at
all -- the doc for get() makes it clearer that a PhantomReference doesn't
keep
an object alive.

I think you've missed this bit:

-----------------------------------------------
Unlike soft and weak references, phantom references are not automatically
cleared by the garbage collector as they are enqueued. An object that is
reachable via phantom references will remain so until all such references
are cleared or themselves become unreachable.
------------------------------------------------

That is, the object *is* still alive until clear() is called, but
PhantomReference.get() returns null anyway.

This is kind of a fruitless discussion, since neither of us has used them,
and I can't think of a way that the behavior we're disagreeing about can be
detected. (I suppose that if the finalizer did something observable, and
that ever occurred before the reference was explicitly cleared, you'd be
proven right. But the finalizer always happening later wouldn't prove me
right.)

 

[Chris Uppal]

I think you've missed this bit:

That is the paragraph I was refering to when I said:

+> The last paragraph of the class javadoc for PhantomReference is, IMO, just
+> plain misleading -- in fact it makes no sense at all

This is kind of a fruitless discussion, since neither of us has used them,
and I can't think of a way that the behavior we're disagreeing about can
be detected. (I suppose that if the finalizer did something observable,
and that ever occurred before the reference was explicitly cleared, you'd
be proven right. But the finalizer always happening later wouldn't prove
me right.)

Sadly, not even that would settle the issue. I believe that [that passage in]
the documentation is clearly wrong, but if the implementation /did/ happen to
do what I think it should, that would still not prove that the implementation
was correct and the documentation wrong.

FWIW, the documentation does state (as part of the definition of "phantomly
reachable") that objects do not enter that state until their finaliser (if any)
has run.

-- chris

 

[Mike Schilling]

That is the paragraph I was refering to when I said:

+> The last paragraph of the class javadoc for PhantomReference is, IMO,
just
+> plain misleading -- in fact it makes no sense at all

But (continuing with the fruitless discussion) it's consistent with the rest
of the javadoc:

* The phrase "pre-mortem cleanup"
* In order to ensure that a reclaimable object remains so, the referent of a
phantom reference may not be retrieved:
The get method of a phantom reference always returns null.
That is, returns null even though the object has not been collected.

This is kind of a fruitless discussion, since neither of us has used
them,
and I can't think of a way that the behavior we're disagreeing about can
be detected. (I suppose that if the finalizer did something observable,
and that ever occurred before the reference was explicitly cleared, you'd
be proven right. But the finalizer always happening later wouldn't prove
me right.)

Sadly, not even that would settle the issue. I believe that [that passage
in]
the documentation is clearly wrong, but if the implementation /did/ happen
to
do what I think it should, that would still not prove that the
implementation
was correct and the documentation wrong.

FWIW, the documentation does state (as part of the definition of
"phantomly
reachable") that objects do not enter that state until their finaliser (if
any)
has run.

Yes, I'd forgotten that. Making the class (as documented, anyway) even more
useless.