Trouble garbage collecting SoftReference to object with finalize()

Discussion in 'Java' started by Scott W Gifford, Dec 16, 2005.

  1. Hello,

    I'm experimenting with the SoftReference class, for possible use in
    implementing a cache. I'm finding, however, that if the class I'm
    holding a SoftReference to has a finalize() method, then I run out of
    memory even if there are still SoftReference objects that should be
    garbage-collectable.

    Here's a small example:

    import java.util.*;
    import java.lang.ref.*;

    public class SoftRef {
    final static int NUM_BLOCKS = 256;
    final static int BLOCK_SIZE = 1000000;

    private final static class MemoryHog {
    byte[] arr;
    public MemoryHog(int size) {
    arr = new byte[size];
    Arrays.fill(arr,(byte) 3);
    }
    public void finalize() {
    System.out.println("Finalizing " + this);
    }
    }

    public static int liveCount(List<? extends Reference<?>> list) {
    int count = 0;
    for(Reference<?> e: list) {
    if (e.get() != null)
    count++;
    }
    return count;
    }

    public static void main(String[] args) {
    ArrayList<SoftReference<MemoryHog>> hogPen = new ArrayList<SoftReference<MemoryHog>>(NUM_BLOCKS);
    for(int i=1;i<=NUM_BLOCKS;i++) {
    hogPen.add(new SoftReference<MemoryHog>(new MemoryHog(BLOCK_SIZE)));
    System.out.println("Allocated " + i + " blocks; " + liveCount(hogPen) + " still alive.");
    }
    }
    }

    When I run this, I get:

    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

    However, if I remove the finalize() method from the MemoryHog class,
    it works fine.

    If I aggressively run System.gc() during the runs, I don't run out of
    memory, which leads me to believe that I'm not holding any extra
    hard-references preventing these objects from being collected.

    I thought maybe the garbage collection thread couldn't keep up with
    running all of these finalizers, so I tried adding a Thread.sleep(100)
    inside the loop, and that didn't help. Calling System.gc() then
    System.runFinalization() once in each loop didn't help, either; I had
    to call them multiple times.

    Anybody know what's going on here, and how to prevent it? I'm using
    Sun's Java 1.5.0_06 (I also tried 1.5.0_01).

    Thanks!

    ----ScottG.
     
    Scott W Gifford, Dec 16, 2005
    #1
    1. Advertising

  2. Scott W Gifford wrote:
    >
    > I'm experimenting with the SoftReference class, for possible use in
    > implementing a cache. I'm finding, however, that if the class I'm
    > holding a SoftReference to has a finalize() method, then I run out of
    > memory even if there are still SoftReference objects that should be
    > garbage-collectable.


    Normally what has to happen for softly referenced memory to be reclaimed
    is: the GC nulls reference, the GC finds no references to the formerly
    softly referenced memory so it can be overwritten.

    With a nasty finaliser, it becomes: the GC nulls the reference, the GC
    finds a finalisable object and puts it on the finaliser queue (might go
    through another queue as well, can't remember), the finaliser thread(s)
    runs the finaliser (may take some time if there's a System.out.println
    in there), the GC finds the finalised object and can reuse the memory.

    So, keep tight control of your resources. And don't add finalisers
    lightly (for instance on to java.awt.Component...).

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
     
    Thomas Hawtin, Dec 16, 2005
    #2
    1. Advertising

  3. Scott W Gifford

    Chris Uppal Guest

    Thomas Hawtin wrote:

    > With a nasty finaliser, it becomes: the GC nulls the reference, the GC
    > finds a finalisable object and puts it on the finaliser queue (might go
    > through another queue as well, can't remember), the finaliser thread(s)
    > runs the finaliser (may take some time if there's a System.out.println
    > in there), the GC finds the finalised object and can reuse the memory.


    True, but I don't see how it explains the behaviour that Scott is seeing. If
    his example code managed to squeeze out the finaliser thread (as the posted
    code does) then that would explain it, but he mentions that other versions of
    his code do allow time for the finaliser to run, but the odd behaviour
    persists.

    Scott said:

    > I thought maybe the garbage collection thread couldn't keep up with
    > running all of these finalizers, so I tried adding a Thread.sleep(100)
    > inside the loop, and that didn't help. Calling System.gc() then
    > System.runFinalization() once in each loop didn't help, either; I had
    > to call them multiple times.


    I have no explanation myself, unless you managed to mess up this part of the
    test somehow (I've done that myself /far/ too often to treat it as unlikely, so
    please don't be offended).

    -- chris
     
    Chris Uppal, Dec 16, 2005
    #3
  4. Chris Uppal wrote:
    > Thomas Hawtin wrote:
    >
    >
    >>With a nasty finaliser, it becomes: the GC nulls the reference, the GC
    >>finds a finalisable object and puts it on the finaliser queue (might go
    >>through another queue as well, can't remember), the finaliser thread(s)
    >>runs the finaliser (may take some time if there's a System.out.println
    >>in there), the GC finds the finalised object and can reuse the memory.

    >
    >
    > True, but I don't see how it explains the behaviour that Scott is seeing. If
    > his example code managed to squeeze out the finaliser thread (as the posted
    > code does) then that would explain it, but he mentions that other versions of
    > his code do allow time for the finaliser to run, but the odd behaviour
    > persists.


    Taking a closer look at the code, Scott appears to be refreshing all his
    SoftReferences after every allocation. That means the soft-references
    will not be cleared right up until the moment an OutOfMemoryError would
    be thrown if no more memory can be reclaimed. At that point the
    finalisable objects are put on the finaliser queue, where they still
    take memory up. No memory is available immediately and hence the JVM
    goes OOME.

    I have adapted his code to keep counters instead of counting. (Thinking
    about it, switching to Reference.isEnqueued should work.) The byte[]
    count is impossible, probably due to the (ancient, unclear) bug linked
    below. My adaptation has various behaviours depending on parameters
    values which determine whether or not there is enough time to run the
    finalisers and recover.

    http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4214755

    Tom Hawtin

    import java.lang.ref.*;
    import java.util.concurrent.atomic.AtomicInteger;

    public class SoftRef2 {
    final static int NUM_BLOCKS =
    Integer.getInteger("NUM_BLOCKS", 256);
    final static int BLOCK_SIZE =
    Integer.getInteger("BLOCK_SIZE", 1000000);
    final static int SLEEP_MILLIS =
    Integer.getInteger("SLEEP_MILLIS", 100);

    private final static class MemoryHog {
    private final byte[] arr;
    public MemoryHog(byte[] arr) {
    this.arr = arr;
    }
    public void finalize() {
    System.out.println("Finalizing " + this);
    }
    }
    private static class WeakArrayReference
    extends WeakReference<byte[]>
    {
    WeakArrayReference(
    byte[] referent, ReferenceQueue<? super byte[]> queue
    ) {
    super(referent, queue);
    }
    }

    public static void main(String[] args) throws Exception {
    System.out.println("NUM_BLOCKS: "+NUM_BLOCKS);
    System.out.println("BLOCK_SIZE: "+BLOCK_SIZE);
    System.out.println("SLEEP_MILLIS: "+SLEEP_MILLIS);

    final AtomicInteger arrayCount = new AtomicInteger();
    final AtomicInteger hogCount = new AtomicInteger();
    final ReferenceQueue<Object> queue =
    new ReferenceQueue<Object>();

    final Thread thread = new Thread(new Runnable() {
    public void run() {
    try {
    for (;;) {
    Reference<?> reference = queue.remove();
    if (
    reference instanceof WeakArrayReference
    ) {
    arrayCount.decrementAndGet();
    } else {
    hogCount.decrementAndGet();
    }
    }
    } catch (java.lang.InterruptedException exc) {
    // Not expecting that.
    exc.printStackTrace();
    }
    }
    }, "Dead counter");
    thread.setDaemon(true);
    thread.setPriority(Thread.MAX_PRIORITY);
    thread.start();

    final java.util.List<Reference<?>> hogPen =
    new java.util.ArrayList<Reference<?>>(NUM_BLOCKS);
    for(int i=0; i<NUM_BLOCKS; ++i) {
    byte[] arr = new byte[BLOCK_SIZE];
    new WeakArrayReference(arr, queue);
    arrayCount.incrementAndGet();

    hogPen.add(new SoftReference<MemoryHog>(
    new MemoryHog(arr), queue
    ));
    hogCount.incrementAndGet();

    System.out.println(
    "Allocated " + (i+1) + "; living: " +
    arrayCount.get() + " byte[], "+
    hogCount.get() + " head of hog."
    );

    if (SLEEP_MILLIS != 0) {
    Thread.sleep(SLEEP_MILLIS);
    }
    }
    }
    }

    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
     
    Thomas Hawtin, Dec 16, 2005
    #4
  5. Re: Trouble garbage collecting SoftReference to object withfinalize()

    Thomas Hawtin <> writes:

    [...]

    > Taking a closer look at the code, Scott appears to be refreshing all
    > his SoftReferences after every allocation.


    Yes indeed, thanks for finding that!

    [...]

    > I have adapted his code to keep counters instead of
    > counting. (Thinking about it, switching to Reference.isEnqueued
    > should work.) The byte[] count is impossible, probably due to the
    > (ancient, unclear) bug linked below. My adaptation has various
    > behaviours depending on parameters values which determine whether or
    > not there is enough time to run the finalisers and recover.


    Do you have a set of parameters that work for you to avoid an OOME?
    With the default parameters, I see the same problem with your code as
    with mine: the JVM collects nothing until it's almost OOM, then throws
    an OOME. I tried increasing SLEEP_MILLIS to 1000, which seems to give
    more than ehough time for the finalizer thread to do its work, but
    that didn't make a difference at all.

    Thanks!

    ----ScottG.
     
    Scott W Gifford, Dec 16, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roger Irwin

    Garbage collecting on nulls.......

    Roger Irwin, Jul 23, 2003, in forum: Java
    Replies:
    4
    Views:
    468
    Dario
    Jul 24, 2003
  2. Robert Potthast

    Safe garbage collecting

    Robert Potthast, Sep 4, 2005, in forum: C++
    Replies:
    10
    Views:
    476
    Robert Potthast
    Sep 5, 2005
  3. Yuanxin Xi
    Replies:
    7
    Views:
    291
    Steve Holden
    Feb 26, 2009
  4. andrey
    Replies:
    0
    Views:
    167
    andrey
    Dec 15, 2007
  5. Andrey Nikitin
    Replies:
    1
    Views:
    174
    Nobuyoshi Nakada
    Dec 16, 2007
Loading...

Share This Page