Doing one last thing to a WeakReference

[Paul J. Lucas]

If I have a WeakReference to a Closeable, how can I call Closeable.close() just
before the referrent is garbage-collected?

One idea would be to derive from WeakReference and override clear() like:

public class MyWeakReference extends WeakReference<Closeable> {
// ...
public void clear() {
Closeable ref = get();
if ( ref != null )
ref.close();
super.clear();
}
}

But will this work and it is guaranteed to do so?

- Paul

 

[Kevin McMurtrie]

If I have a WeakReference to a Closeable, how can I call Closeable.close()
just
before the referrent is garbage-collected?

One idea would be to derive from WeakReference and override clear() like:

public class MyWeakReference extends WeakReference<Closeable> {
// ...
public void clear() {
Closeable ref = get();
if ( ref != null )
ref.close();
super.clear();
}
}

But will this work and it is guaranteed to do so?

- Paul

You can not.

First, clear() is a user's method. The JVM doesn't call it.

Second, there's no order to garbage collection and finalization. The
method Object.finalize() gives you early warning but there's not a whole
lot you can do there. Any computations there will bring GCs to a crawl.
Any data left in your instance might or might not have been finalized
already. Finalization only happens once so making a new hard reference
is very bad. You do not want to call another Object's close() method
from your finalize() method.

In the case of a Reference, the referenced object vanishes without
warning some time after hard references are gone. That can be
milliseconds or hours later, depending on what heap the referred object
settles into. You can be notified of a Reference having been cleared
using a ReferenceQueue. For this to be of any use, you'll need to
extract some metadata early on. Example:

//Declarations
private final ReferenceQueue m_lostQueue= new ReferenceQueue ();
class MyWeakReference extends WeakReference<MyObject>
{
final String key;
public MyWeakReference (MyObject thing)
{
super (thing, m_lostQueue);
key= thing.toString();
}
}

//Periodic cleanup
MyWeakReference lost;
while ((lost= (MyWeakReference)m_lostQueue.poll()) != null)
System.out.println ("Garbage collected " + lost.key);



A PhantomReference is an extreme case. Its only purpose is to notify
you that an Object has been garbage collected. You'd use this to free
external resources, like native memory or a remote session, when the
Java object representing it locally is gone.

 

[Paul J. Lucas]

[ snip ]

You can not.

[ snip ]

So you're saying there's no possible way to accomplish what I want, i.e., to
call close() on an object that is only weakly-reachable?

- Paul

 

[Daniel Pitts]

[ snip ]

You can not.

[ snip ]

So you're saying there's no possible way to accomplish what I want,
i.e., to call close() on an object that is only weakly-reachable?

- Paul

Correct. Generally, leaving resources that are closeable to the whim of
the garbage collector is a bad idea. The better approach is to wrap the
usage of Closeables in a try/finally block. Garbage collection is
useful for just that, collecting unused memory. Other resources aren't
always amenable to the same algorithms unfortunately.

Sorry, this isn't C++ and RAII doesn't work.

 

[Tom Anderson]

If I have a WeakReference to a Closeable, how can I call
Closeable.close() just before the referrent is garbage-collected?

You can't.

I think you have two options.

Firstly, write a finalizer in the Closeable.

Secondly, factor the Closeable out into a Closeable and something like a
Closer, which contains just enough information to do the
closing, and no reference to the Closeable. Then subclass WeakReference:

public class ClosingReference extendsWeakReference {
private Closer closer ;
public ClosingReference(Closeable c, ReferenceQueue q) {
super(c, q) ;
this.closer = c.getCloser() ;
}
public void close() {
closer.close() ;
}
}

You create your ReferenceQueue, create your ClosingReference with a link
to that queue, and then set up a thread which pops references off the
queue and closes them.

If you can't refactor the Closeable, you would seem to be a bit stuffed.

tom

 

[Tom Anderson]

Joshua Bloch, /Effective Java/
<http://java.sun.com/docs/books/effective/index.html>,
Item 7, "Avoid finalizers".

"Finalizers are unpredictable, often dangerous, and generally unncessary.
Their use can cause erratic behaviour, poor performance, and portability
problems. Finalizers have a few valid uses, which we'll cover later in
this item, but as a rule of thumb, finalizers should be avoided."

"C++ programmers are cautioned not to think of finalizers as the analog
of C++ destructors ..."

Josh is telling us to avoid finalizers is we can, but acknowledges that
they do have their uses. And, as he indicates, this rule is mostly aimed
at ex-C++ programmers who might mistake them for destructors. He goes on
to give some good advice about what finalizers shouldn't be used for, then
winds up describing their "two legitimate uses" - as a safety net for when
explicit destruction doesn't happen, and for dealing with native resources
that need to be freed when objects die.

All in all, "rule of thumb" is the key phrase here. Don't use finalizers
when it's not appropriate. But do use them when it is.

Finalizers are hugely expensive to GC - an object with an overridden
'finalize()' takes at least two GC cycles to clean up. This applies
therefore to objects that reference it and that it references.

Sure, but "hugely expensive" is hyperbole. Do we have any actual numbers
on this? I can't imagine that using finalizers on 10, 100 or even 1000
files or database connections or whatever is going to make much difference
to a substantially-sized program. I'd think twice about putting a
finalizer on every object.

They don't run when you want them to, but when the JVM wants them to.

True, but not necessarily a problem.

They are not guaranteed to run at all.

Isn't the only situation where they won't run at all if the program exits?
In most cases, that renders finalization moot anyway, so i don't see this
as a big problem.

I'm not sure if there are any guarantees about finalizers and running out
of memory; i'd hope that the spec promises that all objects will be
finalized before an OOME is thrown, but i don't know if it does.

Follow Daniel Pitts's advice:

If you can do that, absolutely, this is the right thing to do. But there
are times when the lifetime of an object isn't tied to a scope you can put
a try-finally round, or to any other explicit actions you can tie a
close() call to. In those cases, your choices are to use finalizers or
reference queues, or to leak.

When one must use finalizers there are ways to minimize the impact, for
example to use a helper class that has a non-trivial 'finalize()' that
another class composes, and to release the reference to the finalizable
member before releasing the containing instance.

I'm not sure i quite understand. Is this for situations where there are
other objects hanging off the outer class, so you don't get them sucked
into finalization limbo?

tom

 

[Tom Anderson]

And Lew, this is especially for you:

http://weblogs.java.net/blog/tball/archive/2005/08/finally_a_good.html

The evils of finalizers, AND error logging!

tom

 

[Stefan Ram]

http://weblogs.java.net/blog/tball/archive/2005/08/finally_a_good.html

Tom B. wrote:

»The java.io documentation, for example, can warn, argue,
even plead that developers close file streams after using
them, but it cannot enforce that requirement.«

I am not aware of a class or interface »java.io.FileStream«,
but he might think of something like »java.io.FileInputStream«.

Its documentation doesn't even try to warn, argue or plead.
It does not even recommend to close! It just states
about »close«:

»Closes this file input stream and releases any
system resources associated with the stream.«

http://download.java.net/jdk7/docs/api/java/io/FileInputStream.html

I was always vexed by the fact that this documentation does
not even suggest to use »close«, but merely states what it does.
So it is far from warning, arguing, or pleading.

Possibly relevant to this thread, this page also has
documentation for the finalizer of this class:

»protected void finalize() throws IOException

Ensures that the close method of this file input stream is
called when there are no more references to it.«

So, where do you write the »catch« clause for a
java.io.IOException thrown by a finalizer?

(If this is difficult, that would be another reason for
an explicit call to »close«.)

 

[Lew]

And Lew, this is especially for you:

http://weblogs.java.net/blog/tball/archive/2005/08/finally_a_good.html

The evils of finalizers, AND error logging!

That article suggests heavy use of finalizers, which according to
everything I've read would cause slowdowns in the application.
Whether those slowdowns are significant is another question. I think
I'll stick with Mr. Bloch's advice and only use 'finalize()'
sparingly.

Lew:

Tom:
Isn't the only situation where they won't run at all if the program exits?

Not in principle, no. If there isn't enough pressure on memory to
force a collection after the Java object is dereferenced, then
'finalize()' will not happen.

OK, yes, because then by definition the program will exit without
running 'finalize()'.

Even if it does eventually run, it may have been delayed long enough
to put pressure on other resources. For example, if 'finalize()'
releases a resource connection, it may take so long to get around to
'finalize()' that the program runs out of available connections.

If 'finalize()' does get called in time to release the resource, it
still can cause trouble. Because it takes at least two GC cycles to
release a finalizable object, it puts extra pressure on memory.

Things are further complicated by the scenario proposed by the OP,
where additional objects are referenced from the logic. Doing that
from 'finalize()' can cause weirdness.

Lew:

When one must use finalizers there are ways to minimize the impact, for
example to use a helper class that has a non-trivial 'finalize()' that
another class composes, and to release the reference to the finalizable
member before releasing the containing instance.

Tom:
I'm not sure i [sic] quite understand. Is this for situations where there are
other objects hanging off the outer class, so you don't get them sucked
into finalization limbo?

Yes. Not the 'outer' class, but the finalizable one.

If a finalizable object holds a lot of references, their GC is
delayed, too.

This is all out there in the literature. That's where I got the
information, and maybe I don't remember it as well as I should. GIYF.

 

[Tom Anderson]

Possibly relevant to this thread, this page also has
documentation for the finalizer of this class:

»protected void finalize() throws IOException

Ensures that the close method of this file input stream is
called when there are no more references to it.«

So, where do you write the »catch« clause for a
java.io.IOException thrown by a finalizer?

You don't - it gets eaten by the VM. It would be nice if the VM at least
printed or logged it when that happened (as it does when a thread dies),
but i don't think it does.

tom

 

[Tom Anderson]

That article suggests heavy use of finalizers,

I wouldn't say 'heavy' - just in places where cleanup absolutely has to be
done. I don't think there are many places like that in an application.

Bear in mind that the alternative to using finalizers for this is to risk
the cleanup not being done, which is incorrect behaviour.

Lew:

Not in principle, no. If there isn't enough pressure on memory to
force a collection after the Java object is dereferenced, then
'finalize()' will not happen.

OK, yes, because then by definition the program will exit without
running 'finalize()'.

Gotcha!

I guess there's no useful difference between 'never' and 'not for an
arbitrarily long time', so i take your point.

Even if it does eventually run, it may have been delayed long enough to
put pressure on other resources. For example, if 'finalize()' releases
a resource connection, it may take so long to get around to 'finalize()'
that the program runs out of available connections.

That's a good point. But this is a reason not to use finalizers as a
general cleanup mechanism, but not a reason not to use them as a backup -
in fact, quite the opposite. If you didn't have the finalizers, and the
code was failing to close all connections properly, you'd run out of
connections with no ability to do anything about it. If the finalizers are
there, the connections will be closed provided that finalization runs.

What alternatives are there? The only one i can think of is using a
timeout, so that if a connection has been idle for a certain length of
time, it gets released. This has the advantage of guaranteeing that
resources will be released within a certain amount of time, but suffers
from the disadvantage of potentially releasing resources while they're
still in use.

Ideally, the code handling the interaction with the resource would hold on
to enough lightweight state to be able to transparently reopen the
connection if it was needed again, so the calling code never got a
SorryIThoughtYouUsingThatAndGotRidOfItException. I once worked on a CORBA
implementation that did this - it had a 'smart reconnect' feature whereby
if the socket was closed, for instance by an error, there was enough
information in the object references to reopen it. In fact, it went one
better, and stashed information from the CosNaming or CosTrading lookup
originally used to find the object, so if the server hosting the object
actually went away, it could find you a substitute object from somewhere
else!

tom

 

[Mark Space]

That's a good point. But this is a reason not to use finalizers as a
general cleanup mechanism, but not a reason not to use them as a backup
- in fact, quite the opposite. If you didn't have the finalizers, and

I think you're making the mistake of thinking of finalizers as
destructor. The right way to do this is just to close the object. You
don't need finalizers at all if you aren't using weak references. Just
close the object.

What alternatives are there? The only one i can think of is using a
timeout, so that if a connection has been idle for a certain length of

Try-catch-finally. The "finally" portion is always run.

 

[Tom Anderson]

I think you're making the mistake of thinking of finalizers as
destructor.

No, i'm not. I'm quite explicitly not. As i took pains to point out in my
post.

The right way to do this is just to close the object.

Yes.

And if the client code forgets to close the object? Then what? You just
leak the native resource? Please tell me whether you'd be happy to let
that happen, and if not, how you'd present it. Remember - this is if the
client code forgets to explicitly close.

You don't need finalizers at all if you aren't using weak references.

Eh? Finalizers and weak references are basically completely unrelated. In
fact, to an extent, they're alternatives, so if you *are* using weak
references, you don't need finalizers!

Just close the object.

The whole point of my post was about dealing with the case where the
object isn't or can't be explicitly closed.

Try-catch-finally. The "finally" portion is always run.

You didn't read what i wrote. There are times when the lifetime of an
object is not tied to a lexical scope. In that situation, you *cannot* use
a try-finally block. It is *not possible*.

Maybe you don't understand how this can be so. Let's try an example.

Consider a server for doing online hotel booking. The application is all
about showing the user a list of hotels, in the same city or whatever, and
letting them compare rooms and prices, then letting them make a choice and
book a room (or several rooms, etc).

The actual hotel booking is handled by a legacy mainframe app, with which
you communicate using a native library. The library lets you open a handle
on a particular hotel, make queries and reservations for that hotel using
the handle, and close the handle. You can only have one handle to a given
hotel open at once. Opening handles is expensive. Use of a handle is
threadsafe. You wrap the handle in a class, which has methods to interact
with it, and a close method. So far so good.

Now you write the client-facing part of the code. There are servlets or
whatever out front, and then a layer of business objects. You have a
session object (could be a stateful EJB, could just be a pojo) which holds
the conversational state. Part of that state is a set of handles to the
hotels which are being looked at. Handles are expensive, but threadsafe,
so you share the handle for a given hotel between all the client sessions
which are looking at that hotel. The session object has business methods
which are called by the servlet layer when user requests come in, and
return with responses. Sessions are created and destroyed automatically by
the servlet container in response to users coming and going.

I'm happy to ignore handle creation for now, but here's the question: how
do you deal with the closing of handles? Where, pray tell, do you put your
try-finally block?

tom

 

[Stefan Ram]

You wrap the handle in a class, which has methods to interact
with it, and a close method. So far so good. you share the
handle for a given hotel between all the client sessions which
are looking at that hotel. how do you deal with the closing of
handles?

I assume that eventually a session will have a termination
event, either when the customers has logged out or when the
session has timed out.

The session has a list of all handle objects it has requested
so far. Upon session termination, all handle object from this
list get a release call by the termination event handler of
this session object.

Since these handles are shared between multiple session, they
keep a reference count, which is decremented upon a release call.

When a hotel access handle is requested by a session, the
handle manager object will look for it in his list of active
handle objects. If it is found there, it will he used. It will
be given to the session, and its reference count will be
incremented by 1.

If a handle object for a hotel can not be found, a new handle
object for this hotel will be created. Its reference count
will be set to 1, and it will be given to the session.

If a new handle object cannot be created because the resources
for handles are exhausted, the manager object will look for
handles with a reference count smaller than 1 and will dispose
them until the new handle object can be created.

If still no new handle object can be created, the manager will
report failure to its caller. The session will display »Sorry,
try again later.« to the user.

 

[Mark Space]

I'm happy to ignore handle creation for now, but here's the question:
how do you deal with the closing of handles? Where, pray tell, do you
put your try-finally block?

A slightly larger issue is, what if your front end code grabs a handle,
stuffs it in a session or application object, and never gives it up?
You'd be obliged to maintain that handle forever, and you'd have a
de-facto resource leak.

So I wouldn't give the handles out at all. I'd keep them in my CRUD/ORM
whatever object, and give the client something else, that wouldn't
matter if the client abuses it or forget to close it. Don't give state
to your clients, it's bad.

This also lets you handle errors more cleanly. What if your legacy
database coughs up a hairball right when the client needs to use it?
Legacy apps don't always have the nicest error handling characteristics.
Separating the two concerns (an object for your clients, and one for
your connection) lets you deal with events like that independently.

Of course, I'm not actually writing your app, so I don't know your
requirements, but that's the first thing that occurs to me. Bogart the
handles, keep your client-side connection object separate. Just hand
the client the data, for example, and some sort of key it can check back
in with. Never pass around a reference, pointer or handle from a lower
level layer to an upper level one.

 

[Lasse Reichstein Nielsen]

I'm happy to ignore handle creation for now, but here's the question:
how do you deal with the closing of handles?

You call the close method. The more relevant question is: *When* should
you close the handle?
If you can answer that, then that is where the close method should be
called. If you can't give a clear answer, then that is the problem.

Where, pray tell, do you put your try-finally block?

You don't. The lifetime isn't lexical. However, the lifetime is still
managed in some way. It is chosen somewhere that a handle is no longer
worth caching.


Anyway, if it happens to be the best solution to simply let a reference
go out of scope, I'd use a reference queue to catch the weak reference
and close it if necessary.
/L

 

[Mike Schilling]

Sure, but "hugely expensive" is hyperbole. Do we have any actual
numbers on this? I can't imagine that using finalizers on 10, 100 or
even 1000 files or database connections or whatever is going to make
much difference to a substantially-sized program. I'd think twice
about putting a finalizer on every object.

We recently observed that what seemed like an innocent addition of a
small finalize() method to a class of large objects severely impacted
memory usage. With a finalizer defined, the object's memory can't be
reclaimed as quickly, since it has to stay around until the finalizer
thread can run. Admittedly, this was in a test run where the CPU was
kept arterially busy, but the results were enough to make us find a
different approach.

I'm not sure if there are any guarantees about finalizers and
running
out of memory; i'd hope that the spec promises that all objects will
be finalized before an OOME is thrown, but i don't know if it does.

Suppose the finalizer tries to allocate memory?

 

[Daniel Pitts]

We recently observed that what seemed like an innocent addition of a
small finalize() method to a class of large objects severely impacted
memory usage. With a finalizer defined, the object's memory can't be
reclaimed as quickly, since it has to stay around until the finalizer
thread can run. Admittedly, this was in a test run where the CPU was
kept arterially busy, but the results were enough to make us find a
different approach.


Suppose the finalizer tries to allocate memory?

Or suppose the finalizer leaks a hard-reference of itself?

 

[Tom Anderson]

I assume that eventually a session will have a termination
event, either when the customers has logged out or when the
session has timed out.

The session has a list of all handle objects it has requested
so far. Upon session termination, all handle object from this
list get a release call by the termination event handler of
this session object.

Since these handles are shared between multiple session, they
keep a reference count, which is decremented upon a release call.

Yes. Reference counting is a good solution to this problem, probably the
best one.

And it's a solution which doesn't involve try-finally.

tom

 

[Tom Anderson]

You call the close method. The more relevant question is: *When* should
you close the handle?

Yes, that's what i meant with "deal with the closing of handles".

If you can answer that, then that is where the close method should be
called. If you can't give a clear answer, then that is the problem.

Yes, that is the problem. I'm interested in hearing about solutions.

You don't. The lifetime isn't lexical.

EXACTLY! This is the point i was trying to make.

However, the lifetime is still managed in some way. It is chosen
somewhere that a handle is no longer worth caching.

Ah, but where that "somewhere" is is the difficult bit.

Anyway, if it happens to be the best solution to simply let a reference
go out of scope, I'd use a reference queue to catch the weak reference
and close it if necessary.

Okay, sounds fine to me.

tom