finalize called on an object that's still in scope?

[Paul Tomblin]

I've got some code that looks sort of like

LineupProxy lineup = new LineupProxySubClass();
lineup.connect();
while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie) {}}
}

It's supposed to run forever. Methods within the LineupProxy get called
through RMI and it just sits there handling events. But last night, for
no reason that I can fathom, the LineupProxySubClass.finalize() method got
called, which did bad things. We've just started using 1.5_011, up from
1.4, and I don't think it ever happened in 1.4 although we have our
suspicious about some other odd happenings. Is this a known bug in 1.5?

 

[Andrew Thompson]

Paul Tomblin wrote:
...

...Is this a known bug in 1.5?

Best way to find out, is to search the bug database.
<http://bugs.sun.com/bugdatabase/index.jsp>

 

[Tom Hawtin]

I've got some code that looks sort of like

LineupProxy lineup = new LineupProxySubClass();
lineup.connect();
while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie) {}}
}

It's supposed to run forever. Methods within the LineupProxy get called
through RMI and it just sits there handling events. But last night, for
no reason that I can fathom, the LineupProxySubClass.finalize() method got
called, which did bad things. We've just started using 1.5_011, up from
1.4, and I don't think it ever happened in 1.4 although we have our
suspicious about some other odd happenings. Is this a known bug in 1.5?

From a language and VM spec point of view, that looks entirely legal.

The object is not reachable, therefore it can be finalised.

From a practical point of view, I'm not entirely sure what is happening
here. Perhaps lineup and ie share the same local variable slot. You are
sleeping for 5 seconds at a time. So possibly the method may get
compiled after a few hours. The obvious fix is to assign lineup to a
volatile variable every time around the loop, but that's just an evil
hack for deeper design issues.

But it's technically worse than that. You need a happens-before edge to
make sure your last use happens-before the finaliser is called, which is
not automatic. Construction is finished before finalisation, but not
random methods.

The bottom line is that finalisers are difficult, therefore avoid them.

Tom Hawtin

 

[Paul Tomblin]

In a previous article, (e-mail address removed) (Paul Tomblin) said:

I've got some code that looks sort of like

LineupProxy lineup = new LineupProxySubClass();
lineup.connect();
while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie) {}}
}

It's supposed to run forever. Methods within the LineupProxy get called
through RMI and it just sits there handling events. But last night, for
no reason that I can fathom, the LineupProxySubClass.finalize() method got
called, which did bad things. We've just started using 1.5_011, up from

Oh, I should mention that I "kill -3"'ed the java process, and the thread
dump showed that it's still in the while(true) loop, so it didn't exit by
accident.

 

[Piotr Kobzda]

I've got some code that looks sort of like

LineupProxy lineup = new LineupProxySubClass();
lineup.connect();
while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie) {}}
}

To prevent your local variable from dereferencing, cause referencing it
in a loop, e.g. that way:

while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie) {}}
lineup.hashCode(); // or System.identityHashCode(), or ...
}


piotr

 

[Paul Tomblin]

To prevent your local variable from dereferencing, cause referencing it
in a loop, e.g. that way:

while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie) {}}
lineup.hashCode(); // or System.identityHashCode(), or ...
}

Is this a hack, or is it normal that a local variable would be destructed
before it goes out of scope?

 

[Tom Hawtin]

You mean from becoming unreachable. Dereferencing is what you are
proposing to do to it.

http://www.google.co.uk/search?q=define:dereference

That doesn't work, technically.

Is this a hack, or is it normal that a local variable would be destructed
before it goes out of scope?

Destructed is a C++ term. It's probably a bad idea not to think of Java
finalising and destructing, because they are very different concepts. A
finaliser is not the opposite of a constructor.

The local variable is not finalised. The object that was once referred
to by the local variable is (or may be) finalised. Scope doesn't
actually have anything to do with it at all.

Tom Hawtin

 

[Paul Tomblin]

Destructed is a C++ term. It's probably a bad idea not to think of Java
finalising and destructing, because they are very different concepts. A
finaliser is not the opposite of a constructor.

The local variable is not finalised. The object that was once referred
to by the local variable is (or may be) finalised. Scope doesn't
actually have anything to do with it at all.

But the local variable is still in scope, so therefore is still referring
to it. I would not expect the object to be garbage
collected/finalized/destructed and if it is, then there is something
seriously wrong happening here.

 

[Piotr Kobzda]

You mean from becoming unreachable.

You right!

Dereferencing is what you are
proposing to do to it.

http://www.google.co.uk/search?q=define:dereference

That doesn't work, technically.

Ah, I always thought dereferencing is the apposite of referencing. As
decompilation is the opposite of compilation, etc.

Thanks Tom for correcting my (wrong) definition.


piotr

 

[Piotr Kobzda]

But the local variable is still in scope, so therefore is still referring
to it. I would not expect the object to be garbage
collected/finalized/destructed and if it is, then there is something
seriously wrong happening here.

Nothing is wrong here. Each local lives (as a reference) in a local
stack frame space, which (space) is a subject to reuse whenever allowed
by language rules. So, the last use of a local (when it becomes
unreachable), is also a last time of guaranteed referencing of it from
the stack, and it's referenced usually as long as some next operation
replaces its stack space with another reference (in simple methods
though, it may not happen at all before method end).

See JLS3 12.6.1 for details on objects reachability, and what optimizing
transformations of a program are allowed.

Short quote:
"For example, a compiler or code generator may choose to set a variable
or parameter that will no longer be used to null to cause the storage
for such an object to be potentially reclaimable sooner."


piotr

 

[Paul Tomblin]

Nothing is wrong here. Each local lives (as a reference) in a local
stack frame space, which (space) is a subject to reuse whenever allowed
by language rules. So, the last use of a local (when it becomes
unreachable), is also a last time of guaranteed referencing of it from
the stack, and it's referenced usually as long as some next operation
replaces its stack space with another reference (in simple methods
though, it may not happen at all before method end).

Ah, ok. I didn't realize that Java did that sort of optimization. I
still think in terms of scope rules. If I put a bogus use of the local
variable after the while(true) loop, that should prevent that, right?

 

[Patricia Shanahan]

But the local variable is still in scope, so therefore is still referring
to it. I would not expect the object to be garbage
collected/finalized/destructed and if it is, then there is something
seriously wrong happening here.

The rules about finalization say nothing about variable scope. The
question is whether the object in question is "reachable":

"A reachable object is any object that can be accessed in any potential
continuing computation from any live thread. Optimizing transformations
of a program can be designed that reduce the number of objects that are
reachable to be less than those which would naively be considered
reachable. For example, a compiler or code generator may choose to set a
variable or parameter that will no longer be used to null to cause the
storage for such an object to be potentially reclaimable sooner."

[JLS, 12.6.1 Implementing Finalization,
<http://java.sun.com/docs/books/jls/third_edition/html/execution.html#12.6.1>]

This appears to me to be exactly the case in the original code. The
variable lineup will no longer be used, so the implementation has the
option of finalizing the object it referenced, assuming there are no
other references to it.

Patricia

 

[Tom Hawtin]

Ah, ok. I didn't realize that Java did that sort of optimization. I
still think in terms of scope rules. If I put a bogus use of the local
variable after the while(true) loop, that should prevent that, right?

It would have to be something that couldn't be optimised away. Trying to
guess the rules is hard.

Almost certainly you don't want to be using finalisers here.

Tom Hawtin

 

[Piotr Kobzda]

Ah, ok. I didn't realize that Java did that sort of optimization. I
still think in terms of scope rules. If I put a bogus use of the local
variable after the while(true) loop, that should prevent that, right?

Code after the while(true) loop is usually unreachable code, hence
method won't compile. In your case the right place is inside the loop
(as in my earlier suggestion).

Consider also the following approach:

while(true)
{
try { Thread.sleep(5000); } catch (InterruptedException ie)
{
lineup.hashCode();
}
}



piotr

 

[Patricia Shanahan]

It would have to be something that couldn't be optimised away. Trying to
guess the rules is hard.

Why not go by one of the rules that are actually stated in the JLS, such
as the following?

"Note that this sort of optimization is only allowed if references are
on the stack, not stored in the heap."

Changing lineup from local variable to instance field should cure the
problem.

http://java.sun.com/docs/books/jls/third_edition/html/execution.html#12.6.1

Patricia

 

[Tom Hawtin]

Why not go by one of the rules that are actually stated in the JLS, such
as the following?

"Note that this sort of optimization is only allowed if references are
on the stack, not stored in the heap."

That's quote is from a non-normative example. So treat with caution.

Changing lineup from local variable to instance field should cure the
problem.

So long as you have a reachable reference to the instance doing the
referencing. Obviously you can't rely on a local variable causing that.

I think there was a JavaOne talk on implementing finalisers a year or
two ago. It's distinctly non-obvious.

Tom Hawtin

 

[Piotr Kobzda]

It would have to be something that couldn't be optimised away. Trying to
guess the rules is hard.

Sure. Just trying to figure out what that rules might be, and how to
prevent against them, I've found the following:

...
while (true) {
try {
Thread.sleep(5000);
} catch (InterruptedException ie) {
if (ie.equals(lineup)) {
throw new AssertionError();
}
}
}

Do you think Tom, it is possible to optimize the above fragment in the
way that a lineup local is removed form a locals stack (i.e. becomes
unreachable) before a loop ends?


Anyway, Paul, consider also using another object to make a lineup
non-local for a "moment":

...
new Object() {
Object lineup_ = lineup; /* lineup must be final */
void loop() {
while (true) {
try {
Thread.sleep(5000);
} catch (InterruptedException ie) {
/* ignore */
}
}
}
}.loop();


piotr

 

[Tom Hawtin]

while (true) {
try {
Thread.sleep(5000);
} catch (InterruptedException ie) {
if (ie.equals(lineup)) {
throw new AssertionError();
}
}
}

Do you think Tom, it is possible to optimize the above fragment in the
way that a lineup local is removed form a locals stack (i.e. becomes
unreachable) before a loop ends?

I'm not sure. Are you?

Certainly I can't see the required happens-before relationship.

Tom Hawtin

 

[Piotr Kobzda]

I'm not sure. Are you?

No, I'm not. Albeit that's difficult to predict all conditions for
InterruptExeption occurrence here, that's not impossible, I think.

A clever optimizer may know, that there is no other thread allowed to
interrupt our thread; or even ensure that there is well known subset of
InterruptExeptions a sleep() my throw (and optimizer knows, that none of
them equals to lineup), or check something else... So it seams that
there is some chance for optimizing it...

However, without ensuring that, lineup must stay reachable (for possible
use in the exception handler).

The problem is, that all the above optimizations must know the code
execution environment, so I'm not sure, if that's allowed to treat them
as the transformations of a program which are really allowed to impact
reachability of objects? Are they?

Certainly I can't see the required happens-before relationship.

Well, I can't see that either. The reason, I guess, is different: I've
never really checked happens-before relationships of any code,
everything before me.

First, it seams to me, I have to define all the /reachability decision
points/ in my code...


piotr