Reliably call code after object no longer exists or is "unreachable"?

[Jack Bates]

In Python, how can you reliably call code - but wait until an object no
longer exists or is "unreachable"?

I want to ensure that some code is called (excluding some exotic
situations like when the program is killed by a signal not handled by
Python) but can't call it immediately. I want to wait until there are no
references to an object - or the only references to the object are from
unreachable reference cycles


#!/usr/bin/env python

class Goodbye:
def __del__(self):
print 'Goodbye, world!'

ref = Goodbye()


$ ./goodbye
Goodbye, world!
$


Python's __del__ or destructor method works (above) - but only in the
absence of reference cycles (below). An object, with a __del__ method,
in a reference cycle, causes all objects in the cycle to be
"uncollectable". This can cause memory leaks and because the object is
never collected, its __del__ method is never called

Circular references which are garbage are detected when the option
cycle detector is enabled (it's on by default), but can only be
cleaned up if there are no Python-level __del__() methods involved.

#!/usr/bin/env python

class Goodbye:
def __del__(self):
print 'Goodbye, world!'

class Cycle:
def __init__(self, cycle):
self.next = cycle
cycle.next = self

Cycle(Goodbye())


$ ./cycle
$


In PEP 342 I read that an object, with a __del__ method, referenced by a
cycle but not itself participating in the cycle, doesn't cause objects
to be uncollectable. If the cycle is "collectable" then when it's
eventually collected by the garbage collector, the __del__ method is
called

If the generator object participates in a cycle, g.__del__() may not
be called. This is the behavior of CPython's current garbage
collector. The reason for the restriction is that the GC code needs to
"break" a cycle at an arbitrary point in order to collect it, and from
then on no Python code should be allowed to see the objects that
formed the cycle, as they may be in an invalid state. Objects "hanging
off" a cycle are not subject to this restriction.

#!/usr/bin/env python

import sys

class Destruct:
def __init__(self, callback):
self.__del__ = callback

class Goodbye:
def __init__(self):
self.destruct = Destruct(lambda: sys.stdout.write('Goodbye, world!\n'))

class Cycle:
def __init__(self, cycle):
self.next = cycle
cycle.next = self

Cycle(Goodbye())


$ ./dangle
Goodbye, world!
$


However it's *extremely* tricky to ensure that the object with a __del__
method doesn't participate in a cycle, e.g. in the example below, the
__del__ method is never called - I suspect because the object with a
__del__ method is reachable from the global scope, and this forms a
cycle with a frame's f_globals reference? "storing a generator object in
a global variable creates a cycle via the generator frame's f_globals
pointer"


#!/usr/bin/env python

import sys

class Destruct:
def __init__(self, callback):
self.__del__ = callback

class Goodbye:
def __init__(self):
self.destruct = Destruct(lambda: sys.stdout.write('Goodbye, world!\n'))

class Cycle:
def __init__(self, cycle):
self.next = cycle
cycle.next = self

ref = Cycle(Goodbye())


$ ./global
$


Faced with the real potential for reference cycles, how can you reliably
call code - but wait until an object no longer exists or is
"unreachable"?

 

[Thomas 'PointedEars' Lahn]

Python's __del__ or destructor method works (above) - but only in the
absence of reference cycles (below). An object, with a __del__ method,
in a reference cycle, causes all objects in the cycle to be
"uncollectable". This can cause memory leaks and because the object is
never collected, its __del__ method is never called

AIUI there is no guarantee that __del__() will be called automatically at
all:

Faced with the real potential for reference cycles, how can you reliably
call code - but wait until an object no longer exists or is
"unreachable"?

For normal program termination, the solution is the `atexit' module (as used
e.g. in <http://code.activestate.com/recipes/523007-semi-automatic-resource-
management-with-autoclose/>, however it is considered better style to
explicitly call methods that free resources, and `del' object references.

As for abnormal program termination, I think there is no way to deal with
signals that signal.signal() cannot handle.


HTH

 

[Gregory Ewing]

Python's __del__ or destructor method works (above) - but only in the
absence of reference cycles (below). An object, with a __del__ method,
in a reference cycle, causes all objects in the cycle to be
"uncollectable".

Store a weak reference to the object somewhere with a
callback. This should work even in the presence of
cycles, as long as the weak reference itself isn't
part of the cycle.