Instance method decorator garbage collection problem

[John Reid]

Hi,

I've written a decorator that prints exceptions and I'm having some
trouble with garbage collection.

My decorator is:

import sys
def print_exception_decorator(fn):
def decorator(self, *args, **kwds):
try:
return fn(*args, **kwds)
except:
print 'Exception:', sys.exc_info()
raise
return decorator



The class I want to decorate the methods of is:

class InstanceCounted(object):
"A class that keeps track of how many instances there are."
count = 0
def __init__(self):
InstanceCounted.count += 1
def __del__(self):
InstanceCounted.count -= 1

class A(InstanceCounted):
"A class that I want to decorate a method on."
def __init__(self):
super(A, self).__init__()
self.method = print_exception_decorator(self.method)

def __del__(self):
del self.method

def method(self):
pass



When I run the following it does not seem like my object 'a' is garbage
collected:

print 'Have %d instances' % InstanceCounted.count
print 'Creating A'
a = A()
print 'Have %d instances' % InstanceCounted.count
print 'Deleting A'
del a
print 'Have %d instances' % InstanceCounted.count


This is the output:

Have 0 instances
Creating A
Have 1 instances
Deleting A
Have 1 instances


The InstanceCounted.count is 1 at the end. If I omit the call to
"self.method = print_exception_decorator(self.method)" then the instance
count goes down to 0 as desired. I thought that the decorator might be
holding a reference to the instance through the bound method, so I added
the __del__() but it doesn't fix the problem.

Can anyone suggest anything? Is my technique to decorate bound methods
not a good one? How else should I decorate a bound method?

Thanks in advance,
John.

 

[Peter Otten]

Hi,

I've written a decorator that prints exceptions and I'm having some
trouble with garbage collection.

My decorator is:

import sys
def print_exception_decorator(fn):
def decorator(self, *args, **kwds):
try:
return fn(*args, **kwds)
except:
print 'Exception:', sys.exc_info()
raise
return decorator



The class I want to decorate the methods of is:

class InstanceCounted(object):
"A class that keeps track of how many instances there are."
count = 0
def __init__(self):
InstanceCounted.count += 1
def __del__(self):
InstanceCounted.count -= 1

class A(InstanceCounted):
"A class that I want to decorate a method on."
def __init__(self):
super(A, self).__init__()
self.method = print_exception_decorator(self.method)

def __del__(self):
del self.method

def method(self):
pass



When I run the following it does not seem like my object 'a' is garbage
collected:

print 'Have %d instances' % InstanceCounted.count
print 'Creating A'
a = A()
print 'Have %d instances' % InstanceCounted.count
print 'Deleting A'
del a
print 'Have %d instances' % InstanceCounted.count


This is the output:

Have 0 instances
Creating A
Have 1 instances
Deleting A
Have 1 instances


The InstanceCounted.count is 1 at the end. If I omit the call to
"self.method = print_exception_decorator(self.method)" then the instance
count goes down to 0 as desired. I thought that the decorator might be
holding a reference to the instance through the bound method, so I added
the __del__() but it doesn't fix the problem.

Can anyone suggest anything? Is my technique to decorate bound methods
not a good one? How else should I decorate a bound method?

The problem is that cyclic garbage collection cannot cope with __del__()
methods. Quoting http://docs.python.org/library/gc.html#gc.garbage

"""
Objects that have __del__() methods and are part of a reference cycle cause
the entire reference cycle to be uncollectable
"""

The best workaround is to control the object's lifetime explicitly by
turning it into a context manager, see

http://docs.python.org/reference/datamodel.html#with-statement-context-
managers

or providing a close() method and use contextlib:
.... def close(self):
.... print "break cycles here"
........ def __init__(self):
.... self.self = self
.... def __del__(self):
.... print "bye"
.... def close(self):
.... print "breaking cycles here"
.... del self.self
........ print "using b"
....
using b
breaking cycles herebye

Peter