Analyzing Python GC output - what is a "cell", and what informationis available about it.

[John Nagle]

I'm printing out each entry in "gc.garbage" after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like

<cell at 0x00F7C170: function object at 0x00FDD6B0>

That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?

(I'm looking for a possible memory leak, obviously.)

John Nagle

 

[Duncan Booth]

I'm printing out each entry in "gc.garbage" after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like

<cell at 0x00F7C170: function object at 0x00FDD6B0>

That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?

Cell objects are created whenever you have a function that references a
variable in an outer scope. e.g.
x = 42
def inner():
return x
return inner

inner = outer()
inner.func_closure[0]

inner.func_closure[0].cell_contents

42


So in your case, cell.cell_contents.func_name might help.

 

[John Nagle]

Cell objects are created whenever you have a function that references a
variable in an outer scope. e.g.

So in your case, cell.cell_contents.func_name might help.

Tried that:

print repr(item).encode('ascii','replace')
print "Type:",type(item)
try:
print item.cell_contents
except Exception, message:
print "Unprintable:",message

<cell at 0x00F88DF0: function object at 0x0100CFB0>
Type: <type 'cell'>
Unprintable: 'cell' object has no attribute 'cell_contents'

So it doesn't have a "cell_contents" attribute.

Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'

Tried:
print item.__dict__
got:
'cell' object has no attribute '__dict__'

It looks like this is a low-level PyCellObject not generated from Python code.
Any ideas? I'm using the M2Crypto and MySQLdb libraries, and suspect
a reference count bug in one of those.

John Nagle

 

[Francesco Guerrieri]

Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'

I don't know nothing about cell objects...
but why don't you try dir(item) instead?

Francesco

 

[John Nagle]

It's a problem inside MySQLdb's "connections.py":

def _get_unicode_literal():
def unicode_literal(u, dummy=None):
return db.literal(u.encode(unicode_literal.charset))
return unicode_literal

Each time a MySQLdb Connection object is created, it generates a closure,
with another instance of the function "unicode_literal" plus some other
junk. That generates circular references. Eventually those
things get garbage-collected, but if you're running GC in leak detection
mode, they show up.

The reason for the closure is that "unicode_literal.charset" of the
function is being stored into from outside the function. So there
actually is a reason to use a closure.

John Nagle