gc.get_referrers trouble

[Diez B. Roggisch]

Hi,

I'm in the process of debugging a mem-leaking app. A recent thread here on
c.l.py mentioned the use of the gc module, so I gave it a try.

However, using get_referrers() yields in literally thousands of objects for
even the tiniest example like this:

import gc
a = [1]
print gc.get_referrers(1)

Running that from bash gives me this:

# python /tmp/test.py | wc
23 7196 67165

Now I'm confused - how do I interpret the results of get_referrers()
correctly?

 

[Jeremy Bowers]

Hi,

I'm in the process of debugging a mem-leaking app. A recent thread here on
c.l.py mentioned the use of the gc module, so I gave it a try.

However, using get_referrers() yields in literally thousands of objects for
even the tiniest example like this:

import gc
a = [1]
print gc.get_referrers(1)

For space reasons, small ints are re-used. You get thousands of referrers
to the number 1 because, well, there are!

Try this:

import gc
a = [4444444444444]
print gc.get_referrers(4444444444444)


and you'll get something much more reasonable:

[(None, 4444444444444L), [4444444444444L]]

 

[Skip Montanaro]

Diez> import gc
Diez> a = [1]
Diez> print gc.get_referrers(1)

Diez> Running that from bash gives me this:

Diez> # python /tmp/test.py | wc
Diez> 23 7196 67165

Diez> Now I'm confused - how do I interpret the results of
Diez> get_referrers() correctly?

Try your test with an object that's not massively shared:

% python
Python 2.4a2 (#46, Aug 29 2004, 08:53:23)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import>>> import gc

>>> a = [1000]
>>> print gc.get_referrers(1000)

[(1000, None)]

CPython maintains a list of small integers (range(-1, 100) I believe) that
are reused and shared.

Skip

 

[Tim Peters]

[Diez B. Roggisch]

I'm in the process of debugging a mem-leaking app. A recent thread here on
c.l.py mentioned the use of the gc module, so I gave it a try.

However, using get_referrers() yields in literally thousands of objects for
even the tiniest example like this:

import gc
a = [1]
print gc.get_referrers(1)

Running that from bash gives me this:

# python /tmp/test.py | wc
23 7196 67165

That just shows the number of lines (etc) of output, not how many
objects refer to 1. len(gc.get_referrers(1)) would tell you how many
container objects refer to 1. Little integers happen to be shared in
CPython, so there are likely to be referrers you don't know about. I
expect there are several dozen objects total that refer to 1, but not
more than that. Pick a larger number to match your "intuition"
better:

Now I'm confused - how do I interpret the results of get_referrers()
correctly?

Start by not printing it as a string <wink>. It's a list of container
objects. The string representation of any one of those objects can
arbitrarily large.

A better start would be to download one of the test.py scripts for
Zope, and use a debug build. There's a TrackRefs class defined in
test.py that displays deltas in object counts (by type) across calls
to its update() method. Typical is to use that to find out which
kinds of objects are growing at an unreasonable rate, then fiddle the
guts of the TrackRefs class (via calls to things like gc.get_referrers
-- depends on where the evidence leads you) to zero in on a cause.