memory leak troubleshooting techniques

[Adam Deutsch]

I would like to ask some advice about tracking down memory leaks in Python
code.

We have a python application running on Python 2.0.1 in an embedded Linux
environment (kernel version 2.4.7). We have recently detected a memory leak
that we can see from "ps aux" is attributable to the Python processes.
Running Sam Rushing's (http://www.nightmare.com/medusa/memory-leaks.html)
get_refcounts() function does not yield any smoking guns: the reference
counts for all classes remain stable even as memory use climbs.

Is there any systematic method of determining how Python is using its
allocated memory?

Thanks.

 

[Martin v. =?iso-8859-15?q?L=F6wis?=]

Is there any systematic method of determining how Python is using its
allocated memory?

In the debug version, you can get a list of all objects through
sys.getobjects. In the non-debug version, you can get a list of all
container objects through gc.get_objects. Check whether these lists
are growing in size.

Regards,
Martin

 

[Christoph Becker-Freyseng]

I would like to ask some advice about tracking down memory leaks in Python
code.

We have a python application running on Python 2.0.1 in an embedded Linux
environment (kernel version 2.4.7). We have recently detected a memory leak
that we can see from "ps aux" is attributable to the Python processes.
Running Sam Rushing's (http://www.nightmare.com/medusa/memory-leaks.html)
get_refcounts() function does not yield any smoking guns: the reference
counts for all classes remain stable even as memory use climbs.

Is there any systematic method of determining how Python is using its
allocated memory?

Thanks.

There might be one more readon for a memory-leak -- besides an
increasing refcount.

Have you thought about a growing string? (or an similar object)
In Python a string can have 80 MB without problems / not "hurting" python.
Of course the refount won't get higher as there's just one string -- but
this one might be getting *huge*.
I had such a problem when programming PersistentThreads. Making internal
calls to the "ZopeRequestDispatcher" caused sys.path to grow.

Unfortunately I don't know a way in Python to list objects that consume
more than xy bytes of memory.


Christoph Becker-Freyseng