memory leak troubleshooting techniques

Discussion in 'Python' started by Adam Deutsch, Jul 28, 2003.

  1. Adam Deutsch

    Adam Deutsch Guest

    I would like to ask some advice about tracking down memory leaks in Python
    code.

    We have a python application running on Python 2.0.1 in an embedded Linux
    environment (kernel version 2.4.7). We have recently detected a memory leak
    that we can see from "ps aux" is attributable to the Python processes.
    Running Sam Rushing's (http://www.nightmare.com/medusa/memory-leaks.html)
    get_refcounts() function does not yield any smoking guns: the reference
    counts for all classes remain stable even as memory use climbs.

    Is there any systematic method of determining how Python is using its
    allocated memory?

    Thanks.
     
    Adam Deutsch, Jul 28, 2003
    #1
    1. Advertising

  2. Adam Deutsch <> writes:

    > Is there any systematic method of determining how Python is using its
    > allocated memory?


    In the debug version, you can get a list of all objects through
    sys.getobjects. In the non-debug version, you can get a list of all
    container objects through gc.get_objects. Check whether these lists
    are growing in size.

    Regards,
    Martin
     
    Martin v. =?iso-8859-15?q?L=F6wis?=, Jul 29, 2003
    #2
    1. Advertising

  3. Adam Deutsch wrote:
    > I would like to ask some advice about tracking down memory leaks in Python
    > code.
    >
    > We have a python application running on Python 2.0.1 in an embedded Linux
    > environment (kernel version 2.4.7). We have recently detected a memory leak
    > that we can see from "ps aux" is attributable to the Python processes.
    > Running Sam Rushing's (http://www.nightmare.com/medusa/memory-leaks.html)
    > get_refcounts() function does not yield any smoking guns: the reference
    > counts for all classes remain stable even as memory use climbs.
    >
    > Is there any systematic method of determining how Python is using its
    > allocated memory?
    >
    > Thanks.
    >

    There might be one more readon for a memory-leak -- besides an
    increasing refcount.

    Have you thought about a growing string? (or an similar object)
    In Python a string can have 80 MB without problems / not "hurting" python.
    Of course the refount won't get higher as there's just one string -- but
    this one might be getting *huge*.
    I had such a problem when programming PersistentThreads. Making internal
    calls to the "ZopeRequestDispatcher" caused sys.path to grow.

    Unfortunately I don't know a way in Python to list objects that consume
    more than xy bytes of memory.


    Christoph Becker-Freyseng
     
    Christoph Becker-Freyseng, Jul 29, 2003
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. s.subbarayan

    Dynamic memory allocation and memory leak...

    s.subbarayan, Mar 18, 2005, in forum: C Programming
    Replies:
    10
    Views:
    756
    Eric Sosman
    Mar 22, 2005
  2. Chris Thomasson
    Replies:
    0
    Views:
    451
    Chris Thomasson
    Mar 3, 2007
  3. Daniel Peterson

    Troubleshooting ASP.Net memory usage issues?

    Daniel Peterson, Nov 14, 2007, in forum: ASP .Net
    Replies:
    10
    Views:
    804
    Daniel Peterson
    Nov 20, 2007
  4. cham
    Replies:
    5
    Views:
    791
  5. Mark Probert
    Replies:
    4
    Views:
    356
    Mark Probert
    Feb 9, 2005
Loading...

Share This Page