find memory leaks in running program

Discussion in 'Python' started by Marco Hornung, Dec 7, 2010.

  1. Hey,

    ------------------------------------------------------------------------------------------
    questions
    ------------------------------------------------------------------------------------------
    1. What are the best tools to analyze pythons memory stack, while it is running?
    2. Is there a possibility to analyze the memory stack of a program with external programs? (without to change the source code - I am only interested in the object size)
    3. Can I sort of "break" into the memory to see what objects consume how much memory?

    ------------------------------------------------------------------------------------------
    my scenario
    ------------------------------------------------------------------------------------------
    I have to debug a multithreaded server, which is written in the twisted-framework. One of the processes has some sort of memory leak - After one of our jobs is finished the main process has still over 59% of the entire memory allocated. I will probably have to recreate our scenario and equip our server with some memory sensors - but it takes 12h to reproduce the scenario and I will have to change the source code(at least I do not know of other options).
    Therefore I am looking for quicker possibilities to look into what causes our memory leak.

    Regards,
    Marco
    Marco Hornung, Dec 7, 2010
    #1
    1. Advertising

  2. Marco Hornung

    shearichard Guest

    On Dec 8, 5:51 am, Marco Hornung <> wrote:
    > Hey,
    >
    > ------------------------------------------------------------------------------------------
    > questions
    > ------------------------------------------------------------------------------------------
    > 1. What are the best tools to analyze pythons memory stack, while it is running?
    > 2. Is there a possibility to analyze the memory stack of a program with external programs? (without to change the source code - I am only interested in the object size)
    > 3. Can I sort of "break" into the memory to see what objects consume how much memory?
    >
    > ------------------------------------------------------------------------------------------
    > my scenario
    > ------------------------------------------------------------------------------------------
    > I have to debug a multithreaded server, which is written in the twisted-framework. One of the processes has some sort of memory leak - After one of our jobs is finished the main process has still over 59% of the entire memory allocated. I will probably have to recreate our scenario and equip our server with some memory sensors - but  it takes 12h to reproduce the scenario and I will have to change the source code(at least I do not know of other options).
    > Therefore I am looking for quicker possibilities to look into what causes our memory leak.
    >
    > Regards,
    > Marco


    I haven't used this myself but sometime ago I bookmarked yappi which
    may go some way towards helping you ...

    http://code.google.com/p/yappi/
    http://code.google.com/p/yappi/wiki/apiyappi

    .... also sometime ago I was at presentation (Pycon NZ 2009) where the
    twisted.manhole functionality was used to hook a remote console up to
    a running twisted task and examine the interior of the target task.
    Can't remember the details but someone else here may be able to help.
    This touches upon the idea ...

    http://stackoverflow.com/questions/1721699/is-there-any-remote-console-for-twisted-server

    .... and in turn references Heapy which I'd forgotten about but may
    also be useful in your circs ...

    http://guppy-pe.sourceforge.net/

    regards

    Richard.
    shearichard, Dec 7, 2010
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Powell
    Replies:
    12
    Views:
    727
    Phil Powell
    Feb 1, 2004
  2. Divick

    How to find memory leaks?

    Divick, Sep 19, 2005, in forum: C++
    Replies:
    3
    Views:
    356
    Divick
    Sep 21, 2005
  3. Replies:
    19
    Views:
    541
    Alf P. Steinbach
    Jan 30, 2008
  4. Toby DiPasquale
    Replies:
    4
    Views:
    211
    Booker C. Bense
    Mar 17, 2006
  5. Replies:
    4
    Views:
    107
Loading...

Share This Page