Re: Debugging SocketServer.ThreadingTCPServer

Discussion in 'Python' started by Jean-Paul Calderone, Jan 16, 2007.

  1. On Tue, 16 Jan 2007 00:23:35 -0500, "Stuart D. Gathman" <> wrote:
    >I have a ThreadingTCPServer application (pygossip, part of
    >http://sourceforge.net/projects/pymilter). It mostly runs well, but
    >occasionally goes into a loop. How can I get a stack trace of running
    >threads to figure out where the loop is? Is there some equivalent of
    >sending SIGQUIT to Java to get a thread dump? If needed, I can import pdb
    >and set options at startup, but there needs to be some external way of
    >triggering the dump since I can't reproduce it at will.


    Grab the gdbinit out of Python SVN Misc/ directory. Apply this patch:

    http://jcalderone.livejournal.com/28224.html

    Attach to the process using gdb. Make sure you have debugging symbols
    in your build of Python. Run 'thread apply all pystack'.

    Jean-Paul
    Jean-Paul Calderone, Jan 16, 2007
    #1
    1. Advertising

  2. On Tue, 16 Jan 2007 09:11:38 -0500, Jean-Paul Calderone wrote:

    > On Tue, 16 Jan 2007 00:23:35 -0500, "Stuart D. Gathman"
    > <> wrote:
    >>I have a ThreadingTCPServer application (pygossip, part of
    >>http://sourceforge.net/projects/pymilter). It mostly runs well, but
    >>occasionally goes into a loop. How can I get a stack trace of running
    >>threads to figure out where the loop is? Is there some equivalent of
    >>sending SIGQUIT to Java to get a thread dump? If needed, I can import
    >>pdb and set options at startup, but there needs to be some external way
    >>of triggering the dump since I can't reproduce it at will.

    >
    > Grab the gdbinit out of Python SVN Misc/ directory. Apply this patch:
    >
    > http://jcalderone.livejournal.com/28224.html
    >
    > Attach to the process using gdb. Make sure you have debugging symbols
    > in your build of Python. Run 'thread apply all pystack'.


    Did this. gdb displays main thread fine (waiting on accept(), duh). But
    gdb goes into a loop displaying the first worker thread. There are no
    extension modules other than the batteries included ones. In this
    application, I believe, only _socket. (I.e. a pure python server.)

    I will try for a C stack trace next time it loops.

    Also, the looping server needs kill -9. SIGTERM and SIGINT won't stop it.
    And after it dies with SIGKILL, the port is still in use for 5 minutes or
    so (and the server cannot be restarted).

    This is really making me appreciate Java.

    --
    Stuart D. Gathman <>
    Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
    "Confutatis maledictis, flamis acribus addictis" - background song for
    a Microsoft sponsored "Where do you want to go from here?" commercial.
    Stuart D. Gathman, Feb 3, 2007
    #2
    1. Advertising

  3. Jean-Paul Calderone

    R. Bernstein Guest

    "Stuart D. Gathman" <> writes:

    > On Tue, 16 Jan 2007 09:11:38 -0500, Jean-Paul Calderone wrote:
    >
    > > On Tue, 16 Jan 2007 00:23:35 -0500, "Stuart D. Gathman"
    > > <> wrote:
    > >>I have a ThreadingTCPServer application (pygossip, part of
    > >>http://sourceforge.net/projects/pymilter). It mostly runs well, but
    > >>occasionally goes into a loop. How can I get a stack trace of running
    > >>threads to figure out where the loop is? Is there some equivalent of
    > >>sending SIGQUIT to Java to get a thread dump? If needed, I can import
    > >>pdb and set options at startup, but there needs to be some external way
    > >>of triggering the dump since I can't reproduce it at will.


    The problem here is that signals are handled only in the main
    thread. If that thread is blocked for some reason, your signals will
    also be blocked.

    Given that you were considering using pdb let me suggest instead pydb
    - http://bashdb.sf.net/pydb; pdb has no notion of threads but pydb
    does if you give it the --threading option. (It's thread support might
    stand a bit of improvement, but at least it's there and is as good or
    better than any other Python debugger that I am aware of.)

    > >
    > > Grab the gdbinit out of Python SVN Misc/ directory. Apply this patch:
    > >
    > > http://jcalderone.livejournal.com/28224.html
    > >
    > > Attach to the process using gdb. Make sure you have debugging symbols
    > > in your build of Python. Run 'thread apply all pystack'.

    >
    > Did this. gdb displays main thread fine (waiting on accept(), duh). But
    > gdb goes into a loop displaying the first worker thread. There are no
    > extension modules other than the batteries included ones. In this
    > application, I believe, only _socket. (I.e. a pure python server.)
    >
    > I will try for a C stack trace next time it loops.
    >
    > Also, the looping server needs kill -9. SIGTERM and SIGINT won't stop it.
    > And after it dies with SIGKILL, the port is still in use for 5 minutes or
    > so (and the server cannot be restarted).


    See again above why SIGTERM and SIGINT might not necessarily do
    anything. But if you get into pydb with thread support, at least you
    should be able to determine if you are blocked in the main thread; so
    you can know if a SIGTERM or SIGINT will be seen.

    And in pydb one can kill specific threads, but you do this at
    your own risk because you could cause a deadlock here. As a
    convenience, there is a debugger "kill" command so you don't have to
    remember the pid. And that's there because it is really the only
    reliable way I know how to terminate the program. There is also a
    "signal" debugger command if you wanted to issue SIGTERM or SIGINT
    signals.

    >
    > This is really making me appreciate Java.


    Note that Java has preemptive threads, Python does not.
    R. Bernstein, Feb 4, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. UrgeOverkill
    Replies:
    2
    Views:
    435
    Peter Hansen
    Feb 19, 2004
  2. Dustin
    Replies:
    2
    Views:
    1,032
    G. S. Hayes
    Aug 9, 2004
  3. Stuart D. Gathman

    Debugging SocketServer.ThreadingTCPServer

    Stuart D. Gathman, Jan 16, 2007, in forum: Python
    Replies:
    0
    Views:
    322
    Stuart D. Gathman
    Jan 16, 2007
  4. Brad Brock
    Replies:
    0
    Views:
    330
    Brad Brock
    May 19, 2007
  5. Okko Willeboordse
    Replies:
    0
    Views:
    422
    Okko Willeboordse
    Nov 9, 2008
Loading...

Share This Page