Debugging segmentation faults

Discussion in 'Python' started by George Sakkis, Mar 7, 2007.

  1. I have a pure python program (no C extensions) that occasionally core
    dumps in a non-reproducible way. The program is started by a (non-
    python) cgi script when a form is submitted. It involves running a
    bunch of other programs through subprocess in multiple threads and
    writing its output in several files. So the only suspicious parts I
    can think of is subprocess and/or multithreading. For the
    multithreading part I'm using a modified version of threadpool.py
    (http://www.chrisarndt.de/en/software/python/threadpool/), which is
    built on top of the threading and Queue stdlib modules. Whatever bugs
    may linger there, I'd hope that they would show up as normal Python
    exceptions instead of segfaults.

    All I have now is a few not particularly insightful core files (actual
    path names and args changed for privacy):

    /home/gsakkis/foo>gdb --core core.20140
    GNU gdb Red Hat Linux (5.2-2)
    Copyright 2002 Free Software Foundation, Inc.
    GDB is free software, covered by the GNU General Public License, and
    you are
    welcome to change it and/or distribute copies of it under certain
    conditions.
    Type "show copying" to see the conditions.
    There is absolutely no warranty for GDB. Type "show warranty" for
    details.
    This GDB was configured as "i386-redhat-linux".
    Core was generated by `python2.5 /home/gsakkis/foo/foo.py --XXX --
    max=30 --bar=/tmp/83840`
    Program terminated with signal 11, Segmentation fault.
    #0 0x080b222d in ?? ()
    (gdb) backtrace
    #0 0x080b222d in ?? ()
    #1 0x080b28d1 in ?? ()
    #2 0x080fa8ab in ?? ()
    #3 0x0805c918 in ?? ()
    (...)
    #28 0x080b310f in ?? ()
    #29 0x080dbfdd in ?? ()
    #30 0x40021fef in ?? ()

    Any hints ?
    George Sakkis, Mar 7, 2007
    #1
    1. Advertising

  2. If the error is reproducable and you can run wxPython, you could use
    the excellent WinPdb debugger, which ships with SPE (python editor):
    http://www.digitalpeers.com/pythondebugger/

    .... but if you can only run your script on a remote server this won't
    help you.

    Stani

    --
    SPE - http://pythonide.stani.be

    On 7 Mrz., 20:15, "George Sakkis" <> wrote:
    > I have a pure python program (no C extensions) that occasionally core
    > dumps in a non-reproducible way. The program is started by a (non-
    > python) cgi script when a form is submitted. It involves running a
    > bunch of other programs through subprocess in multiple threads and
    > writing its output in several files. So the only suspicious parts I
    > can think of is subprocess and/or multithreading. For the
    > multithreading part I'm using a modified version of threadpool.py
    > (http://www.chrisarndt.de/en/software/python/threadpool/), which is
    > built on top of the threading and Queue stdlib modules. Whatever bugs
    > may linger there, I'd hope that they would show up as normal Python
    > exceptions instead of segfaults.
    >
    > All I have now is a few not particularly insightful core files (actual
    > path names and args changed for privacy):
    >
    > /home/gsakkis/foo>gdb --core core.20140
    > GNU gdb Red Hat Linux (5.2-2)
    > Copyright 2002 Free Software Foundation, Inc.
    > GDB is free software, covered by the GNU General Public License, and
    > you are
    > welcome to change it and/or distribute copies of it under certain
    > conditions.
    > Type "show copying" to see the conditions.
    > There is absolutely no warranty for GDB. Type "show warranty" for
    > details.
    > This GDB was configured as "i386-redhat-linux".
    > Core was generated by `python2.5 /home/gsakkis/foo/foo.py --XXX --
    > max=30 --bar=/tmp/83840`
    > Program terminated with signal 11, Segmentation fault.
    > #0 0x080b222d in ?? ()
    > (gdb) backtrace
    > #0 0x080b222d in ?? ()
    > #1 0x080b28d1 in ?? ()
    > #2 0x080fa8ab in ?? ()
    > #3 0x0805c918 in ?? ()
    > (...)
    > #28 0x080b310f in ?? ()
    > #29 0x080dbfdd in ?? ()
    > #30 0x40021fef in ?? ()
    >
    > Any hints ?
    SPE - Stani's Python Editor, Mar 7, 2007
    #2
    1. Advertising

  3. George Sakkis

    John Nagle Guest

    You're using Python on a web server to do something
    complicated. You must suffer.

    Are you trying to fork off a subprocess in a multithreaded
    program? That's unlikely to work. The sematics differ
    from OS to OS (Solaris forks all the threads, most other
    operating systems don't; most UNIX-based OSs copy all the
    open file descriptors, but some give you control of which
    open files are passed), and Python may not be thread-safe
    in that area.

    John Nagle

    SPE - Stani's Python Editor wrote:
    > If the error is reproducable and you can run wxPython, you could use
    > the excellent WinPdb debugger, which ships with SPE (python editor):
    > http://www.digitalpeers.com/pythondebugger/
    >
    > ... but if you can only run your script on a remote server this won't
    > help you.
    >
    > Stani
    >
    > --
    > SPE - http://pythonide.stani.be
    >
    > On 7 Mrz., 20:15, "George Sakkis" <> wrote:
    >
    >>I have a pure python program (no C extensions) that occasionally core
    >>dumps in a non-reproducible way. The program is started by a (non-
    >>python) cgi script when a form is submitted. It involves running a
    >>bunch of other programs through subprocess in multiple threads and
    >>writing its output in several files. So the only suspicious parts I
    >>can think of is subprocess and/or multithreading. For the
    >>multithreading part I'm using a modified version of threadpool.py
    >>(http://www.chrisarndt.de/en/software/python/threadpool/), which is
    >>built on top of the threading and Queue stdlib modules. Whatever bugs
    >>may linger there, I'd hope that they would show up as normal Python
    >>exceptions instead of segfaults.
    >>
    >>All I have now is a few not particularly insightful core files (actual
    >>path names and args changed for privacy):
    >>
    >>/home/gsakkis/foo>gdb --core core.20140
    >>GNU gdb Red Hat Linux (5.2-2)
    >>Copyright 2002 Free Software Foundation, Inc.
    >>GDB is free software, covered by the GNU General Public License, and
    >>you are
    >>welcome to change it and/or distribute copies of it under certain
    >>conditions.
    >>Type "show copying" to see the conditions.
    >>There is absolutely no warranty for GDB. Type "show warranty" for
    >>details.
    >>This GDB was configured as "i386-redhat-linux".
    >>Core was generated by `python2.5 /home/gsakkis/foo/foo.py --XXX --
    >>max=30 --bar=/tmp/83840`
    >>Program terminated with signal 11, Segmentation fault.
    >>#0 0x080b222d in ?? ()
    >>(gdb) backtrace
    >>#0 0x080b222d in ?? ()
    >>#1 0x080b28d1 in ?? ()
    >>#2 0x080fa8ab in ?? ()
    >>#3 0x0805c918 in ?? ()
    >>(...)
    >>#28 0x080b310f in ?? ()
    >>#29 0x080dbfdd in ?? ()
    >>#30 0x40021fef in ?? ()
    >>
    >>Any hints ?

    >
    >
    >
    John Nagle, Mar 7, 2007
    #3
  4. On Mar 7, 4:15 pm, John Nagle <> wrote:

    > You're using Python on a web server to do something
    > complicated. You must suffer.
    >
    > Are you trying to fork off a subprocess in a multithreaded
    > program? That's unlikely to work. The sematics differ
    > from OS to OS (Solaris forks all the threads, most other
    > operating systems don't; most UNIX-based OSs copy all the
    > open file descriptors, but some give you control of which
    > open files are passed), and Python may not be thread-safe
    > in that area.


    I see the potential problem in general, but in my case every thread is
    exclusively responsible for the subprocesses it forks; no subprocess
    is inherited from the main thread or is shared in among the worker
    threads.

    George
    George Sakkis, Mar 8, 2007
    #4
  5. George Sakkis

    John Nagle Guest

    George Sakkis wrote:
    > On Mar 7, 4:15 pm, John Nagle <> wrote:
    >
    >
    >> You're using Python on a web server to do something
    >>complicated. You must suffer.
    >>
    >> Are you trying to fork off a subprocess in a multithreaded
    >>program? That's unlikely to work. The sematics differ
    >>from OS to OS (Solaris forks all the threads, most other
    >>operating systems don't; most UNIX-based OSs copy all the
    >>open file descriptors, but some give you control of which
    >>open files are passed), and Python may not be thread-safe
    >>in that area.

    >
    >
    > I see the potential problem in general, but in my case every thread is
    > exclusively responsible for the subprocesses it forks; no subprocess
    > is inherited from the main thread or is shared in among the worker
    > threads.


    Forking itself may not be thread safe. Forking is a process-level
    operation, for historical reasons.

    John Nagle
    John Nagle, Mar 8, 2007
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mathias

    Segmentation faults using threads

    Mathias, Feb 13, 2007, in forum: Python
    Replies:
    8
    Views:
    307
    John Nagle
    Feb 14, 2007
  2. Stanley S
    Replies:
    16
    Views:
    2,491
    Keith Thompson
    Dec 22, 2005
  3. Digital Puer
    Replies:
    18
    Views:
    693
    Ron Natalie
    Dec 28, 2005
  4. ZillionDollarSadist

    Segmentation faults on "new"

    ZillionDollarSadist, Jan 17, 2007, in forum: C++
    Replies:
    6
    Views:
    348
    Jacek Dziedzic
    Jan 18, 2007
  5. Kaspar Schiess

    YAML custom load: Segmentation faults

    Kaspar Schiess, Jun 25, 2004, in forum: Ruby
    Replies:
    2
    Views:
    130
    Kaspar Schiess
    Jun 25, 2004
Loading...

Share This Page