bug: subprocess.Popen() hangs

Discussion in 'Python' started by Jonathan Amsterdam, Oct 25, 2007.

  1. This is a bug in python 2.4 under Linux 2.6.

    I occasionally see subprocess.Popen() fail to return, and I have
    finally figured out roughly what's going on. It involves the GC and
    stderr.

    1. os.fork()

    2. Parent blocks reading from errpipe_read (subprocess.py:982)

    3. In child, a GC occurs before the exec.

    4. The GC attempts to free a file descriptor, calling file_dealloc.

    5. That function gets an error closing the descriptor ("close failed:
    [Errno 9] Bad file descriptor\n," is the string I extracted via gdb).

    6. It attempts to write the error to stderr and blocks. Since it never
    execs or writes to errpipe_write, both child and parent are hung.

    Here is the pstack output on the child:

    #0 0x006587a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
    #1 0x0072f11b in __write_nocancel () from /lib/tls/libc.so.6
    #2 0x006d409f in _IO_new_file_write () from /lib/tls/libc.so.6
    #3 0x006d42ec in _IO_new_file_xsputn () from /lib/tls/libc.so.6
    #4 0x006afce9 in buffered_vfprintf () from /lib/tls/libc.so.6
    #5 0x006afe8b in vfprintf () from /lib/tls/libc.so.6
    #6 0x080dd4af in mywrite ()
    #7 0x080dd505 in PySys_WriteStderr ()
    #8 0x08064cd0 in file_dealloc ()
    #9 0x08079c88 in dict_dealloc ()
    #10 0x0808931d in subtype_dealloc ()
    #11 0x08079af3 in PyDict_Clear ()
    #12 0x0807bb6a in dict_tp_clear ()
    #13 0x080e0ead in collect ()
    #14 0x080e1912 in _PyObject_GC_New ()
    #15 0x0805e50b in PyInstance_NewRaw ()
    #16 0x0805e5d7 in PyInstance_New ()
    #17 0x0805bdc0 in PyObject_Call ()
    #18 0x080aecef in PyEval_CallObjectWithKeywords ()
    #19 0x080ca975 in PyErr_NormalizeException ()
    #20 0x080b492c in PyEval_EvalFrame ()
    #21 0x080b5eb2 in PyEval_EvalCodeEx ()
    #22 0x080b3c83 in PyEval_EvalFrame ()
    #23 0x080b5734 in PyEval_EvalFrame ()
    #24 0x080b5eb2 in PyEval_EvalCodeEx ()
    #25 0x080fce92 in function_call ()
    #26 0x0805bdc0 in PyObject_Call ()
    #27 0x080641e5 in instancemethod_call ()
    #28 0x0805bdc0 in PyObject_Call ()
    #29 0x0808ffce in slot_tp_init ()
    #30 0x08088b3a in type_call ()
    #31 0x0805bdc0 in PyObject_Call ()
    #32 0x080b0f05 in PyEval_EvalFrame ()
    #33 0x080b5eb2 in PyEval_EvalCodeEx ()
    #34 0x080fce92 in function_call ()
    #35 0x0805bdc0 in PyObject_Call ()
    #36 0x080641e5 in instancemethod_call ()
    #37 0x0805bdc0 in PyObject_Call ()
    #38 0x0808ffce in slot_tp_init ()
    #39 0x08088b3a in type_call ()
    #40 0x0805bdc0 in PyObject_Call ()
    #41 0x080b0f05 in PyEval_EvalFrame ()
    #42 0x080b5734 in PyEval_EvalFrame ()
    #43 0x080b5eb2 in PyEval_EvalCodeEx ()
    #44 0x080fce92 in function_call ()
    #45 0x0805bdc0 in PyObject_Call ()
    #46 0x080641e5 in instancemethod_call ()
    #47 0x0805bdc0 in PyObject_Call ()
    #48 0x0808ffce in slot_tp_init ()
    #49 0x08088b3a in type_call ()
    #50 0x0805bdc0 in PyObject_Call ()
    #51 0x080b0f05 in PyEval_EvalFrame ()
    #52 0x080b5eb2 in PyEval_EvalCodeEx ()
    #53 0x080fce92 in function_call ()
    #54 0x0805bdc0 in PyObject_Call ()
    #55 0x080b075f in PyEval_EvalFrame ()
    #56 0x080b5734 in PyEval_EvalFrame ()
    #57 0x080b5734 in PyEval_EvalFrame ()
    #58 0x080b5734 in PyEval_EvalFrame ()
    #59 0x080b5eb2 in PyEval_EvalCodeEx ()
    #60 0x080b3c83 in PyEval_EvalFrame ()
    #61 0x080b5734 in PyEval_EvalFrame ()
    #62 0x080b5734 in PyEval_EvalFrame ()
    #63 0x080b5eb2 in PyEval_EvalCodeEx ()
    #64 0x080b3c83 in PyEval_EvalFrame ()
    #65 0x080b5eb2 in PyEval_EvalCodeEx ()
    #66 0x080b3c83 in PyEval_EvalFrame ()
    #67 0x080b5eb2 in PyEval_EvalCodeEx ()
    #68 0x080b3c83 in PyEval_EvalFrame ()
    #69 0x080b5734 in PyEval_EvalFrame ()
    #70 0x080b5734 in PyEval_EvalFrame ()
    #71 0x080b5734 in PyEval_EvalFrame ()
    #72 0x080b5734 in PyEval_EvalFrame ()
    #73 0x080b5734 in PyEval_EvalFrame ()
    #74 0x080b5eb2 in PyEval_EvalCodeEx ()
    #75 0x080fce92 in function_call ()
    #76 0x0805bdc0 in PyObject_Call ()
    #77 0x080b075f in PyEval_EvalFrame ()
    #78 0x080b5eb2 in PyEval_EvalCodeEx ()
    #79 0x080b3c83 in PyEval_EvalFrame ()
    #80 0x080b5eb2 in PyEval_EvalCodeEx ()
    #81 0x080b3c83 in PyEval_EvalFrame ()
    #82 0x080b5eb2 in PyEval_EvalCodeEx ()
    #83 0x080b3c83 in PyEval_EvalFrame ()
    #84 0x080b5734 in PyEval_EvalFrame ()
    #85 0x080b5734 in PyEval_EvalFrame ()
    #86 0x080b5eb2 in PyEval_EvalCodeEx ()
    #87 0x080b601a in PyEval_EvalCode ()
    #88 0x080d9ff4 in PyRun_FileExFlags ()
    #89 0x080da8d6 in PyRun_SimpleFileExFlags ()
    #90 0x08055617 in Py_Main ()
    #91 0x08054e3f in main ()
     
    Jonathan Amsterdam, Oct 25, 2007
    #1
    1. Advertising

  2. Jonathan Amsterdam <> wrote:
    > This is a bug in python 2.4 under Linux 2.6.
    >
    > I occasionally see subprocess.Popen() fail to return, and I have
    > finally figured out roughly what's going on. It involves the GC and
    > stderr.


    Interesting

    Do you have a program to demonstrate the problem?

    You are best off reporting bugs here - then they won't get lost!

    http://bugs.python.org/

    --
    Nick Craig-Wood <> -- http://www.craig-wood.com/nick
     
    Nick Craig-Wood, Oct 26, 2007
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    7
    Views:
    1,295
    Karthik Gurusamy
    Sep 25, 2007
  2. Kenneth McDonald

    subprocess.Popen hangs at times?

    Kenneth McDonald, Sep 10, 2008, in forum: Python
    Replies:
    0
    Views:
    317
    Kenneth McDonald
    Sep 10, 2008
  3. Tim Johnson

    subprocess.Popen instance hangs

    Tim Johnson, Aug 29, 2013, in forum: Python
    Replies:
    0
    Views:
    93
    Tim Johnson
    Aug 29, 2013
  4. MRAB
    Replies:
    0
    Views:
    83
  5. Prasad, Ramit

    RE: subprocess.Popen instance hangs

    Prasad, Ramit, Aug 29, 2013, in forum: Python
    Replies:
    0
    Views:
    99
    Prasad, Ramit
    Aug 29, 2013
Loading...

Share This Page