subprocess.Popen deadlocks

E

eraserix

Hi

From time to time I observe a deadlock in subprocess.Popen. I see the
childprocess beeing <defunct> and if I attach gdb to the stuck python
script, I can see it waiting on line 1128 in subprocess.py ( data =
_eintr_retry_call(os.read, errpipe_read, 1048576) ).

Any ideas of whats going on? I'm on linux using python 2.6.5. I
already found http://bugs.python.org/issue1731717 and stubbed out
subprocess._cleanup, but this didn't really help and doesn't seem to
be the problem anyway...

A bit of information about what I'm trying to do:
I try to control several process from a python script. Each process is
started from a thread, the threads just wait() for the child to exit
and can then be joined. Main waits for SIGINT. After it received the
signal, it will send a signal to the processes spawned by the threads
and then join() the threads (which should terminate after their
process exits)

Christoph
 
L

Lawrence D'Oliveiro

In message
I try to control several process from a python script. Each process is
started from a thread, the threads just wait() for the child to exit
and can then be joined. Main waits for SIGINT. After it received the
signal, it will send a signal to the processes spawned by the threads
and then join() the threads (which should terminate after their
process exits)

Threading is always likely to be a source of bugs, whether in your code or
the Python library. In this case, your threads don’t even seem to be doing
anything useful.

Why not skip all the threads and just spawn all the processes in your main
loop? Then after getting the SIGINT, send that signal to all your processes,
and then do a bunch of os.waitpid calls to clean them all up.
 
E

eraserix

In message

Why not skip all the threads and just spawn all the processes in your main
loop? Then after getting the SIGINT, send that signal to all your processes,
and then do a bunch of os.waitpid calls to clean them all up.

Thanks for the hint. You are right, the problem could be solved
singlethreaded by doing some sort of polling (my description
simplified things a bit) avoiding the multithreading problems
altogether.

I had another look at the subprocesses module and discovered the
actual bug. I describe it in issue 10394.

Christoph
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top