Python deadlocks when Popen in multithread.


I

INADA Naoki

On Ubuntu 10.10 amd64 and dual-core CPU, following code deadlocks.
What's wrong on this code?

#http://codepad.org/GkoXHbik

#!/usr/bin/env python

from subprocess import *
from threading import *

THREAD_COUNT=50

def worker():
p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
out = p.communicate("hoge")[0]
print "%s %s" % (current_thread().name, out)

threads = []
for i in range(THREAD_COUNT):
threads.append(Thread(target=worker, name=str(i)))
for t in threads: t.start()
for t in threads: t.join()
 
Ad

Advertisements

J

Jed Smith

def worker():
   p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
   out = p.communicate("hoge")[0]
   print "%s %s" % (current_thread().name, out)

If, instead of spawning workers you directly call worker(), does it succeed?

I suspect the threading is a red herring here.
 
I

INADA Naoki

def worker():
   p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
   out = p.communicate("hoge")[0]
   print "%s %s" % (current_thread().name, out)

If, instead of spawning workers you directly call worker(), does it succeed?
Yes.

I suspect the threading is a red herring here.

But thread & subprocess is a normal way to invoke outer commands in
parallel.
 
Ad

Advertisements

I

INADA Naoki

I've found the bug http://bugs.python.org/issue1731717.
It seems wating subprocess can be multithreaded but creating
subprocess cannot.

def worker():
   p = Popen(["cat"], stdin=PIPE, stdout=PIPE, stderr=STDOUT)
   out = p.communicate("hoge")[0]
   print "%s %s" % (current_thread().name, out)
If, instead of spawning workers you directly call worker(), does it succeed?
Yes.

I suspect the threading is a red herring here.

But thread & subprocess is a normal way to invoke outer commands in
parallel.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top