How do I use the subprocess module with mswindows?

[Darren Dale]

I'm a developer on the matplotlib project, and I am having trouble with the
subprocess module on windows (Python 2.4.2 on winXP). No trouble to report
with linux. I need to use _subprocess instead of pywin32, but my trouble
exists with either option:

import subprocess
process = subprocess.Popen(['dir'], stderr=subprocess.STDOUT,
stdout=subprocess.PIPE)
stat = process.wait()
print process.stdout.read()
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Documents and Settings\Darren\Desktop\subprocess_test.py",
line 3, in ?
    process = subprocess.Popen(['dir'], stderr=subprocess.STDOUT,
stdout=subprocess.PIPE)#, stdout=PIPE)
  File "C:\Python24\lib\subprocess.py", line 533, in __init__
    (p2cread, p2cwrite,
  File "C:\Python24\lib\subprocess.py", line 593, in _get_handles
    p2cread = self._make_inheritable(p2cread)
  File "C:\Python24\lib\subprocess.py", line 634, in _make_inheritable
    DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
----------------------------------------------------------------------

If I change my script a bit, I get a different error:

import subprocess
process = subprocess.Popen(['dir'])
stat = process.wait()
print process.stdout.read()
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Documents and Settings\Darren\Desktop\subprocess_test.py",
line 3, in ?
    process = subprocess.Popen(['dir'])#, stderr=subprocess.STDOUT,
stdout=subprocess.PIPE)#, stdout=PIPE)
  File "C:\Python24\lib\subprocess.py", line 542, in __init__
    errread, errwrite)
  File "C:\Python24\lib\subprocess.py", line 706, in _execute_child
    startupinfo)
WindowsError: [Errno 2] The system cannot find the file specified
----------------------------------------------------------------------

Can anyone tell me what I am doing wrong?

Thanks,
Darren

 

[Fredrik Lundh]

If I change my script a bit, I get a different error:

import subprocess
process = subprocess.Popen(['dir'])
stat = process.wait()
print process.stdout.read()

WindowsError: [Errno 2] The system cannot find the file specified

"dir" is a shell command under Windows, not an executable. to run
commands via the shell, use

process = subprocess.Popen(['dir'], shell=True)

</F>

 

[Darren Dale]

If I change my script a bit, I get a different error:

import subprocess
process = subprocess.Popen(['dir'])
stat = process.wait()
print process.stdout.read()

WindowsError: [Errno 2] The system cannot find the file specified

"dir" is a shell command under Windows, not an executable. to run
commands via the shell, use

process = subprocess.Popen(['dir'], shell=True)

</F>

Thank You!

 

[Thomas Bellman]

import subprocess
process = subprocess.Popen(['dir'], stderr=subprocess.STDOUT,
stdout=subprocess.PIPE)
stat = process.wait()
print process.stdout.read()

You have already gotten the answer to why 'dir' doesn't work for
you, but there is a bug hiding in that code that you might not
notice in simple tests.

You are waiting for your subprocess to complete without reading
away what it prints. That will quickly fill the buffer available
in the pipe between you and the subprocess, and the subprocess
will block. Try calling Popen() with something that prints more
data, like ['find', '/', '-print'] on a Unix box, and you will
notice that problem.

What you should do is:

output = process.stdout.read()
stat = process.wait()
print output

Or, you could use the .communicate() method on the Popen object
instead of .stdout.read().

If you find yourself juggling several subprocesses running in
parallel, producing and/or consuming data "incrementally", or
just trying to handle both sending input to and reading output
from a single subprocess without deadlocking, you may be helped
by using my asyncproc module, which you can download from

http://www.lysator.liu.se/~bellman/download/asyncproc.py

I suspect that it only works on Unix, though.