Spaces in path name

[joep]

Sure enough, passing shell=True -- which is probably quite
a rare requirement -- causes the code to change the call
from "a.exe b.doc" to '%COMSPEC% /c "a.exe" "b.doc"'.
The quoting rules (from cmd /?) are slightly involved but
I can't see at first why this shouldn't work. However it
clearly doesn't so I'll try to put together either a patch
to the subprocess code or to the docs warning of the behaviour.

I think that, in general, you need to pass shell=True far less
often that you might imagine. (Possibly only for internal
commands like dir, copy etc.).

TJG

Thanks, I didn't know it is possible to drop the shell=True. The
explanation of the subprocess model are not very clear to me and the
examples are quite incomplete. I got it to work by trial and error and
looking at the usage in different packages.

Dropping shell=True works well for my case, no error messages and it
still captures the standard output:

p = subprocess.Popen([
r"C:\Program Files\WinRAR\Rar.exe","v",
r"C:\temp\Copy of papers.rar"], stdout=subprocess.PIPE)

I am still looking at different versions, and the cases when it works
and when it doesn't are still confusing.

Josef

 

[Tim Golden]

Sure enough, passing shell=True -- which is probably quite
a rare requirement -- causes the code to change the call
from "a.exe b.doc" to '%COMSPEC% /c "a.exe" "b.doc"'.
The quoting rules (from cmd /?) are slightly involved but
I can't see at first why this shouldn't work. However it
clearly doesn't so I'll try to put together either a patch
to the subprocess code or to the docs warning of the behaviour.

I think that, in general, you need to pass shell=True far less
often that you might imagine. (Possibly only for internal
commands like dir, copy etc.).

TJG

Thanks, I didn't know it is possible to drop the shell=True. The
explanation of the subprocess model are not very clear to me and the
examples are quite incomplete. I got it to work by trial and error and
looking at the usage in different packages.

Dropping shell=True works well for my case, no error messages and it
still captures the standard output:

p = subprocess.Popen([
r"C:\Program Files\WinRAR\Rar.exe","v",
r"C:\temp\Copy of papers.rar"], stdout=subprocess.PIPE)

I am still looking at different versions, and the cases when it works
and when it doesn't are still confusing.

Well I've got a patch ready to go (which basically just
wraps a shell=True command line with an *extra* pair of
double-quotes, the same as you do for an os.system call).
I'll try to put some time into the subprocess docs as well,
at least as far as a Win32-how-do-I on my own site if not
an update to the official docs.

TJG

 

[Tim Golden]

Well I've got a patch ready to go (which basically just
wraps a shell=True command line with an *extra* pair of
double-quotes, the same as you do for an os.system call).
I'll try to put some time into the subprocess docs as well,
at least as far as a Win32-how-do-I on my own site if not
an update to the official docs.

For the record, I've raised this as issue 2304 with a
patch attached:

http://bugs.python.org/issue2304

TJG

 

[joep]

One more try (using popen instead of call is not necessary for these
cases, but I want to see the behavior of popen):

shell=True and executable and at least one argument with spaces does
not work:
---------------------------------------------------------------------------------------------------------------------

p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin
\Copy of cp.exe",
r"C:\Temp\temp with spaces\test out.txt",
r"C:\Temp\temp with spaces\test out3.txt"],
shell=True, stdout=subprocess.PIPE)

causes:
'C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin\Copy' is not recognized
as an int
ernal or external command, operable program or batch file.


the following all work:
================

without shell=True, and executable and at least one argument with
spaces
-----------------------------------------------------------------------------------------------------------

p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin
\Copy of cp.exe",
r"C:\Temp\temp with spaces\test out.txt",
r"C:\Temp\temp with spaces\test out4.txt"],
stdout=subprocess.PIPE)

with shell=True, and executable without spaces, even if two arguments
with spaces
------------------------------------------------------------------------------------------------------------------------

p = subprocess.Popen(["copy",
r"C:\Temp\temp with spaces\test out.txt",
r"C:\Temp\temp with spaces\test out2.txt"],
shell=True, stdout=subprocess.PIPE)

with shell=True, and executable without spaces, even if two arguments
with spaces
------------------------------------------------------------------------------------------------------------------------
but here shell=True is not necessary

p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin
\cp.exe",
r"C:\Temp\temp with spaces\test out.txt",
r"C:\Temp\temp with spaces\test out5.txt"],
shell=True, stdout=subprocess.PIPE)

without shell=True, and executable and at least one arguments with
spaces
-------------------------------------------------------------------------------------------------------------

p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin
\Copy of cp.exe",
r"C:\Temp\temp with spaces\test out.txt",
r"C:\Temp\temp with spaces\test out4.txt"],
stdout=subprocess.PIPE)

My conclusions:
============

Assuming shell=True is only required for build-in shell commands, non
of which has spaces.

There is no problem, if you know when *not* to use shell=True:

* More than two arguments with spaces are never problem, as long as
the executable does not have spaces
* If shell=True is required, then the executable is a build-in shell
command, which does not contain spaces, and, therefore, has no
problems
* If you use a non-build in executable, then don't use shell=True.
This works correctly even if the executable and at least one
additional argument have spaces.

It took a lot of time to figure this out, but now at least I know, how
to construct the call to subprocess.Popen, so that it works in the
cases I used it so far.

Josef

 

[Gertjan Klein]

* If shell=True is required, then the executable is a build-in shell
command, which does not contain spaces, and, therefore, has no
problems

This is only true when you are referring to directly executable files.
However, Shell=True is also required when you want to "execute" a file
by it's association. For example, if you want to execute a ".pyw" file
directly (i.e., by using whichever python executable is associated with
that extension), you need Shell=True.

IMHO, the subprocess module should not have been included as-is in the
Python library. It is counter-intuitive, buggy, ill-documented, and has
needless limitations in functionality.

Regards,
Gertjan.

 

[Tim Golden]

This is only true when you are referring to directly executable files.
However, Shell=True is also required when you want to "execute" a file
by it's association. For example, if you want to execute a ".pyw" file
directly (i.e., by using whichever python executable is associated with
that extension), you need Shell=True.

Thanks for that clarification. I hadn't tried that path.

TJG