Re: Waiting for a subprocess to exit

Discussion in 'Python' started by Miles Kaufmann, Aug 21, 2009.

  1. On Aug 20, 2009, at 10:13 PM, Ben Finney wrote:
    > The module documentation has a section on replacing ‘os.system’
    > <http://docs.python.org/library/subprocess#replacing-os-system>, which
    > says to use::
    >
    > process = subprocess.Popen("mycmd" + " myarg", shell=True)
    > status = os.waitpid(process.pid, 0)
    >
    > But a ‘Popen’ instance has its own ‘wait’ method, which waits for exit
    > <URL:http://docs.python.org/library/subprocess#subprocess.Popen.wait>.
    > Why would I use ‘os.waitpid’ instead of::
    >
    > process = subprocess.Popen("mycmd" + " myarg", shell=True)
    > process.wait()
    > status = process.returncode


    Really, you can just use:

    process = subprocess.Popen("mycmd" + " myarg", shell=True)
    status = process.wait()

    I'm not sure why the documentation suggests using os.waitpid.

    I would recommend avoiding shell=True whenever possible. It's used in
    the examples, I suspect, to ease the transition from the functions
    being replaced, but all it takes is for a filename or some other input
    to unexpectedly contain whitespace or a metacharacter and your script
    will stop working--or worse, do damage (cf. the iTunes 2 installer
    debacle[1]). Leaving shell=False makes scripts more secure and
    robust; besides, when I'm putting together a command and its
    arguments, it's as convenient to build a list (['mycmd', 'myarg']) as
    it is a string (if not more so).

    -Miles

    [1]: http://apple.slashdot.org/article.pl?sid=01/11/04/0412209#comment_2518563
    Miles Kaufmann, Aug 21, 2009
    #1
    1. Advertising

  2. Miles Kaufmann

    Chris Rebert Guest

    Re: Sanitising arguments to shell commands (was: Waiting for asubprocess to exit)

    On Fri, Aug 21, 2009 at 2:08 AM, Ben Finney<> wrote:
    <snip>
    > How can I take a string that is intended to be part of a command line,
    > representing multiple arguments and the shell's own escape characters as
    > in the above example, and end up with a sane command argument list for
    > ‘subprocess.Popen’?


    http://docs.python.org/library/shlex.html

    module shlex — Simple lexical analysis
    New in version 1.5.2.
    "The shlex class makes it easy to write lexical analyzers for simple
    syntaxes resembling that of the Unix shell."

    Cheers,
    Chris
    --
    http://blog.rebertia.com
    Chris Rebert, Aug 21, 2009
    #2
    1. Advertising

  3. Re: Sanitising arguments to shell commands

    Ben Finney wrote:
    > Miles Kaufmann <> writes:
    >
    >
    >> I would recommend avoiding shell=True whenever possible. It's used in
    >> the examples, I suspect, to ease the transition from the functions
    >> being replaced, but all it takes is for a filename or some other input
    >> to unexpectedly contain whitespace or a metacharacter and your script
    >> will stop working--or worse, do damage (cf. the iTunes 2 installer
    >> debacle[1]).
    >>

    >
    > Agreed, and that's my motivation for learning about ‘subprocess.Popen’.
    >


    Can someone explain the difference with the shell argument ? giving for
    instance an example of what True will do that False won't. I mean, I've
    read the doc, and to be honest, I didn't get it.
    I'm concerned because I'm using subprocess, but I guess my shell arg has
    been filled a little bit random..

    JM
    Jean-Michel Pichavant, Aug 21, 2009
    #3
  4. Re: Sanitising arguments to shell commands

    Ben Finney wrote:
    > Jean-Michel Pichavant <> writes:
    >
    >
    >> Can someone explain the difference with the shell argument ? giving
    >> for instance an example of what True will do that False won't.
    >>

    >
    > The ‘shell’ argument to the ‘subprocess.Popen’ constructor specifies
    > whether the command-line should be invoked directly (‘shell=False’) or
    > indirectly through invoking a shell (‘shell=True’).
    >
    > If ‘shell=False’, the command-line arguments are used as direct
    > arguments to the kernel's “run this program for meâ€.
    >
    > If ‘shell=True’ the command-line arguments are themselves passed to a
    > new instance of the user's current shell, as a command line that *it*
    > should invoke on the program's behalf. This allows the command line to
    > be manipulated and interpolated etc., the way it would be if typed at a
    > new shell prompt. Then, that shell will in turn ask the kernel “run this
    > program for me†as it normally does after processing the arguments.
    >
    >
    >> I mean, I've read the doc, and to be honest, I didn't get it. I'm
    >> concerned because I'm using subprocess, but I guess my shell arg has
    >> been filled a little bit random..
    >>

    >
    > Use ‘shell=False’ by default (which, since that's the default for
    > ‘subprocess.Popen’, means you can omit it entirely), and specify exactly
    > the command line arguments you want the kernel to execute. Only if you
    > know you want a shell process to be involved should you use
    > ‘shell=True’.
    >
    >

    Thank you Ben for the update. It's clear for me now, I've checked and I
    use it with no shell arg, except at one place, but I don't think it's
    intended and it happens to work anyway. I've added a small comment just
    in case it fails in the future.

    JM
    Jean-Michel Pichavant, Aug 21, 2009
    #4
  5. Miles Kaufmann

    Rick King Guest

    Re: Sanitising arguments to shell commands

    shlex doesn't handle unicode input though, so, in general, it's not a
    good solution.

    Rick King
    Southfield MI

    > http://docs.python.org/library/shlex.html
    >
    > module shlex — Simple lexical analysis
    > New in version 1.5.2.
    > "The shlex class makes it easy to write lexical analyzers for simple
    > syntaxes resembling that of the Unix shell."
    >
    Rick King, Aug 21, 2009
    #5
  6. Miles Kaufmann

    Chris Rebert Guest

    Re: Sanitising arguments to shell commands

    On Fri, Aug 21, 2009 at 3:55 PM, Ben Finney<> wrote:
    > Rick King <> writes:
    >
    >> shlex doesn't handle unicode input though, so, in general, it's not a
    >> good solution.

    >
    > Argh. Is there a Python bug tracker number for fixing that?


    Indeed there is:
    http://bugs.python.org/issue1170

    It even has a patch. I wonder why it's unapplied.

    Cheers,
    Chris
    --
    http://blog.rebertia.com
    Chris Rebert, Aug 22, 2009
    #6
  7. Miles Kaufmann

    Aahz Guest

    In article <>,
    Miles Kaufmann <> wrote:
    >
    >debacle[1]). Leaving shell=3DFalse makes scripts more secure and =20
    >robust; besides, when I'm putting together a command and its =20
    >arguments, it's as convenient to build a list (['mycmd', 'myarg']) as =20=
    >
    >it is a string (if not more so).


    .....unless you want to use "~" on Mac or Unix.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "I support family values -- Addams family values" --www.nancybuttons.com
    Aahz, Aug 25, 2009
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    356
  2. Flyzone
    Replies:
    2
    Views:
    292
    Flyzone
    Jun 15, 2007
  3. erikcw
    Replies:
    6
    Views:
    458
    Michael Palmer
    Sep 19, 2008
  4. Replies:
    17
    Views:
    397
    Terry Reedy
    Sep 19, 2013
  5. Jean-Michel Pichavant

    Re: subprocess call is not waiting.

    Jean-Michel Pichavant, Sep 13, 2012, in forum: Python
    Replies:
    0
    Views:
    182
    Jean-Michel Pichavant
    Sep 13, 2012
Loading...

Share This Page