question about subprocess and shells

[Ross Boylan]

If one uses subprocess.Popen(args, ..., shell=True, ...)

When args finishes execution, does the shell terminate? Either way
seems problematic.

If it does not terminate, then it seems as if calls like wait and
communicate would never return. It also seems the subprocess would
never die, and that most of the examples with the shell True would leave
processes lying around.

If it does terminate, then how can you stuff new commands down the pipe
to the subprocesses stdin?

Does this module contemplate receiving multiple commands for the shell
to execute?

I'm also unsure of the semantics of the pipes for the processes standard
file handles. Do they need to be closed (judging from the examples,
no)? When reads/writes to them return, and what state is the stream in
at the time?

Thanks for any wisdom you can offer.
Ross Boylan

 

[Lie Ryan]

If one uses subprocess.Popen(args, ..., shell=True, ...)

When args finishes execution, does the shell terminate? Either way
seems problematic.

If it does not terminate, then it seems as if calls like wait and
communicate would never return. It also seems the subprocess would
never die, and that most of the examples with the shell True would leave
processes lying around.

No, p.wait() will immediately return, and p.communicate() discards
everything new coming.

If it does terminate, then how can you stuff new commands down the pipe
to the subprocesses stdin?

start a new shell, of course.

Does this module contemplate receiving multiple commands for the shell
to execute?

yes, use several subprocess.Popen() or call a batch/shell script.

 

[Floris Bruynooghe]

If one uses subprocess.Popen(args, ..., shell=True, ...)

When args finishes execution, does the shell terminate?  Either way
seems problematic.

Essentially this is executing "/bin/sh args" so if you're unsure as to
the behaviour just try it on your command line. Basically once the
pipeline in "args" had finished the shell has nothing more to do and
will return itself (and the return code for the shell depends on the
return code of the pipeline executed which is normally the return code
of the last process executed).

Of course when I say "pipeline" it could also be a single command or a
list or any valid shell.

Regards
Floris

 

[Nobody]

If one uses subprocess.Popen(args, ..., shell=True, ...)

When args finishes execution, does the shell terminate? Either way
seems problematic.

That depends upon what "args" is. On Unix, if args ends with a "&", the
shell will terminate as soon as it has started the command. Typically, the
shell will terminate once it has executed the last command, where
"executed" means fork()+exec() for a background command and
fork()+exec()+wait() for a non-background command.

If it does not terminate, then it seems as if calls like wait and
communicate would never return. It also seems the subprocess would
never die, and that most of the examples with the shell True would leave
processes lying around.

That would be the case if the shell didn't terminate.

If it does terminate, then how can you stuff new commands down the pipe
to the subprocesses stdin?

You don't.

shell=True executes ["/bin/sh", "-c", args], not an interactive shell
reading commands from stdin (unless you explicitly run an interactive
shell with e.g. Popen(args = "sh", ...)).

Does this module contemplate receiving multiple commands for the shell
to execute?

No. It behaves like an extended version of the Unix popen() function, i.e.
like system() but you get to provide pipes for the standard handles.

I'm also unsure of the semantics of the pipes for the processes standard
file handles. Do they need to be closed (judging from the examples,
no)?

If args is a command which reads from stdin, then you will need to close
stdin, otherwise it will never terminate. Otherwise, there's no need.

When reads/writes to them return, and what state is the stream in
at the time?

Any streams created when std{in,out,err}= are specified as PIPE or as a
descriptor number are unbuffered by default, but this can be changed by
the bufsize parameter to the Popen() constructor.

For an unbuffered stream, read() and write() will return when the
underlying system calls return. write() to stdin will return once the data
has been written to the pipe (not necessarily when the child has consumed
it), while read() from stdout/stderr will return once the requested
amount of data has been read from the pipe (or EOF is reached).