os.pipe and subprocess under Windows

Discussion in 'Python' started by Jeremy Sanders, Nov 17, 2008.

  1. Hi - I have some code which works under linux. It starts a remote python
    process using subprocess and communicates to it via a pipe created by
    os.pipe. As far as I understand, child processes should inherit file
    descriptors from the parent if close_fds=False on the suprocess.Popen
    command line.

    This code doesn't work under Window, but gives "bad file descriptor" when
    trying to read from the pipe in the child process. I have some example code
    which fails. It consists of two files, master.py and slave.py. The file
    descriptor for reading from the pipe is passed as a argument to slave.py.

    ----------------------------------------------------------------
    #!/usr/bin/env python
    # This is master.py

    import os
    import os.path
    import sys
    import subprocess

    def runMaster():
    # create pipe to communicate with remote process
    rpipe, wpipe = os.pipe()

    # start remote process
    cmdline = [sys.executable,
    os.path.join( os.path.dirname(
    os.path.abspath(__file__)), 'slave.py' ),
    str(rpipe) ]

    remote = subprocess.Popen(cmdline, shell=False, bufsize=0,
    close_fds=False)

    # send text to remote process via pipe
    os.write(wpipe, 'hi there$')

    # wait until remote exit
    remote.wait()

    if __name__ == '__main__':
    runMaster()

    ------------------------------------------------------------------

    # This is slave.py

    import sys
    import os

    def runSlave(fd):
    """Copy text to stderr from file descriptor until a $ symbol."""
    while True:
    intext = os.read(fd, 1)
    if intext == '$':
    break
    elif intext:
    # write text from pipe to stderr
    sys.stderr.write('* %s\n' % intext)

    if __name__ == '__main__':
    fd = int(sys.argv[1])
    runSlave(fd)

    -------------------------------------------------------------------

    Does anyone have any ideas how to get this to work under Windows? Is it
    correct code under unix?

    Thanks

    Jeremy

    --
    Jeremy Sanders
    http://www.jeremysanders.net/
     
    Jeremy Sanders, Nov 17, 2008
    #1
    1. Advertising

  2. Jeremy Sanders wrote:

    > As far as I understand, child processes should inherit file
    > descriptors from the parent if close_fds=False on the suprocess.Popen
    > command line.
    >
    > This code doesn't work under Window, but gives "bad file descriptor" when
    > trying to read from the pipe in the child process.


    <http://docs.python.org/library/subprocess.html>:

    If close_fds is true, all file descriptors except 0, 1 and 2 will be
    closed before the child process is executed. (Unix only). Or, on
    Windows, if close_fds is true then no handles will be inherited by the
    child process.

    Windows has no fork(2).
     
    Lawrence D'Oliveiro, Nov 17, 2008
    #2
    1. Advertising

  3. Lawrence D'Oliveiro wrote:

    > <http://docs.python.org/library/subprocess.html>:
    >
    > If close_fds is true, all file descriptors except 0, 1 and 2 will be
    > closed before the child process is executed. (Unix only). Or, on
    > Windows, if close_fds is true then no handles will be inherited by the
    > child process.
    >
    > Windows has no fork(2).


    Yes - I saw that - thanks. This suggests that as I have used
    closed_fds=False, then the child process will inherit the handles under
    Windows, which it doesn't seem to. This documentation looks wrong to me.

    I know Windows has no fork - that's why I used subprocess. This MSDN page
    suggests you you need to do something with SetHandleInformation to get them
    to be inherited. Doesn't subprocess do that?

    http://msdn.microsoft.com/en-us/library/ms682499(VS.85).aspx

    --
    Jeremy Sanders
    http://www.jeremysanders.net/
     
    Jeremy Sanders, Nov 17, 2008
    #3
  4. Jeremy Sanders wrote:

    > Hi - I have some code which works under linux. It starts a remote python
    > process using subprocess and communicates to it via a pipe created by
    > os.pipe. As far as I understand, child processes should inherit file
    > descriptors from the parent if close_fds=False on the suprocess.Popen
    > command line.


    Hmm... examining the code for os.pipe in posixmodule.c, it looks like pipes
    are create specifically to be non-inheritable in Windows. I can't see why
    you would want a non-inheritable pipe, so I would call this a bug.

    I suppose I could try this trick from subprocess.py to make the pipes
    inheritable:

    def _make_inheritable(self, handle):
    """Return a duplicate of handle, which is inheritable"""
    return DuplicateHandle(GetCurrentProcess(), handle,
    GetCurrentProcess(), 0, 1,
    DUPLICATE_SAME_ACCESS)

    Pretty nasty to have to do this though, and I would have to add a win32api
    dependency, or hack around with the _subprocess module.

    Jeremy

    --
    Jeremy Sanders
    http://www.jeremysanders.net/
     
    Jeremy Sanders, Nov 17, 2008
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. lee, wonsun
    Replies:
    1
    Views:
    498
    Jack Klein
    Nov 2, 2004
  2. Tom Brown
    Replies:
    0
    Views:
    470
    Tom Brown
    Sep 22, 2005
  3. Rafael Giannetti Viotti

    Subprocess and pipe-fork-exec primitive

    Rafael Giannetti Viotti, Jul 30, 2007, in forum: Python
    Replies:
    2
    Views:
    3,031
    Rafael V.
    Aug 1, 2007
  4. Replies:
    1
    Views:
    237
    Ben Morrow
    Jun 2, 2004
  5. Kevin Walzer
    Replies:
    2
    Views:
    139
    Terry Reedy
    Dec 22, 2012
Loading...

Share This Page