subprocess.Popen inheriting

Discussion in 'Python' started by Aaron Brady, Dec 16, 2008.

  1. Aaron Brady

    Aaron Brady Guest

    Hi,

    I have a file handle I want to inherit in a child process. I am
    looking at '_make_inheritable' in 'Popen', but it needs an instance,
    and by the time I have one, the subprocess is already running.

    Can't I call 'Popen._make_inheritable( None, handle )'? The method
    does not use 'self'.
    Aaron Brady, Dec 16, 2008
    #1
    1. Advertising

  2. En Tue, 16 Dec 2008 07:29:19 -0200, Aaron Brady <>
    escribió:

    > I have a file handle I want to inherit in a child process. I am
    > looking at '_make_inheritable' in 'Popen', but it needs an instance,
    > and by the time I have one, the subprocess is already running.
    >
    > Can't I call 'Popen._make_inheritable( None, handle )'? The method
    > does not use 'self'.


    File handles are inherited by default, I think. What's your specific
    problem?

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 16, 2008
    #2
    1. Advertising

  3. Aaron Brady

    Aaron Brady Guest

    On Dec 16, 4:15 am, "Gabriel Genellina" <>
    wrote:
    > En Tue, 16 Dec 2008 07:29:19 -0200, Aaron Brady <>  
    > escribió:
    >
    > > I have a file handle I want to inherit in a child process.  I am
    > > looking at '_make_inheritable' in 'Popen', but it needs an instance,
    > > and by the time I have one, the subprocess is already running.

    >
    > > Can't I call 'Popen._make_inheritable( None, handle )'?  The method
    > > does not use 'self'.

    >
    > File handles are inherited by default, I think. What's your specific  
    > problem?
    >
    > --
    > Gabriel Genellina


    I want a dual-console application for some multi-threaded output. The
    main process spawns a second process in a second window, and directs
    its readout through a pipe. That is, the second process is just a
    dummy, print pipe directly to console.

    It worked when I used the 'stdin= PIPE' keyword in Popen, but passing
    my own pipe handle on the command line isn't working. The keyword is
    a workaround.

    > File handles are inherited by default, I think.


    I thought so too. The web seems to say that on Linux they are, and on
    Windows, you need to call DuplicateHandle for it.

    By the way, there are a few solutions to the inheritance problem.

    1. Override __init__ and __del__ in a subclass.
    2. Assign DummyClass= Popen._make_inheritable
    3. Override Popen._execute_child to call _make_inheritable, then call
    the super method.
    Aaron Brady, Dec 16, 2008
    #3
  4. Aaron Brady wrote:

    > I thought so too. The web seems to say that on Linux they are, and on
    > Windows, you need to call DuplicateHandle for it.


    I hit this problem - it looks like pipes aren't very versatile on Windows.
    There's also the complicating factor that the handles in windows aren't the
    same as the file numbers that Python uses, so you have to convert between
    them.

    It would be nice if Python created pipes that are properly inheritable by
    default by child processes, as they're mostly used for IPC.

    It was so painful that I converted my code to use sockets instead, which
    seem much more portable between Windows and Unix (though you don't get to
    use socketpair and AF_UNIX in Windows).

    Jeremy

    --
    Jeremy Sanders
    http://www.jeremysanders.net/
    Jeremy Sanders, Dec 17, 2008
    #4
  5. En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders
    <> escribió:

    > Aaron Brady wrote:
    >
    >> I thought so too. The web seems to say that on Linux they are, and on
    >> Windows, you need to call DuplicateHandle for it.


    Or set bInheritHandle=True when creating the pipe initially. os.pipe()
    doesn't do that.

    > I hit this problem - it looks like pipes aren't very versatile on
    > Windows.
    > There's also the complicating factor that the handles in windows aren't
    > the
    > same as the file numbers that Python uses, so you have to convert between
    > them.
    >
    > It would be nice if Python created pipes that are properly inheritable by
    > default by child processes, as they're mostly used for IPC.


    I'd say it is a bug in os.pipe implementation; they should be inheritable
    by default, as in posix (after all, the code is in "posixmodule.c").

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 17, 2008
    #5
  6. Aaron Brady

    Aaron Brady Guest

    On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    wrote:
    > En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    > <> escribió:
    >
    > > Aaron Brady wrote:

    >
    > >> I thought so too.  The web seems to say that on Linux they are, and on
    > >> Windows, you need to call DuplicateHandle for it.

    >
    > Or set bInheritHandle=True when creating the pipe initially. os.pipe()  
    > doesn't do that.
    >
    > > I hit this problem - it looks like pipes aren't very versatile on  
    > > Windows.
    > > There's also the complicating factor that the handles in windows aren't  
    > > the
    > > same as the file numbers that Python uses, so you have to convert between
    > > them.

    >
    > > It would be nice if Python created pipes that are properly inheritable by
    > > default by child processes, as they're mostly used for IPC.

    >
    > I'd say it is a bug in os.pipe implementation; they should be inheritable  
    > by default, as in posix (after all, the code is in "posixmodule.c").


    The code looks like this:

    ok = CreatePipe(&read, &write, NULL, 0);
    Py_END_ALLOW_THREADS
    if (!ok)
    return win32_error("CreatePipe", NULL);
    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    write_fd = _open_osfhandle((Py_intptr_t)write, 1);

    'If lpPipeAttributes is NULL, the handle cannot be inherited.' You
    could populate a 'SECURITY_ATTRIBUTES' structure, or call
    DuplicateHandle on both of them.

    A patch would look like this:

    SECURITY_ATTRIBUTES sattribs;
    sattribs.nLength = sizeof(sattribs);
    sattribs.lpSecurityDescriptor = NULL;
    sattribs.bInheritHandle = TRUE;
    ok = CreatePipe(&read, &write, &sattribs, 0);

    This still doesn't answer whether the file descriptor return by
    '_open_osfhandle' can be inherited too.
    Aaron Brady, Dec 18, 2008
    #6
  7. En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <>
    escribió:
    > On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    > wrote:
    >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    >> <> escribió:
    >>
    >> > It would be nice if Python created pipes that are properly

    >> inheritable by
    >> > default by child processes, as they're mostly used for IPC.

    >>
    >> I'd say it is a bug in os.pipe implementation; they should be
    >> inheritable  
    >> by default, as in posix (after all, the code is in "posixmodule.c").

    >
    > The code looks like this:
    >
    > ok = CreatePipe(&read, &write, NULL, 0);
    > Py_END_ALLOW_THREADS
    > if (!ok)
    > return win32_error("CreatePipe", NULL);
    > read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    > write_fd = _open_osfhandle((Py_intptr_t)write, 1);
    >
    > 'If lpPipeAttributes is NULL, the handle cannot be inherited.' You
    > could populate a 'SECURITY_ATTRIBUTES' structure, or call
    > DuplicateHandle on both of them.
    >
    > A patch would look like this:
    >
    > SECURITY_ATTRIBUTES sattribs;
    > sattribs.nLength = sizeof(sattribs);
    > sattribs.lpSecurityDescriptor = NULL;
    > sattribs.bInheritHandle = TRUE;
    > ok = CreatePipe(&read, &write, &sattribs, 0);


    Yes, that's exactly how os.popen does it (in posixmodule.c)

    > This still doesn't answer whether the file descriptor return by
    > '_open_osfhandle' can be inherited too.


    It doesn't matter. The OS only cares about file handles, not C RTL
    structures.

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 18, 2008
    #7
  8. Aaron Brady

    Aaron Brady Guest

    On Dec 17, 7:16 pm, "Gabriel Genellina" <>
    wrote:
    > En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <>  
    > escribió:
    >
    >
    >
    > > On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    > > wrote:
    > >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    > >> <> escribió:

    >
    > >> > It would be nice if Python created pipes that are properly  
    > >> inheritable by
    > >> > default by child processes, as they're mostly used for IPC.

    >
    > >> I'd say it is a bug in os.pipe implementation; they should be  
    > >> inheritable  
    > >> by default, as in posix (after all, the code is in "posixmodule.c").

    >
    > > The code looks like this:

    >
    > >    ok = CreatePipe(&read, &write, NULL, 0);
    > >    Py_END_ALLOW_THREADS
    > >    if (!ok)
    > >            return win32_error("CreatePipe", NULL);
    > >    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    > >    write_fd = _open_osfhandle((Py_intptr_t)write, 1);

    >
    > > 'If lpPipeAttributes is NULL, the handle cannot be inherited.'  You
    > > could populate a 'SECURITY_ATTRIBUTES' structure, or call
    > > DuplicateHandle on both of them.

    >
    > > A patch would look like this:

    >
    > > SECURITY_ATTRIBUTES sattribs;
    > > sattribs.nLength = sizeof(sattribs);
    > > sattribs.lpSecurityDescriptor = NULL;
    > > sattribs.bInheritHandle = TRUE;
    > > ok = CreatePipe(&read, &write, &sattribs, 0);

    >
    > Yes, that's exactly how os.popen does it (in posixmodule.c)
    >
    > > This still doesn't answer whether the file descriptor return by
    > > '_open_osfhandle' can be inherited too.

    >
    > It doesn't matter. The OS only cares about file handles, not C RTL  
    > structures.
    >
    > --
    > Gabriel Genellina


    Ah, I see. Was it an executive decision about what is Pythonic, or
    just a bug? Do you think the patch would be accepted? I probably
    ought to mimic a small Python embedding to see if it needs anything
    else.
    Aaron Brady, Dec 18, 2008
    #8
  9. Aaron Brady

    Aaron Brady Guest

    On Dec 17, 7:16 pm, "Gabriel Genellina" <>
    wrote:
    > En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <>  
    > escribió:
    >
    >
    >
    > > On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    > > wrote:
    > >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    > >> <> escribió:

    >
    > >> > It would be nice if Python created pipes that are properly  
    > >> inheritable by
    > >> > default by child processes, as they're mostly used for IPC.

    >
    > >> I'd say it is a bug in os.pipe implementation; they should be  
    > >> inheritable  
    > >> by default, as in posix (after all, the code is in "posixmodule.c").

    >
    > > The code looks like this:

    >
    > >    ok = CreatePipe(&read, &write, NULL, 0);
    > >    Py_END_ALLOW_THREADS
    > >    if (!ok)
    > >            return win32_error("CreatePipe", NULL);
    > >    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    > >    write_fd = _open_osfhandle((Py_intptr_t)write, 1);

    >
    > > 'If lpPipeAttributes is NULL, the handle cannot be inherited.'  You
    > > could populate a 'SECURITY_ATTRIBUTES' structure, or call
    > > DuplicateHandle on both of them.

    >
    > > A patch would look like this:

    >
    > > SECURITY_ATTRIBUTES sattribs;
    > > sattribs.nLength = sizeof(sattribs);
    > > sattribs.lpSecurityDescriptor = NULL;
    > > sattribs.bInheritHandle = TRUE;
    > > ok = CreatePipe(&read, &write, &sattribs, 0);

    >
    > Yes, that's exactly how os.popen does it (in posixmodule.c)
    >
    > > This still doesn't answer whether the file descriptor return by
    > > '_open_osfhandle' can be inherited too.

    >
    > It doesn't matter. The OS only cares about file handles, not C RTL  
    > structures.


    Sorry for the multiple posts. File handles are inheritable by child
    processes, if the permissions are right. File descriptors are not.
    Is there a way that we can get the handles of a pipe into code, so
    that we can pass them to a subprocess? Will it take calling
    'CreatePipe' from ctypes directly if on Windows? Or can 'os.pipe' be
    made to abstract that? If Windows can't inherit descriptors,
    'os.pipe' should return handles, and 'os.read' &co. should accept
    them.

    It is a fairly large patch.
    Aaron Brady, Dec 18, 2008
    #9
  10. En Thu, 18 Dec 2008 08:35:58 -0200, Aaron Brady <>
    escribió:

    > On Dec 17, 7:16 pm, "Gabriel Genellina" <>
    > wrote:
    >> En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <>  
    >> escribió:
    >>
    >>
    >>
    >> > On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    >> > wrote:
    >> >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    >> >> <> escribió:

    >>
    >> >> > It would be nice if Python created pipes that are properly  
    >> >> inheritable by
    >> >> > default by child processes, as they're mostly used for IPC.

    >>
    >> >> I'd say it is a bug in os.pipe implementation; they should be  
    >> >> inheritable  
    >> >> by default, as in posix (after all, the code is in "posixmodule.c").

    >>
    >> > The code looks like this:

    >>
    >> >    ok = CreatePipe(&read, &write, NULL, 0);
    >> >    Py_END_ALLOW_THREADS
    >> >    if (!ok)
    >> >            return win32_error("CreatePipe", NULL);
    >> >    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    >> >    write_fd = _open_osfhandle((Py_intptr_t)write, 1);

    >>
    >> > 'If lpPipeAttributes is NULL, the handle cannot be inherited.'  You
    >> > could populate a 'SECURITY_ATTRIBUTES' structure, or call
    >> > DuplicateHandle on both of them.

    >>
    >> > A patch would look like this:

    >>
    >> > SECURITY_ATTRIBUTES sattribs;
    >> > sattribs.nLength = sizeof(sattribs);
    >> > sattribs.lpSecurityDescriptor = NULL;
    >> > sattribs.bInheritHandle = TRUE;
    >> > ok = CreatePipe(&read, &write, &sattribs, 0);

    >>
    >> Yes, that's exactly how os.popen does it (in posixmodule.c)
    >>
    >> > This still doesn't answer whether the file descriptor return by
    >> > '_open_osfhandle' can be inherited too.

    >>
    >> It doesn't matter. The OS only cares about file handles, not C RTL  
    >> structures.
    >>
    >> --
    >> Gabriel Genellina

    >
    > Ah, I see. Was it an executive decision about what is Pythonic, or
    > just a bug? Do you think the patch would be accepted? I probably
    > ought to mimic a small Python embedding to see if it needs anything
    > else.


    I don't know - I guess someone (years ago) blindly just replaced the
    pipe() system call by a CreatePipe call without further analysis.

    This is how I would summarize the issue:

    Pros (of changing os.pipe() to return inheritable pipes):

    - it isn't explicitely documented whether os.pipe() returns inheritable
    pipes or not, so both versions are "right" according to the documentation.
    - if someone relies on pipes being non-inheritable on Windows, that is
    undocumented behaviour, and Python has the right to change it.
    - the change would improve POSIX compatibility, it mimics what os.pipe()
    does on those OS.
    - inheritable pipes are less surprising for guys coming from other OS
    - inheritable pipes are a lot more useful than non-inheritable ones when
    doing IPC (probably its main usage).

    Cons:

    - os.pipe has behaved that way since long time ago.
    - some programs *might* break, if they relied on pipes being
    non-inheritable on Windows, even if that was undocumented behaviour.

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 19, 2008
    #10
  11. En Thu, 18 Dec 2008 19:46:45 -0200, Aaron Brady <>
    escribió:

    > On Dec 17, 7:16 pm, "Gabriel Genellina" <>
    > wrote:
    >> En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <>  
    >> escribió:
    >>
    >>
    >>
    >> > On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    >> > wrote:
    >> >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    >> >> <> escribió:

    >>
    >> >> > It would be nice if Python created pipes that are properly  
    >> >> inheritable by
    >> >> > default by child processes, as they're mostly used for IPC.

    >>
    >> >> I'd say it is a bug in os.pipe implementation; they should be  
    >> >> inheritable  
    >> >> by default, as in posix (after all, the code is in "posixmodule.c").

    >>
    >> > The code looks like this:

    >>
    >> >    ok = CreatePipe(&read, &write, NULL, 0);
    >> >    Py_END_ALLOW_THREADS
    >> >    if (!ok)
    >> >            return win32_error("CreatePipe", NULL);
    >> >    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    >> >    write_fd = _open_osfhandle((Py_intptr_t)write, 1);

    >>
    >> > 'If lpPipeAttributes is NULL, the handle cannot be inherited.'  You
    >> > could populate a 'SECURITY_ATTRIBUTES' structure, or call
    >> > DuplicateHandle on both of them.

    >>
    >> > A patch would look like this:

    >>
    >> > SECURITY_ATTRIBUTES sattribs;
    >> > sattribs.nLength = sizeof(sattribs);
    >> > sattribs.lpSecurityDescriptor = NULL;
    >> > sattribs.bInheritHandle = TRUE;
    >> > ok = CreatePipe(&read, &write, &sattribs, 0);

    >>
    >> Yes, that's exactly how os.popen does it (in posixmodule.c)
    >>
    >> > This still doesn't answer whether the file descriptor return by
    >> > '_open_osfhandle' can be inherited too.

    >>
    >> It doesn't matter. The OS only cares about file handles, not C RTL  
    >> structures.

    >
    > Sorry for the multiple posts. File handles are inheritable by child
    > processes, if the permissions are right. File descriptors are not.
    > Is there a way that we can get the handles of a pipe into code, so
    > that we can pass them to a subprocess?


    On Windows, file handles are the real OS stuff, the "true" reference to an
    open file. File descriptors are not, they exist only to please the C
    runtime library. Programs not written in C (directly, or indirectly like
    Python) don't care at all about file descriptors. And in case one actually
    cares, there is _open_osfhandle in the C RTL (available as
    msvcrt.open_osfhandle from Python).
    A subprocess may inherit handles from its parent [there are two filters:
    the parameter "bInheritHandles" in the CreateProcess call provides global
    control, and individual handles can be made inheritable or not, before
    creating the new subprocess].
    "Anonymous" pipes are good to replace stdin/stdout/stderr, because there
    is no need to explicitely communicate the handle value to the subprocess:
    one just replaces the corresponding handle with the desired pipe, and the
    subprocess might not even notice it.
    In case this is not enough, one might pass the handle (as a number) in the
    command line, but probably a "named pipe" would be better. As this is not
    transparent for the child process, one must explicitely code such things.

    > Will it take calling
    > 'CreatePipe' from ctypes directly if on Windows? Or can 'os.pipe' be
    > made to abstract that? If Windows can't inherit descriptors,
    > 'os.pipe' should return handles, and 'os.read' &co. should accept
    > them.


    I think the best way would be to modify os.pipe so it returns inheritable
    pipes, as it should have been from the beginning.

    > It is a fairly large patch.


    Not at all, you have already posted most of it.

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 19, 2008
    #11
  12. Aaron Brady

    Aaron Brady Guest

    On Dec 18, 6:11 pm, "Gabriel Genellina" <>
    wrote:
    > En Thu, 18 Dec 2008 08:35:58 -0200, Aaron Brady <>
    > escribió:
    >
    >
    >
    > > On Dec 17, 7:16 pm, "Gabriel Genellina" <>
    > > wrote:
    > >> En Wed, 17 Dec 2008 22:46:32 -0200, Aaron Brady <>  
    > >> escribió:

    >
    > >> > On Dec 17, 5:05 pm, "Gabriel Genellina" <>
    > >> > wrote:
    > >> >> En Wed, 17 Dec 2008 12:21:38 -0200, Jeremy Sanders  
    > >> >> <> escribió:

    >
    > >> >> > It would be nice if Python created pipes that are properly  
    > >> >> inheritable by
    > >> >> > default by child processes, as they're mostly used for IPC.

    >
    > >> >> I'd say it is a bug in os.pipe implementation; they should be  
    > >> >> inheritable  
    > >> >> by default, as in posix (after all, the code is in "posixmodule.c")..

    >
    > >> > The code looks like this:

    >
    > >> >    ok = CreatePipe(&read, &write, NULL, 0);
    > >> >    Py_END_ALLOW_THREADS
    > >> >    if (!ok)
    > >> >            return win32_error("CreatePipe", NULL);
    > >> >    read_fd = _open_osfhandle((Py_intptr_t)read, 0);
    > >> >    write_fd = _open_osfhandle((Py_intptr_t)write, 1);

    >
    > >> > 'If lpPipeAttributes is NULL, the handle cannot be inherited.'  You
    > >> > could populate a 'SECURITY_ATTRIBUTES' structure, or call
    > >> > DuplicateHandle on both of them.

    >
    > >> > A patch would look like this:

    >
    > >> > SECURITY_ATTRIBUTES sattribs;
    > >> > sattribs.nLength = sizeof(sattribs);
    > >> > sattribs.lpSecurityDescriptor = NULL;
    > >> > sattribs.bInheritHandle = TRUE;
    > >> > ok = CreatePipe(&read, &write, &sattribs, 0);

    >
    > >> Yes, that's exactly how os.popen does it (in posixmodule.c)

    >
    > >> > This still doesn't answer whether the file descriptor return by
    > >> > '_open_osfhandle' can be inherited too.

    >
    > >> It doesn't matter. The OS only cares about file handles, not C RTL  
    > >> structures.

    >
    > >> --
    > >> Gabriel Genellina

    >
    > > Ah, I see.  Was it an executive decision about what is Pythonic, or
    > > just a bug?  Do you think the patch would be accepted?  I probably
    > > ought to mimic a small Python embedding to see if it needs anything
    > > else.

    >
    > I don't know - I guess someone (years ago) blindly just replaced the
    > pipe() system call by a CreatePipe call without further analysis.
    >
    > This is how I would summarize the issue:
    >
    > Pros (of changing os.pipe() to return inheritable pipes):
    >
    > - it isn't explicitely documented whether os.pipe() returns inheritable
    > pipes or not, so both versions are "right" according to the documentation..
    > - if someone relies on pipes being non-inheritable on Windows, that is
    > undocumented behaviour, and Python has the right to change it.
    > - the change would improve POSIX compatibility, it mimics what os.pipe()
    > does on those OS.
    > - inheritable pipes are less surprising for guys coming from other OS
    > - inheritable pipes are a lot more useful than non-inheritable ones when
    > doing IPC (probably its main usage).
    >
    > Cons:
    >
    > - os.pipe has behaved that way since long time ago.
    > - some programs *might* break, if they relied on pipes being
    > non-inheritable on Windows, even if that was undocumented behaviour.
    >
    > --
    > Gabriel Genellina


    Hi,

    Microsoft has this example:

    http://msdn.microsoft.com/en-us/library/aa298531.aspx

    It creates two descriptors (not handles) with '_pipe'. Then it spawns
    a subprocess using 'spawnl', which correctly inherits a descriptor.
    So, if 'Popen' could mimic 'spawnl', then 'os.pipe', and consequently,
    'os.read', 'os.write', &c. could stay as is.
    Aaron Brady, Dec 19, 2008
    #12
  13. Aaron Brady

    Aaron Brady Guest

    On Dec 18, 7:21 pm, "Gabriel Genellina" <>
    wrote:
    > En Thu, 18 Dec 2008 19:46:45 -0200, Aaron Brady <>  
    > escribió:

    snip
    > On Windows, file handles are the real OS stuff, the "true" reference to an  
    > open file. File descriptors are not, they exist only to please the C  
    > runtime library. Programs not written in C (directly, or indirectly like  
    > Python) don't care at all about file descriptors. And in case one actually  
    > cares, there is _open_osfhandle in the C RTL (available as  
    > msvcrt.open_osfhandle from Python).
    > A subprocess may inherit handles from its parent [there are two filters:  
    > the parameter "bInheritHandles" in the CreateProcess call provides global  
    > control, and individual handles can be made inheritable or not, before  
    > creating the new subprocess].
    > "Anonymous" pipes are good to replace stdin/stdout/stderr, because there  
    > is no need to explicitely communicate the handle value to the subprocess:  
    > one just replaces the corresponding handle with the desired pipe, and the  
    > subprocess might not even notice it.
    > In case this is not enough, one might pass the handle (as a number) in the  
    > command line, but probably a "named pipe" would be better. As this is not  
    > transparent for the child process, one must explicitely code such things.
    >
    > > Will it take calling
    > > 'CreatePipe' from ctypes directly if on Windows?  Or can 'os.pipe' be
    > > made to abstract that?  If Windows can't inherit descriptors,
    > > 'os.pipe' should return handles, and 'os.read' &co. should accept
    > > them.

    >
    > I think the best way would be to modify os.pipe so it returns inheritable  
    > pipes, as it should have been from the beginning.
    >
    > > It is a fairly large patch.

    >
    > Not at all, you have already posted most of it.


    I have marginally tested the patch on a custom build. It works, but
    there is a catch. The descriptor can't be passed directly to the
    child on the command line. You need to call 'msvcrt.get_osfhandle' on
    the descriptor, pass the result, then call 'msvcrt.open_osfhandle' in
    the child.
    Aaron Brady, Dec 21, 2008
    #13
  14. Aaron Brady

    Aaron Brady Guest

    On Dec 20, 7:06 pm, Aaron Brady <> wrote:
    > On Dec 18, 7:21 pm, "Gabriel Genellina" <>
    > wrote:
    >
    >
    >
    > > En Thu, 18 Dec 2008 19:46:45 -0200, Aaron Brady <>  
    > > escribió:

    > snip
    > > > Will it take calling
    > > > 'CreatePipe' from ctypes directly if on Windows?  Or can 'os.pipe' be
    > > > made to abstract that?  If Windows can't inherit descriptors,
    > > > 'os.pipe' should return handles, and 'os.read' &co. should accept
    > > > them.

    >
    > > I think the best way would be to modify os.pipe so it returns inheritable  
    > > pipes, as it should have been from the beginning.

    >
    > > > It is a fairly large patch.

    >
    > > Not at all, you have already posted most of it.

    >
    > I have marginally tested the patch on a custom build.  It works, but
    > there is a catch.  The descriptor can't be passed directly to the
    > child on the command line.  You need to call 'msvcrt.get_osfhandle' on
    > the descriptor, pass the result, then call 'msvcrt.open_osfhandle' in
    > the child.


    Issue is at:
    http://bugs.python.org/issue4708
    Aaron Brady, Dec 21, 2008
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michele Simionato

    subprocess.Popen

    Michele Simionato, Dec 12, 2004, in forum: Python
    Replies:
    5
    Views:
    17,630
  2. Ivo Woltring
    Replies:
    4
    Views:
    3,522
    Jeff Shannon
    Jan 27, 2005
  3. John Abel
    Replies:
    2
    Views:
    3,267
    John Abel
    May 12, 2005
  4. Earl Eiland
    Replies:
    3
    Views:
    514
    Peter Hansen
    May 16, 2005
  5. File.popen/IO.popen

    , May 20, 2006, in forum: Ruby
    Replies:
    1
    Views:
    213
    Robert Klemme
    May 20, 2006
Loading...

Share This Page