subprocess considered harmfull?

Discussion in 'Python' started by Uri Nix, Sep 25, 2005.

  1. Uri Nix

    Uri Nix Guest

    Hi all,

    I've been trying to use (Python 2.4 on WinXP) the subprocess module to
    execute a shell command (nmake in this case), and pass its output to a
    higher level.

    Using the following snippet:
    p =
    subprocess.Popen(nmake,stderr=subprocess.PIPE,stdout=subprocess.PIPE, \
    universal_newlines=True, bufsize=1)
    os.sys.stdout.writelines(p.stdout)
    os.sys.stdout.writelines(p.stderr)
    Works fine on the command line, but fails when called from within
    Visual Studio, with the following error:
    File "C:\Python24\lib\subprocess.py", line 549, in __init__
    (p2cread, p2cwrite,
    File "C:\Python24\lib\subprocess.py", line 609, in _get_handles
    p2cread = self._make_inheritable(p2cread)
    File "C:\Python24\lib\subprocess.py", line 650, in _make_inheritable
    DUPLICATE_SAME_ACCESS)
    TypeError: an integer is required

    If I replace the functionality with:
    p = os.popen4(nmake)
    # p[1] = stdout_and_stderr result pipe
    p[1].flush()
    os.sys.stdout.writelines(p[1].readlines())
    All is well.

    I have a feeling this has been encountered before (by googling here),
    but didn't see any concise answer as to subprocess' robustness.
    So what is the matter here? And should I consider the subprocess
    module still unstable?

    Cheers,
    Uri
     
    Uri Nix, Sep 25, 2005
    #1
    1. Advertising

  2. Hi also !

    In other fields, I also found uses which did not function with subprocess,
    but OK with popen2/4

    @-salutations

    Michel Claveau
     
    Do Re Mi chel La Si Do, Sep 25, 2005
    #2
    1. Advertising

  3. Uri Nix wrote:
    > Using the following snippet:
    > p =
    > subprocess.Popen(nmake,stderr=subprocess.PIPE,stdout=subprocess.PIPE, \
    > universal_newlines=True, bufsize=1)
    > os.sys.stdout.writelines(p.stdout)
    > os.sys.stdout.writelines(p.stderr)
    > Works fine on the command line, but fails when called from within
    > Visual Studio, with the following error:
    > File "C:\Python24\lib\subprocess.py", line 549, in __init__
    > (p2cread, p2cwrite,
    > File "C:\Python24\lib\subprocess.py", line 609, in _get_handles
    > p2cread = self._make_inheritable(p2cread)
    > File "C:\Python24\lib\subprocess.py", line 650, in _make_inheritable
    > DUPLICATE_SAME_ACCESS)
    > TypeError: an integer is required


    This looks like these known bugs:
    http://python.org/sf/1124861
    http://python.org/sf/1126208

    Try setting stderr to subprocess.PIPE. I think that was what worked for
    me. (You might also try setting shell=True. That's what I currently
    have in my code that didn't work before.)

    STeVe
     
    Steven Bethard, Sep 25, 2005
    #3
  4. Steven Bethard wrote:

    > > Using the following snippet:
    > > p =
    > > subprocess.Popen(nmake,stderr=subprocess.PIPE,stdout=subprocess.PIPE, \
    > > universal_newlines=True, bufsize=1)
    > > os.sys.stdout.writelines(p.stdout)
    > > os.sys.stdout.writelines(p.stderr)
    > > Works fine on the command line, but fails when called from within
    > > Visual Studio, with the following error:
    > > File "C:\Python24\lib\subprocess.py", line 549, in __init__
    > > (p2cread, p2cwrite,
    > > File "C:\Python24\lib\subprocess.py", line 609, in _get_handles
    > > p2cread = self._make_inheritable(p2cread)
    > > File "C:\Python24\lib\subprocess.py", line 650, in _make_inheritable
    > > DUPLICATE_SAME_ACCESS)
    > > TypeError: an integer is required

    >
    > This looks like these known bugs:
    > http://python.org/sf/1124861
    > http://python.org/sf/1126208
    >
    > Try setting stderr to subprocess.PIPE. I think that was what worked for
    > me. (You might also try setting shell=True. That's what I currently
    > have in my code that didn't work before.)


    if someone wants to investigate, is seeing this problem, and have the win32
    extensions on their machine, try changing this line in subprocess.py:

    if 0: # <-- change this to use pywin32 instead of the _subprocess driver

    to:

    if 1: # <-- change this to use _subprocess instead of the pywin32 driver

    and see if it either fixes the problem (not very likely) or gives you a better
    error message (very likely).

    </F>
     
    Fredrik Lundh, Sep 25, 2005
    #4
  5. Uri Nix

    Roger Upole Guest

    "Fredrik Lundh" <> wrote in message news:...
    > Steven Bethard wrote:
    >
    >> > Using the following snippet:
    >> > p =
    >> > subprocess.Popen(nmake,stderr=subprocess.PIPE,stdout=subprocess.PIPE, \
    >> > universal_newlines=True, bufsize=1)
    >> > os.sys.stdout.writelines(p.stdout)
    >> > os.sys.stdout.writelines(p.stderr)
    >> > Works fine on the command line, but fails when called from within
    >> > Visual Studio, with the following error:
    >> > File "C:\Python24\lib\subprocess.py", line 549, in __init__
    >> > (p2cread, p2cwrite,
    >> > File "C:\Python24\lib\subprocess.py", line 609, in _get_handles
    >> > p2cread = self._make_inheritable(p2cread)
    >> > File "C:\Python24\lib\subprocess.py", line 650, in _make_inheritable
    >> > DUPLICATE_SAME_ACCESS)
    >> > TypeError: an integer is required

    >>
    >> This looks like these known bugs:
    >> http://python.org/sf/1124861
    >> http://python.org/sf/1126208
    >>
    >> Try setting stderr to subprocess.PIPE. I think that was what worked for
    >> me. (You might also try setting shell=True. That's what I currently
    >> have in my code that didn't work before.)

    >
    > if someone wants to investigate, is seeing this problem, and have the win32
    > extensions on their machine, try changing this line in subprocess.py:
    >
    > if 0: # <-- change this to use pywin32 instead of the _subprocess driver
    >
    > to:
    >
    > if 1: # <-- change this to use _subprocess instead of the pywin32 driver
    >
    > and see if it either fixes the problem (not very likely) or gives you a better
    > error message (very likely).
    >
    > </F>
    >


    The error msg is only slightly better:

    error: (6, 'DuplicateHandle', 'The handle is invalid.')

    Basically, gui apps like VS don't have a console, so
    GetStdHandle returns 0. _subprocess.GetStdHandle
    returns None if the handle is 0, which gives the original
    error. Pywin32 just returns the 0, so the process gets
    one step further but still hits the above error.

    Subprocess.py should probably check the
    result of GetStdHandle for None (or 0)
    and throw a readable error that says something like
    "No standard handle available, you must specify one"

    Roger




    ----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
    http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
    ----= East and West-Coast Server Farms - Total Privacy via Encryption =----
     
    Roger Upole, Sep 26, 2005
    #5
  6. Uri Nix

    Uri Nix Guest

    Roger Upole wrote:
    > "Fredrik Lundh" <> wrote in message news:...
    > > Steven Bethard wrote:
    > >
    > >> > Using the following snippet:
    > >> > p =
    > >> > subprocess.Popen(nmake,stderr=subprocess.PIPE,stdout=subprocess.PIPE, \
    > >> > universal_newlines=True, bufsize=1)
    > >> > os.sys.stdout.writelines(p.stdout)
    > >> > os.sys.stdout.writelines(p.stderr)
    > >> > Works fine on the command line, but fails when called from within
    > >> > Visual Studio, with the following error:
    > >> > File "C:\Python24\lib\subprocess.py", line 549, in __init__
    > >> > (p2cread, p2cwrite,
    > >> > File "C:\Python24\lib\subprocess.py", line 609, in _get_handles
    > >> > p2cread = self._make_inheritable(p2cread)
    > >> > File "C:\Python24\lib\subprocess.py", line 650, in _make_inheritable
    > >> > DUPLICATE_SAME_ACCESS)
    > >> > TypeError: an integer is required
    > >>
    > >> This looks like these known bugs:
    > >> http://python.org/sf/1124861
    > >> http://python.org/sf/1126208
    > >>
    > >> Try setting stderr to subprocess.PIPE. I think that was what worked for
    > >> me. (You might also try setting shell=True. That's what I currently
    > >> have in my code that didn't work before.)

    > >
    > > if someone wants to investigate, is seeing this problem, and have the win32
    > > extensions on their machine, try changing this line in subprocess.py:
    > >
    > > if 0: # <-- change this to use pywin32 instead of the _subprocess driver
    > >
    > > to:
    > >
    > > if 1: # <-- change this to use _subprocess instead of the pywin32 driver
    > >
    > > and see if it either fixes the problem (not very likely) or gives you a better
    > > error message (very likely).
    > >
    > > </F>
    > >

    >
    > The error msg is only slightly better:
    >
    > error: (6, 'DuplicateHandle', 'The handle is invalid.')
    >
    > Basically, gui apps like VS don't have a console, so
    > GetStdHandle returns 0. _subprocess.GetStdHandle
    > returns None if the handle is 0, which gives the original
    > error. Pywin32 just returns the 0, so the process gets
    > one step further but still hits the above error.
    >
    > Subprocess.py should probably check the
    > result of GetStdHandle for None (or 0)
    > and throw a readable error that says something like
    > "No standard handle available, you must specify one"
    >
    > Roger
    >
    >
    >
    >
    > ----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
    > http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
    > ----= East and West-Coast Server Farms - Total Privacy via Encryption =----


    I gathered as much about why this happens in VS. A further question is
    why does n't os.popen fall in the same trap?

    Cheers,
    Uri
     
    Uri Nix, Sep 26, 2005
    #6
  7. Uri Nix

    Roger Upole Guest

    "Uri Nix" <> wrote in message news:...
    > Roger Upole wrote:
    >> "Fredrik Lundh" <> wrote in message news:...
    >> > Steven Bethard wrote:
    >> >
    >> >> > Using the following snippet:
    >> >> > p =
    >> >> > subprocess.Popen(nmake,stderr=subprocess.PIPE,stdout=subprocess.PIPE, \
    >> >> > universal_newlines=True, bufsize=1)
    >> >> > os.sys.stdout.writelines(p.stdout)
    >> >> > os.sys.stdout.writelines(p.stderr)
    >> >> > Works fine on the command line, but fails when called from within
    >> >> > Visual Studio, with the following error:
    >> >> > File "C:\Python24\lib\subprocess.py", line 549, in __init__
    >> >> > (p2cread, p2cwrite,
    >> >> > File "C:\Python24\lib\subprocess.py", line 609, in _get_handles
    >> >> > p2cread = self._make_inheritable(p2cread)
    >> >> > File "C:\Python24\lib\subprocess.py", line 650, in _make_inheritable
    >> >> > DUPLICATE_SAME_ACCESS)
    >> >> > TypeError: an integer is required
    >> >>
    >> >> This looks like these known bugs:
    >> >> http://python.org/sf/1124861
    >> >> http://python.org/sf/1126208
    >> >>
    >> >> Try setting stderr to subprocess.PIPE. I think that was what worked for
    >> >> me. (You might also try setting shell=True. That's what I currently
    >> >> have in my code that didn't work before.)
    >> >
    >> > if someone wants to investigate, is seeing this problem, and have the win32
    >> > extensions on their machine, try changing this line in subprocess.py:
    >> >
    >> > if 0: # <-- change this to use pywin32 instead of the _subprocess driver
    >> >
    >> > to:
    >> >
    >> > if 1: # <-- change this to use _subprocess instead of the pywin32 driver
    >> >
    >> > and see if it either fixes the problem (not very likely) or gives you a better
    >> > error message (very likely).
    >> >
    >> > </F>
    >> >

    >>
    >> The error msg is only slightly better:
    >>
    >> error: (6, 'DuplicateHandle', 'The handle is invalid.')
    >>
    >> Basically, gui apps like VS don't have a console, so
    >> GetStdHandle returns 0. _subprocess.GetStdHandle
    >> returns None if the handle is 0, which gives the original
    >> error. Pywin32 just returns the 0, so the process gets
    >> one step further but still hits the above error.
    >>
    >> Subprocess.py should probably check the
    >> result of GetStdHandle for None (or 0)
    >> and throw a readable error that says something like
    >> "No standard handle available, you must specify one"
    >>
    >> Roger
    >>
    >>
    >>
    >>
    >> ----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
    >> http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
    >> ----= East and West-Coast Server Farms - Total Privacy via Encryption =----

    >
    > I gathered as much about why this happens in VS. A further question is
    > why does n't os.popen fall in the same trap?
    >
    > Cheers,
    > Uri
    >



    From a quick glance at the source, it looks like it always
    creates new pipes.

    Roger



    ----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
    http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
    ----= East and West-Coast Server Farms - Total Privacy via Encryption =----
     
    Roger Upole, Sep 27, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Woody Splawn

    What is considered orthodox?

    Woody Splawn, May 7, 2004, in forum: ASP .Net
    Replies:
    4
    Views:
    328
    David Jessee
    May 8, 2004
  2. Replies:
    4
    Views:
    539
    Ajay Patil
    Nov 26, 2003
  3. David Combs
    Replies:
    0
    Views:
    333
    David Combs
    Jul 8, 2004
  4. hiral
    Replies:
    2
    Views:
    614
    Jean-Michel Pichavant
    May 5, 2010
  5. Josef 'Jupp' Schugt

    [Alpha Delta Victor] prefix considered harmfull

    Josef 'Jupp' Schugt, Dec 23, 2004, in forum: Ruby
    Replies:
    0
    Views:
    142
    Josef 'Jupp' Schugt
    Dec 23, 2004
Loading...

Share This Page