subprocess "handle is invalid" error

Discussion in 'Python' started by Grant Edwards, Apr 18, 2007.

  1. I'm trying to use the py-gnuplot module on windows, and have
    been unable to get it to work reliably under Win2K and WinXP.

    By default, it uses popen(gnuplotcmd,'w'), but in some
    situations that consistently gets an "invalid operand" IOError
    when write() is called on the pipe.

    So I switched to subprocess. It works fine when executed
    "normally" (e.g. "python progname.py"), but when bundled by
    py2exe, it always does this:

    Traceback (most recent call last):
    File "surfedit.py", line 28, in ?
    File "Gnuplot\_Gnuplot.pyc", line 178, in __init__
    File "Gnuplot\gp_win32.pyc", line 117, in __init__
    File "subprocess.pyc", line 533, in __init__
    File "subprocess.pyc", line 607, in _get_handles
    File "subprocess.pyc", line 634, in _make_inheritable
    WindowsError: [Errno 6] The handle is invalid

    How does one troubleshoot errors that happen three layers deep
    in the subprocess module?

    --
    Grant Edwards grante Yow! I'm a fuschia bowling
    at ball somewhere in Brittany
    visi.com
     
    Grant Edwards, Apr 18, 2007
    #1
    1. Advertising

  2. Grant Edwards

    Erik Johnson Guest

    "Grant Edwards" <> wrote in message
    news:...

    <snip stuff about gnu-plot and py2exe problems>
    > How does one troubleshoot errors that happen three layers deep
    > in the subprocess module?


    I think the problem is more likely in how your py2exe is getting
    bundled, rather than any real problem with the subprocess module. I have
    been on the py2exe email list but not really paying much attention... If I
    recall correctly, there are some special considerations when using gnuplot
    with py2exe? I would suggest digging through the py2exe email archives to
    see if you can locate emails on that subject (one copy of which I believe
    can be found at:
    http://sourceforge.net/mailarchive/forum.php?forum_name=py2exe-users), and
    if that doesn't work, to direct your question to the py2exe users list at:


    (I believe the current home page of the py2exe project is at:
    http://sourceforge.net/projects/py2exe/)

    Hope that helps,
    -ej
     
    Erik Johnson, Apr 18, 2007
    #2
    1. Advertising

  3. On 2007-04-18, Erik Johnson <> wrote:
    >
    > "Grant Edwards" <> wrote in message
    > news:...
    >
    ><snip stuff about gnu-plot and py2exe problems>
    >> How does one troubleshoot errors that happen three layers deep
    >> in the subprocess module?

    >
    > I think the problem is more likely in how your py2exe is
    > getting bundled, rather than any real problem with the
    > subprocess module.


    I've switched back to os.popen() since I seem to get the same
    problems with either. The odd thing is that it only seems to
    fail when the current directory is mounted via the network
    _and_ when it's being started by clicking on a file whose
    filetype is associated with the application.

    Running the program from the command line (cmd.exe or bash)
    always works for any current directory. Running the program by
    cliking on a shortcut icon on the desktop always works for any
    'start in' directory.

    The one case that doesn't work is when its run by clicking on
    an associated file on a network drive.

    --
    Grant Edwards grante Yow! Now I'm concentrating
    at on a specific tank battle
    visi.com toward the end of World
    War II!
     
    Grant Edwards, Apr 18, 2007
    #3
  4. Grant Edwards schrieb:
    > I'm trying to use the py-gnuplot module on windows, and have
    > been unable to get it to work reliably under Win2K and WinXP.
    >
    > By default, it uses popen(gnuplotcmd,'w'), but in some
    > situations that consistently gets an "invalid operand" IOError
    > when write() is called on the pipe.
    >
    > So I switched to subprocess. It works fine when executed
    > "normally" (e.g. "python progname.py"), but when bundled by
    > py2exe, it always does this:
    >
    > Traceback (most recent call last):
    > File "surfedit.py", line 28, in ?
    > File "Gnuplot\_Gnuplot.pyc", line 178, in __init__
    > File "Gnuplot\gp_win32.pyc", line 117, in __init__
    > File "subprocess.pyc", line 533, in __init__
    > File "subprocess.pyc", line 607, in _get_handles
    > File "subprocess.pyc", line 634, in _make_inheritable
    > WindowsError: [Errno 6] The handle is invalid
    >
    > How does one troubleshoot errors that happen three layers deep
    > in the subprocess module?
    >


    I think this is a subprocess bug. It is often attributed to py2exe because
    usually developers do never run the script in pythonW.exe instead of python.exe,
    and later build a *windows* program with py2exe (the *windows* program has no
    console, and that triggers the bug).

    Consider ths little script:

    """
    import os, sys, subprocess

    sys.stderr = open("errors.txt", "w")

    if os.path.exists("output.txt"):
    os.remove("output.txt")

    proc = subprocess.Popen("dir", shell=True,
    stdout=subprocess.PIPE,
    ## stderr=subprocess.PIPE,
    ## stdin=subprocess.PIPE,
    )
    ##proc.stderr.close()
    ##proc.stdin.close()
    data = proc.stdout.read()

    open("output.txt", "w").write(data)
    """

    It calls 'dir' in the current directory, and writes the output
    to the file 'output.txt'. Any errors are written to 'errors.txt'.

    When you run this script with python.exe, everything works.
    If the script is run with pythonW.exe, nothing works and 'errors.txt'
    contains this:

    c:\svn\theller>type errors.txt
    Traceback (most recent call last):
    File "test_subproc.py", line 9, in <module>
    stdout=subprocess.PIPE,
    File "c:\python25\lib\subprocess.py", line 586, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
    File "c:\python25\lib\subprocess.py", line 699, in _get_handles
    p2cread = self._make_inheritable(p2cread)
    File "c:\python25\lib\subprocess.py", line 744, in _make_inheritable
    DUPLICATE_SAME_ACCESS)
    WindowsError: [Error 6] Das Handle ist ung├╝ltig
    c:\svn\theller>

    The error message, translated to english, is 'the handle is invalid'.

    The script can be made to work correctly even with pythonW.exe (and also
    as py2exe'd windows program, I just checked it out) when the 4 commented out
    lines are uncommented. subprocess cannot inherit the standard handles when
    the process has no console, you have to create pipes for all 3 channels, and
    close those that are not needed.

    I thought that this bug was fixed in Python2.5.1 (the release candidate),
    but it seems it wasn't. The bug is at
    http://sourceforge.net/tracker/index.php?func=detail&aid=1124861&group_id=5470&atid=105470

    If all this is correct, I hope that someone adds a section to the py2exe wiki;
    and reopens the above bug report.

    Thomas
     
    Thomas Heller, Apr 18, 2007
    #4
  5. On 2007-04-18, Thomas Heller <> wrote:

    > I think this is a subprocess bug. It is often attributed to
    > py2exe because usually developers do never run the script in
    > pythonW.exe instead of python.exe, and later build a *windows*
    > program with py2exe (the *windows* program has no console, and
    > that triggers the bug).


    [...]

    Ah, excellent analysis. It does indeed to appear to be a
    subprocess bug. Switching back to os.popen() makes things
    "work" again. There's a different problem when the program is
    started by double-clicking an associated file in a networked
    directory (works fine in a local directory). That was the
    initial problem I was trying to troubleshoot by switching
    trying subprocess.Popen().

    > The script can be made to work correctly even with pythonW.exe
    > (and also as py2exe'd windows program, I just checked it out)
    > when the 4 commented out lines are uncommented. subprocess
    > cannot inherit the standard handles when the process has no
    > console, you have to create pipes for all 3 channels, and
    > close those that are not needed.


    Perhaps I'll try switching back to subprocess.Popen() and
    creating all three pipes.

    > I thought that this bug was fixed in Python2.5.1 (the release candidate),
    > but it seems it wasn't. The bug is at
    > http://sourceforge.net/tracker/index.php?func=detail&aid=1124861&group_id=5470&atid=105470
    >
    >> If all this is correct, I hope that someone adds a section to the py2exe wiki;


    It had been documented at the py2exe wiki -- though it mentions a
    different exception. I added the bug-tracker link and also
    that it throws the "invalid handle" exception.

    http://www.py2exe.org/index.cgi/Py2ExeSubprocessInteractions

    > and reopens the above bug report.


    Since I'm still using 2.4.3, I don't think it would be
    appropriate for me to do so.

    --
    Grant Edwards grante Yow! It don't mean a
    at THING if you ain't got
    visi.com that SWING!!
     
    Grant Edwards, Apr 18, 2007
    #5
  6. Thomas Heller-2 wrote:
    >
    > Grant Edwards schrieb:
    >
    > [snip]
    >
    >>
    >> Traceback (most recent call last):
    >> File "surfedit.py", line 28, in ?
    >> File "Gnuplot\_Gnuplot.pyc", line 178, in __init__
    >> File "Gnuplot\gp_win32.pyc", line 117, in __init__
    >> File "subprocess.pyc", line 533, in __init__
    >> File "subprocess.pyc", line 607, in _get_handles
    >> File "subprocess.pyc", line 634, in _make_inheritable
    >> WindowsError: [Errno 6] The handle is invalid
    >>
    >> How does one troubleshoot errors that happen three layers deep
    >> in the subprocess module?
    >>

    >
    > I think this is a subprocess bug. It is often attributed to py2exe
    > because
    > usually developers do never run the script in pythonW.exe instead of
    > python.exe,
    > and later build a *windows* program with py2exe (the *windows* program has
    > no
    > console, and that triggers the bug).
    >
    > [snip]
    >
    > I thought that this bug was fixed in Python2.5.1 (the release candidate),
    > but it seems it wasn't. The bug is at
    > http://sourceforge.net/tracker/index.php?func=detail&aid=1124861&group_id=5470&atid=105470
    >
    > If all this is correct, I hope that someone adds a section to the py2exe
    > wiki;
    > and reopens the above bug report.
    >
    > Thomas
    >

    I've been bitten by this same bug in 2.5.1, and the bug you linked to (which
    is now @ http://bugs.python.org/issue1124861) is still marked as fixed.

    I've tried to reopen it but it seems only the reporter (or some "admin"
    user) is able to do it.

    --
    View this message in context: http://www.nabble.com/subprocess-"handle-is-invalid"-error-tp10060967p14740517.html
    Sent from the Python - python-list mailing list archive at Nabble.com.
     
    Daniel Serodio, Jan 10, 2008
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?RWN1dGVy?=

    The handle is invalid

    =?Utf-8?B?RWN1dGVy?=, Dec 19, 2006, in forum: ASP .Net
    Replies:
    1
    Views:
    2,570
  2. Gunawan
    Replies:
    0
    Views:
    397
    Gunawan
    Feb 27, 2007
  3. 7stud

    subprocess -- broken pipe error

    7stud, Jul 2, 2007, in forum: Python
    Replies:
    11
    Views:
    1,260
    Steve Holden
    Jul 2, 2007
  4. hiral
    Replies:
    2
    Views:
    633
    Jean-Michel Pichavant
    May 5, 2010
  5. John Briggs
    Replies:
    10
    Views:
    217
    Luis Lavena
    Apr 21, 2010
Loading...

Share This Page