Subprocess confusion: how file-like must stdin be?

Discussion in 'Python' started by Cameron Laird, Aug 17, 2006.

  1. Question:
    import subprocess, StringIO

    input = StringIO.StringIO("abcdefgh\nabc\n")
    # I don't know of a compact, evocative, and
    # cross-platform way to exhibit this behavior.
    # For now, depend on cat(1).
    p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
    stdin = response)

    Why this is a question:
    A. it tosses an AttributeError.
    B. I *expected* it to do the equivalent of
    cat << HERE
    abcdefgh
    abc
    HERE

    In <URL: http://docs.python.org/dev/lib/node530.html >, I read "Valid
    values are ... an existing file object ..." Even though StringIO is
    a "file-like object", it lacks a fileno. Is there a way to get what
    I'm after?
    Cameron Laird, Aug 17, 2006
    #1
    1. Advertising

  2. On Thu, 17 Aug 2006 17:16:25 +0000, (Cameron Laird)
    declaimed the following in comp.lang.python:

    > Question:
    > import subprocess, StringIO
    >
    > input = StringIO.StringIO("abcdefgh\nabc\n")


    Here you override the builtin function "input()"
    > # I don't know of a compact, evocative, and
    > # cross-platform way to exhibit this behavior.
    > # For now, depend on cat(1).
    > p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
    > stdin = response)


    Here you specify the non-existant "response"
    --
    Wulfraed Dennis Lee Bieber KD6MOG

    HTTP://wlfraed.home.netcom.com/
    (Bestiaria Support Staff: )
    HTTP://www.bestiaria.com/
    Dennis Lee Bieber, Aug 18, 2006
    #2
    1. Advertising

  3. Dennis Lee Bieber <> wrote:
    > On Thu, 17 Aug 2006 17:16:25 +0000, (Cameron Laird)
    > declaimed the following in comp.lang.python:
    >
    > > Question:
    > > import subprocess, StringIO
    > >
    > > input = StringIO.StringIO("abcdefgh\nabc\n")

    >
    > Here you override the builtin function "input()"
    > > # I don't know of a compact, evocative, and
    > > # cross-platform way to exhibit this behavior.
    > > # For now, depend on cat(1).
    > > p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
    > > stdin = response)

    >
    > Here you specify the non-existant "response"


    Assume the OP meant to write this

    >>> import subprocess, StringIO
    >>> inp = StringIO.StringIO("abcdefgh\nabc\n")
    >>> p = subprocess.Popen(["cat"], stdout = subprocess.PIPE, stdin = inp)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "/usr/lib/python2.4/subprocess.py", line 534, in __init__
    (p2cread, p2cwrite,
    File "/usr/lib/python2.4/subprocess.py", line 830, in _get_handles
    p2cread = stdin.fileno()
    AttributeError: StringIO instance has no attribute 'fileno'
    >>>




    --
    Nick Craig-Wood <> -- http://www.craig-wood.com/nick
    Nick Craig-Wood, Aug 18, 2006
    #3
  4. In article <-wood.com>,
    Nick Craig-Wood <> wrote:
    >Dennis Lee Bieber <> wrote:
    >> On Thu, 17 Aug 2006 17:16:25 +0000, (Cameron Laird)
    >> declaimed the following in comp.lang.python:
    >>
    >> > Question:
    >> > import subprocess, StringIO
    >> >
    >> > input = StringIO.StringIO("abcdefgh\nabc\n")

    >>
    >> Here you override the builtin function "input()"
    >> > # I don't know of a compact, evocative, and
    >> > # cross-platform way to exhibit this behavior.
    >> > # For now, depend on cat(1).
    >> > p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
    >> > stdin = response)

    >>
    >> Here you specify the non-existant "response"

    >
    >Assume the OP meant to write this
    >
    >>>> import subprocess, StringIO
    >>>> inp = StringIO.StringIO("abcdefgh\nabc\n")
    >>>> p = subprocess.Popen(["cat"], stdout = subprocess.PIPE, stdin = inp)

    >Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > File "/usr/lib/python2.4/subprocess.py", line 534, in __init__
    > (p2cread, p2cwrite,
    > File "/usr/lib/python2.4/subprocess.py", line 830, in _get_handles
    > p2cread = stdin.fileno()
    >AttributeError: StringIO instance has no attribute 'fileno'
    >>>>

    .
    .
    .
    Yes; my apologies for the confusion I introduced by "editing
    for publication", and doing it badly.

    Your interactive session does indeed exhibit the behavior that
    puzzles me. My expectation was that StringIO and the std*
    parameters to Popen() were made for each other; certainly there
    are many cases where stdout and stderr can be redirected *to* a
    StringIO. Is it simply the case that stdin demands a more
    file-like object? While that disappoints me, I certainly can
    program around it. My question, then: does stdin effectively
    require something really in the filesystem, or perhaps the
    stdout of a previous subprocess? Is there no built-in way to
    feed it an in-memory construct?
    Cameron Laird, Aug 18, 2006
    #4
  5. Cameron Laird a écrit :
    > In article <-wood.com>,
    > Nick Craig-Wood <> wrote:
    >> Dennis Lee Bieber <> wrote:
    >>> On Thu, 17 Aug 2006 17:16:25 +0000, (Cameron Laird)
    >>> declaimed the following in comp.lang.python:
    >>>
    >>>> Question:
    >>>> import subprocess, StringIO
    >>>>
    >>>> input = StringIO.StringIO("abcdefgh\nabc\n")
    >>> Here you override the builtin function "input()"
    >>>> # I don't know of a compact, evocative, and
    >>>> # cross-platform way to exhibit this behavior.
    >>>> # For now, depend on cat(1).
    >>>> p = subprocess.Popen(["cat"], stdout = subprocess.PIPE,
    >>>> stdin = response)
    >>> Here you specify the non-existant "response"

    >> Assume the OP meant to write this
    >>
    >>>>> import subprocess, StringIO
    >>>>> inp = StringIO.StringIO("abcdefgh\nabc\n")
    >>>>> p = subprocess.Popen(["cat"], stdout = subprocess.PIPE, stdin = inp)

    >> Traceback (most recent call last):
    >> File "<stdin>", line 1, in ?
    >> File "/usr/lib/python2.4/subprocess.py", line 534, in __init__
    >> (p2cread, p2cwrite,
    >> File "/usr/lib/python2.4/subprocess.py", line 830, in _get_handles
    >> p2cread = stdin.fileno()
    >> AttributeError: StringIO instance has no attribute 'fileno'

    > .
    > .
    > .
    > Yes; my apologies for the confusion I introduced by "editing
    > for publication", and doing it badly.
    >
    > Your interactive session does indeed exhibit the behavior that
    > puzzles me. My expectation was that StringIO and the std*
    > parameters to Popen() were made for each other; certainly there
    > are many cases where stdout and stderr can be redirected *to* a
    > StringIO. Is it simply the case that stdin demands a more
    > file-like object? While that disappoints me, I certainly can
    > program around it. My question, then: does stdin effectively
    > require something really in the filesystem, or perhaps the
    > stdout of a previous subprocess? Is there no built-in way to
    > feed it an in-memory construct?


    As this is a pipe at OS level, there may be no other way than using
    os-level tools (ie. real files with fileno), maybe creating an anonymous
    pipe, writing to it (relying on pipe buffering by the OS to avoid the
    creation of a writer thread), and giving this pipe (which should have a
    fileno) to the subprocess.Popen stdin parameter.
    Such a construction (pipe/writer thread) would be welcome as standard
    subprocess tool.


    A+

    Laurent.
    Laurent Pointal, Aug 18, 2006
    #5
  6. Cameron Laird wrote:

    > Your interactive session does indeed exhibit the behavior that
    > puzzles me. My expectation was that StringIO and the std*
    > parameters to Popen() were made for each other; certainly there
    > are many cases where stdout and stderr can be redirected *to* a
    > StringIO. Is it simply the case that stdin demands a more
    > file-like object? While that disappoints me, I certainly can
    > program around it. My question, then: does stdin effectively
    > require something really in the filesystem, or perhaps the
    > stdout of a previous subprocess? Is there no built-in way to
    > feed it an in-memory construct?


    set the appropriate stream to subprocess.PIPE, and write to it.

    p = subprocess.Popen(..., stdin=subprocess.PIPE)
    p.stdin.write("hello")
    p.stdin.close() # signal end of file

    </F>
    Fredrik Lundh, Aug 18, 2006
    #6
  7. In article <>,
    Fredrik Lundh <> wrote:
    >Cameron Laird wrote:
    >
    >> Your interactive session does indeed exhibit the behavior that
    >> puzzles me. My expectation was that StringIO and the std*
    >> parameters to Popen() were made for each other; certainly there
    >> are many cases where stdout and stderr can be redirected *to* a
    >> StringIO. Is it simply the case that stdin demands a more
    >> file-like object? While that disappoints me, I certainly can
    >> program around it. My question, then: does stdin effectively
    >> require something really in the filesystem, or perhaps the
    >> stdout of a previous subprocess? Is there no built-in way to
    >> feed it an in-memory construct?

    >
    >set the appropriate stream to subprocess.PIPE, and write to it.
    >
    > p = subprocess.Popen(..., stdin=subprocess.PIPE)
    > p.stdin.write("hello")
    > p.stdin.close() # signal end of file

    .
    .
    .
    Of course! My; it's a sign of how far I was from the right
    subprocess mentality that I so missed what is obvious after
    the fact. The source code (for the 2.4 release) also comes
    close to making it clear, as I realized about the time you
    posted your follow-up, Fredrik.

    Is this--streaming data to a subprocess--too lightweight to
    deserve write-up in the (online) Cookbook? I'll volunteer.
    In retrospect, I see there are plenty of examples on-line
    of the usage, but I don't think an ignorant searcher (like
    me) will find them on his own.

    Thanks, Fredrik.
    Cameron Laird, Aug 18, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Johnathan Doe

    peek at stdin, flush stdin

    Johnathan Doe, May 15, 2004, in forum: C Programming
    Replies:
    5
    Views:
    24,844
    Chatoyer
    May 17, 2013
  2. Charlie Zender

    Reading stdin once confuses second stdin read

    Charlie Zender, Jun 19, 2004, in forum: C Programming
    Replies:
    6
    Views:
    770
    Dan Pop
    Jun 21, 2004
  3. Ben
    Replies:
    2
    Views:
    1,327
    jacob navia
    Aug 29, 2009
  4. hiral
    Replies:
    2
    Views:
    587
    Jean-Michel Pichavant
    May 5, 2010
  5. Terry Cooper
    Replies:
    7
    Views:
    420
    Janos Sebok
    Jun 9, 2009
Loading...

Share This Page