subprocess.Popen - file like object from stdout=PIPE

Discussion in 'Python' started by Helmut Jarausch, Feb 4, 2009.

  1. Hi,

    using e.g.
    import subprocess
    Package='app-arch/lzma-utils'
    EQ=subprocess.Popen(['/usr/bin/equery','depends',Package],stdout=subprocess.PIPE)
    EQ_output= EQ.communicate()[0]

    EQ_output is a string containing multiple lines.

    I'd prefer a file-like object, e.g. EQ_OUT
    so that I can loop over the lines of it like

    for line in EQ_OUT :
    ...

    I could use StringIO.StringIO applied to EQ_output
    but this reads all of the command's output into a big
    string first.

    On Unix/Linux a pipe is a file-like object after all,
    so how to get hold of it.


    Many thanks for a hint,
    Helmut.

    --
    Helmut Jarausch

    Lehrstuhl fuer Numerische Mathematik
    RWTH - Aachen University
    D 52056 Aachen, Germany
    Helmut Jarausch, Feb 4, 2009
    #1
    1. Advertising

  2. Helmut Jarausch

    Chris Rebert Guest

    On Wed, Feb 4, 2009 at 1:22 AM, Helmut Jarausch <> wrote:
    > Hi,
    >
    > using e.g.
    > import subprocess
    > Package='app-arch/lzma-utils'
    > EQ=subprocess.Popen(['/usr/bin/equery','depends',Package],stdout=subprocess.PIPE)
    > EQ_output= EQ.communicate()[0]
    >
    > EQ_output is a string containing multiple lines.
    >
    > I'd prefer a file-like object, e.g. EQ_OUT
    > so that I can loop over the lines of it like
    >
    > for line in EQ_OUT :
    > ...


    Is there some reason that:

    for line in EQ_OUT.splitlines():
    #...

    Does not meet your needs?

    Cheers,
    Chris

    --
    Follow the path of the Iguana...
    http://rebertia.com
    Chris Rebert, Feb 4, 2009
    #2
    1. Advertising

  3. Chris Rebert wrote:
    > On Wed, Feb 4, 2009 at 1:22 AM, Helmut Jarausch <> wrote:
    >> Hi,
    >>
    >> using e.g.
    >> import subprocess
    >> Package='app-arch/lzma-utils'
    >> EQ=subprocess.Popen(['/usr/bin/equery','depends',Package],stdout=subprocess.PIPE)
    >> EQ_output= EQ.communicate()[0]
    >>
    >> EQ_output is a string containing multiple lines.
    >>
    >> I'd prefer a file-like object, e.g. EQ_OUT
    >> so that I can loop over the lines of it like
    >>
    >> for line in EQ_OUT :
    >> ...

    >
    > Is there some reason that:
    >
    > for line in EQ_OUT.splitlines():
    > #...
    >
    > Does not meet your needs?
    >


    It works, but it still reads the complete output of the
    command executed by Popen at once.

    Helmut.


    --
    Helmut Jarausch

    Lehrstuhl fuer Numerische Mathematik
    RWTH - Aachen University
    D 52056 Aachen, Germany
    Helmut Jarausch, Feb 4, 2009
    #3
  4. 2009/2/4 Helmut Jarausch <>:
    > using e.g.
    > import subprocess
    > Package='app-arch/lzma-utils'
    > EQ=subprocess.Popen(['/usr/bin/equery','depends',Package],stdout=subprocess.PIPE)
    > EQ_output= EQ.communicate()[0]
    > EQ_output is a string containing multiple lines.
    > I'd prefer a file-like object, e.g. EQ_OUT
    > so that I can loop over the lines of it like


    EQ.stdout is the filelike object you're looking for.
    communicate() grabs entire output at once so don't use it.

    import subprocess
    Package = 'app-arch/lzma-utils'
    EQ = subprocess.Popen(['/usr/bin/equery', 'depends', Package],
    stdout=subprocess.PIPE)
    for line in EQ.stdout:
    do_stuff_with(line)
    Clovis Fabricio, Feb 4, 2009
    #4
  5. Clovis Fabricio wrote:
    > 2009/2/4 Helmut Jarausch <>:
    >> using e.g.
    >> import subprocess
    >> Package='app-arch/lzma-utils'
    >> EQ=subprocess.Popen(['/usr/bin/equery','depends',Package],stdout=subprocess.PIPE)
    >> EQ_output= EQ.communicate()[0]
    >> EQ_output is a string containing multiple lines.
    >> I'd prefer a file-like object, e.g. EQ_OUT
    >> so that I can loop over the lines of it like

    >
    > EQ.stdout is the filelike object you're looking for.
    > communicate() grabs entire output at once so don't use it.
    >
    > import subprocess
    > Package = 'app-arch/lzma-utils'
    > EQ = subprocess.Popen(['/usr/bin/equery', 'depends', Package],
    > stdout=subprocess.PIPE)
    > for line in EQ.stdout:
    > do_stuff_with(line)


    Thanks a lot, I haven't found that in the official documentation.
    Helmut.

    --
    Helmut Jarausch

    Lehrstuhl fuer Numerische Mathematik
    RWTH - Aachen University
    D 52056 Aachen, Germany
    Helmut Jarausch, Feb 4, 2009
    #5
  6. Clovis Fabricio wrote:
    > 2009/2/4 Helmut Jarausch <>:
    >> using e.g.
    >> import subprocess
    >> Package='app-arch/lzma-utils'
    >> EQ=subprocess.Popen(['/usr/bin/equery','depends',Package],stdout=subprocess.PIPE)
    >> EQ_output= EQ.communicate()[0]
    >> EQ_output is a string containing multiple lines.
    >> I'd prefer a file-like object, e.g. EQ_OUT
    >> so that I can loop over the lines of it like

    >
    > EQ.stdout is the filelike object you're looking for.
    > communicate() grabs entire output at once so don't use it.
    >
    > import subprocess
    > Package = 'app-arch/lzma-utils'
    > EQ = subprocess.Popen(['/usr/bin/equery', 'depends', Package],
    > stdout=subprocess.PIPE)
    > for line in EQ.stdout:
    > do_stuff_with(line)


    Thanks a lot, I haven't found that in the official documentation.
    Helmut.

    --
    Helmut Jarausch

    Lehrstuhl fuer Numerische Mathematik
    RWTH - Aachen University
    D 52056 Aachen, Germany
    Helmut Jarausch, Feb 4, 2009
    #6
  7. 2009/2/4 Helmut Jarausch <>:
    >> EQ.stdout is the filelike object you're looking for.
    >> communicate() grabs entire output at once so don't use it.

    > Thanks a lot, I haven't found that in the official documentation.
    > Helmut.


    That would be a documentation bug.
    Fortunately it is not true. Here is it in the documentation:

    http://docs.python.org/library/subprocess.html#subprocess.Popen.stdout
    Clovis Fabricio, Feb 4, 2009
    #7
  8. Clovis Fabricio wrote:
    > 2009/2/4 Helmut Jarausch <>:
    >>> EQ.stdout is the filelike object you're looking for.
    >>> communicate() grabs entire output at once so don't use it.

    >> Thanks a lot, I haven't found that in the official documentation.
    >> Helmut.

    >
    > That would be a documentation bug.
    > Fortunately it is not true. Here is it in the documentation:
    >
    > http://docs.python.org/library/subprocess.html#subprocess.Popen.stdout


    Thanks!

    For people like me which don't always read until the end of a document
    it would be nice if just after the paragraph

    stdin, stdout and stderr specify the executed programs’ standard input, standard output and standard error file handles,
    respectively. Valid values are PIPE, an existing file descriptor (a positive integer), an existing file object, and
    None. PIPE indicates that a new pipe to the child should be created. With None, no redirection will occur; the child’s
    file handles will be inherited from the parent. Additionally, stderr can be STDOUT, which indicates that the stderr data
    from the applications should be captured into the same file handle as for stdout.

    there would some lines like

    These pipes are exposed as file-like objects which can be accessed via the
    stdin, stdout or stderr (resp.) attributes of an object of class subprocess.Popen .

    Please be indulgent to people like me.

    Helmut.


    --
    Helmut Jarausch

    Lehrstuhl fuer Numerische Mathematik
    RWTH - Aachen University
    D 52056 Aachen, Germany
    Helmut Jarausch, Feb 4, 2009
    #8
  9. Helmut Jarausch

    Aahz Guest

    In article <49899185$0$2861$>,
    Helmut Jarausch <> wrote:
    >
    >These pipes are exposed as file-like objects which can be accessed via
    >the stdin, stdout or stderr (resp.) attributes of an object of class
    >subprocess.Popen .


    Please file a doc request at bugs.python.org
    --
    Aahz () <*> http://www.pythoncraft.com/

    Weinberg's Second Law: If builders built buildings the way programmers wrote
    programs, then the first woodpecker that came along would destroy civilization.
    Aahz, Feb 10, 2009
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tom Brown
    Replies:
    0
    Views:
    456
    Tom Brown
    Sep 22, 2005
  2. Elad
    Replies:
    0
    Views:
    401
  3. Kevin Walzer
    Replies:
    1
    Views:
    373
    Martin P. Hellwig
    Dec 12, 2008
  4. Andrew Parker

    subprocess.Popen not creating a pipe

    Andrew Parker, Feb 1, 2009, in forum: Python
    Replies:
    0
    Views:
    246
    Andrew Parker
    Feb 1, 2009
  5. Steven K. Wong
    Replies:
    6
    Views:
    1,009
    Nobody
    Jan 10, 2010
Loading...

Share This Page