Capturing stdout without waiting for the process end

Discussion in 'Python' started by Luigi, Apr 3, 2006.

  1. Luigi

    Luigi Guest

    Hi to all!

    I'd like to execute an external program capturing the stdout/stderr
    messages at "real-time". I mean that I don't want to wait for the end
    of the process. If I write a code like this:

    import os
    import sys

    class Runner:

    def run(self, arg):
    try:
    fin, fout = os.popen4(arg)
    self.outData = fout.readlines()
    self.outStatus = fout.close()

    except Exception, err:
    print err

    def printOut(self):
    print "###"
    print self.outStatus
    print "###"
    for line in self.outData:
    print line


    r = Runner()
    r.run("ls /tmp")
    r.printOut()

    I can print out (in this case in the os.stdout, but it may be
    elsewhere) the whole external program output only once it ends. How can
    I do to intercept the external program output during the processing?

    Thank you in advance

    Luigi
    Luigi, Apr 3, 2006
    #1
    1. Advertising

  2. Luigi

    Guest

    What OS are you doing this on?

    I had an issue similar to this and it was due to default buffering
    behavior of my tty's. If I recall correctly I executed a bunch of
    settty to eliminate the buffering behavior of stdout. The set the
    terminal back to it's original setting when my program was done.

    Anand
    , Apr 3, 2006
    #2
    1. Advertising

  3. Luigi

    Luigi Guest

    I use Linux.

    So, if I modify my sdtout behaviour, when I execute:

    fin, fout = os.popen4(arg)

    this is executed asyncronously? And how can I intercept a fout.write()
    event?

    The question is that I have a C program (by third part) that streams
    the logs into the stderr and stdout devices. I need to create an
    envelopment that captures the outputs and puts them in a file
    generating log events (for a real-time view).

    Thanks

    Luigi
    Luigi, Apr 3, 2006
    #3
  4. > fin, fout = os.popen4(arg)
    >
    > this is executed asyncronously? And how can I intercept a fout.write()
    > event?


    You don't intercept anything, you read from the child process stdin, which
    from your POV is a readable stream.

    Diez
    Diez B. Roggisch, Apr 3, 2006
    #4
  5. Luigi

    Guest

    Diez is correct, the C program is writting to stdout, you are reading
    from stdout. Default bahavior in most modern Unix like systems is to
    buffer std out. I stumbled on this a long time ago, so I am trying to
    remember the details.

    What I think is happening here, you call the child process, it does
    it's thing and puts thing into stdout.

    How long does the child process run typically?

    What also is the actual behavior? How is it wrong, exactly? Are you
    getting the full output from the C program? Part of it? I still
    suspect a buffering problem of some sort, but I am not clear on what is
    going wrong.
    , Apr 3, 2006
    #5
  6. Luigi

    Donn Cave Guest

    In article <>,
    "Luigi" <> wrote:

    > The question is that I have a C program (by third part) that streams
    > the logs into the stderr and stdout devices. I need to create an
    > envelopment that captures the outputs and puts them in a file
    > generating log events (for a real-time view).


    As suggested in another followup, the C program's
    output will probably be "block buffered" when its
    output is a pipe. In this case, you'll get output
    only when the buffer is full, or when the program
    exits.

    If the program can be modified, it only needs to
    flush stdout after each output event. Otherwise,
    you need to use a device that looks like a tty,
    just so the C stdio library will switch to line
    buffering as it generally does with terminals.

    This is called a pseudotty. It's a little more
    difficult to use than a pipe, but you can probably
    get something going with openpty or forkpty from
    the os/posix module, or there may still be 3rd
    party packages for this.

    Donn Cave,
    Donn Cave, Apr 3, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Moosebumps

    Capturing stdout incrementally

    Moosebumps, Apr 3, 2004, in forum: Python
    Replies:
    5
    Views:
    501
    David Bolen
    Apr 7, 2004
  2. Rex Gustavus Adolphus
    Replies:
    5
    Views:
    136
    Rex Gustavus Adolphus
    Jul 4, 2003
  3. Steffen Knobloch

    kick external script without waiting for it to end

    Steffen Knobloch, Jan 11, 2005, in forum: Perl Misc
    Replies:
    1
    Views:
    91
    Sherm Pendley
    Jan 11, 2005
  4. Cod
    Replies:
    6
    Views:
    128
  5. PerlFAQ Server
    Replies:
    0
    Views:
    168
    PerlFAQ Server
    Apr 23, 2011
Loading...

Share This Page