Python Interactive Shell - outputting to stdout?

Discussion in 'Python' started by Avi Berkovich, Dec 23, 2004.

  1. Hello,

    I was unable to use popen2.popen4 to grab python.exe's (2.3) output, for
    starts, it doesn't show the version information at the beginning and
    won't return anything when writing to the stdin pipe, it seems that if I
    give it some error nous expression, the pipe would return the exception
    data, though nothing else comes through.

    A friend of mine also tried this using win32api on delphi, and got the
    same result.

    I also tried "python.exe > data.txt" and was amazed to find out that it
    won't redirect the output to the file.

    So, may someone please shed some light over this?

    Avi.
    Avi Berkovich, Dec 23, 2004
    #1
    1. Advertising

  2. Avi Berkovich

    Steve Holden Guest

    Avi Berkovich wrote:

    > Hello,
    >
    > I was unable to use popen2.popen4 to grab python.exe's (2.3) output, for
    > starts, it doesn't show the version information at the beginning and
    > won't return anything when writing to the stdin pipe, it seems that if I
    > give it some error nous expression, the pipe would return the exception
    > data, though nothing else comes through.
    >
    > A friend of mine also tried this using win32api on delphi, and got the
    > same result.
    >
    > I also tried "python.exe > data.txt" and was amazed to find out that it
    > won't redirect the output to the file.
    >
    > So, may someone please shed some light over this?
    >
    > Avi.


    Well, your first misunderstanding appears to be the omission of the
    standard error file. Interactive messages and prompts are written to
    stderr, output to stdout:

    C:\Steve>python.exe > python.txt
    Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> 123
    >>> 345
    >>> ^Z



    C:\Steve>type python.txt
    123
    345

    When you try to write a program that communicates with a client that's
    intended for interactive use (such as the Python interpreter when called
    with no arguments), you have the problem of divining exactly when the
    client is waiting for input.

    Rather than blame the Python interpreter, first write a program that
    interacts with (say) the Windows command-line interpreter.

    If your experience is substantially different then it *might* be a
    problem with the interpreter. If your experience is more or less the
    same then the problem is probably to do with the nature of the
    interactions, which are bad enough when only considering stdin and
    stdout. When you add the stderr stream into the picture it's sometimes
    impossible to know when the interactive client is waiting for input.

    One final comment: you haven't so far said *why* you want to interact
    with Python in this way. Given the availability of exec and eval(), just
    what is it that you are trying to do that they won't let you?

    regards
    Steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
    Steve Holden, Dec 23, 2004
    #2
    1. Advertising

  3. Hey Steve,

    I did write a program to deal with the windows command interpreter, and
    it works.

    I don't need to do this, but a friend of mine needed to issue commands
    to the interpreter via pipes from a non python program, and he has a
    fully functional component for using pipes, and has done it with other
    interpreters before.

    I guess I'll have to separate the std's and continue from there.

    Thanks for your input.

    Avi.
    Avi Berkovich, Dec 23, 2004
    #3
  4. Hey,

    I can't make it work, I don't get any data from either stdout nor stderr.
    If I send lines and then close the stdin pipe, I may get an exception
    message from several lines up.

    I tried manually reading from the stdout pipe, but it just blocks and
    hangs no matter what I send over via the stdin pipe.

    This behavior isn't presented by the command line interpreter by any chance.

    Any suggestions?
    Avi Berkovich, Dec 24, 2004
    #4
  5. Avi Berkovich

    Steve Holden Guest

    Avi Berkovich wrote:

    > Hey,
    >
    > I can't make it work, I don't get any data from either stdout nor stderr.
    > If I send lines and then close the stdin pipe, I may get an exception
    > message from several lines up.
    >
    > I tried manually reading from the stdout pipe, but it just blocks and
    > hangs no matter what I send over via the stdin pipe.
    >
    > This behavior isn't presented by the command line interpreter by any
    > chance.
    >
    > Any suggestions?


    Yes: post your code along with messages (if any)!

    regards
    Steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
    Steve Holden, Dec 27, 2004
    #5
  6. Avi Berkovich

    Steve Holden Guest

    Steve Holden wrote:

    > Avi Berkovich wrote:
    >
    >> Hey,
    >>
    >> I can't make it work, I don't get any data from either stdout nor stderr.
    >> If I send lines and then close the stdin pipe, I may get an exception
    >> message from several lines up.
    >>
    >> I tried manually reading from the stdout pipe, but it just blocks and
    >> hangs no matter what I send over via the stdin pipe.
    >>
    >> This behavior isn't presented by the command line interpreter by any
    >> chance.
    >>
    >> Any suggestions?

    >
    >
    > Yes: post your code along with messages (if any)!
    >
    > regards
    > Steve


    Sorry, for some reason I can no longer find the message you posted the
    code in -- it didn't get correctly threaded in Mozilla. Anyway, I can't
    say that I exactly understand what's going on, but here are a couple of
    observations:

    1. The interpreter performs buffering even when running in interactive
    mode unless it can see a terminal (which here it can't). Hence adding a
    "-u" to the interpreter command line is useful.

    2. There's a problem with your stop condition, which is why the attached
    modified version sleeps before closing the pipe to the interpreter.

    import threading
    import sys
    import popen2

    class Piper(threading.Thread):

    def __init__(self, readPipe, output):
    threading.Thread.__init__(self)

    if not isinstance(readPipe, file):
    raise TypeError, "readPipe parameter must be of File type"
    #if not isinstance(output, file):
    # raise TypeError, "output parameter must be of File type"

    self.readPipe = readPipe
    self.output = output
    self.toStop = False

    def run(self):
    print "Running"
    while not self.toStop and not self.readPipe.closed:
    read = self.readPipe.readline()
    self.output.write(read)
    self.output.flush()
    print "Stopped"

    def stop(self):
    self.toStop = True


    if __name__ == "__main__":
    r, w = popen2.popen4('c:\\python24\\python.exe -u')
    piper = Piper(r, sys.stdout)
    piper.start()
    w.write("print 'Hello!'\r\n")
    w.flush()
    w.write("print 'Goodbye!'\r\n")
    w.flush()
    import time; time.sleep(2)
    w.close()
    #import time; time.sleep(3)
    #r.close()
    piper.stop()

    Various other bits and pieces of messing about didn't really yield any
    useful conclusions, so I pass this on only in the hope that you may be
    more motivated to follow up now you can actually see some interpreter
    output. I would have thought the flush() calls would have made output
    appear one line at a time, but sadly they do not.

    regards
    Steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
    Steve Holden, Dec 27, 2004
    #6
  7. Hey Steve,

    Well, I've tried flush() before, but I didn't know about the "-u" switch.

    Thank you for tinkering on this, I shall post again if I make any progress.

    Avi


    Steve Holden wrote:
    > Steve Holden wrote:
    >
    >> Avi Berkovich wrote:
    >>
    >>> Hey,
    >>>
    >>> I can't make it work, I don't get any data from either stdout nor
    >>> stderr.
    >>> If I send lines and then close the stdin pipe, I may get an exception
    >>> message from several lines up.
    >>>
    >>> I tried manually reading from the stdout pipe, but it just blocks and
    >>> hangs no matter what I send over via the stdin pipe.
    >>>
    >>> This behavior isn't presented by the command line interpreter by any
    >>> chance.
    >>>
    >>> Any suggestions?

    >>
    >>
    >>
    >> Yes: post your code along with messages (if any)!
    >>
    >> regards
    >> Steve

    >
    >
    > Sorry, for some reason I can no longer find the message you posted the
    > code in -- it didn't get correctly threaded in Mozilla. Anyway, I can't
    > say that I exactly understand what's going on, but here are a couple of
    > observations:
    >
    > 1. The interpreter performs buffering even when running in interactive
    > mode unless it can see a terminal (which here it can't). Hence adding a
    > "-u" to the interpreter command line is useful.
    >
    > 2. There's a problem with your stop condition, which is why the attached
    > modified version sleeps before closing the pipe to the interpreter.
    >
    > import threading
    > import sys
    > import popen2
    >
    > class Piper(threading.Thread):
    >
    > def __init__(self, readPipe, output):
    > threading.Thread.__init__(self)
    >
    > if not isinstance(readPipe, file):
    > raise TypeError, "readPipe parameter must be of File type"
    > #if not isinstance(output, file):
    > # raise TypeError, "output parameter must be of File type"
    >
    > self.readPipe = readPipe
    > self.output = output
    > self.toStop = False
    >
    > def run(self):
    > print "Running"
    > while not self.toStop and not self.readPipe.closed:
    > read = self.readPipe.readline()
    > self.output.write(read)
    > self.output.flush()
    > print "Stopped"
    >
    > def stop(self):
    > self.toStop = True
    >
    >
    > if __name__ == "__main__":
    > r, w = popen2.popen4('c:\\python24\\python.exe -u')
    > piper = Piper(r, sys.stdout)
    > piper.start()
    > w.write("print 'Hello!'\r\n")
    > w.flush()
    > w.write("print 'Goodbye!'\r\n")
    > w.flush()
    > import time; time.sleep(2)
    > w.close()
    > #import time; time.sleep(3)
    > #r.close()
    > piper.stop()
    >
    > Various other bits and pieces of messing about didn't really yield any
    > useful conclusions, so I pass this on only in the hope that you may be
    > more motivated to follow up now you can actually see some interpreter
    > output. I would have thought the flush() calls would have made output
    > appear one line at a time, but sadly they do not.
    >
    > regards
    > Steve
    Avi Berkovich, Dec 27, 2004
    #7
  8. Avi Berkovich wrote:
    > Hey,
    >
    > I can't make it work, I don't get any data from either stdout nor stderr.
    > If I send lines and then close the stdin pipe, I may get an exception
    > message from several lines up.
    >
    > I tried manually reading from the stdout pipe, but it just blocks and
    > hangs no matter what I send over via the stdin pipe.
    >
    > This behavior isn't presented by the command line interpreter by any
    > chance.
    >
    > Any suggestions?


    Well,

    I didn't manage to get the pipes working, and finally decided to embed
    the interpreter into the program, though another option is to embed it
    into a simple console program consisting of the interpreter
    initialization and input reading, and just pipe into that.

    Code for simple interpreter embedded program:

    #include <stdio.h>
    #include <python.h>

    int main()
    {
    char execString[128];

    Py_Initialize();

    while (1)
    {
    gets(execString);
    if (!strcmp(execString, "QUIT PROGRAM"))
    break;
    PyRun_SimpleString(execString);
    }

    Py_Finalize();
    }


    This program works well with pipes, though in order to issue the command
    I have to write "\n\r\n" (not "\r\n" or "\n" ?!) to the pipe.

    Avi.
    Avi Berkovich, Dec 28, 2004
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. A. L.
    Replies:
    5
    Views:
    65,871
    bruce
    Sep 16, 2005
  2. Replies:
    2
    Views:
    894
    Micah Elliott
    Nov 16, 2005
  3. raashid bhatt
    Replies:
    4
    Views:
    302
  4. Replies:
    7
    Views:
    2,577
    Stef Mientki
    Oct 1, 2008
  5. Chris Rebert
    Replies:
    1
    Views:
    1,133
    Ricardo Abreu
    Sep 30, 2013
Loading...

Share This Page