Suppress output to stdout/stderr in InteractiveInterpreter

Discussion in 'Python' started by Dave W., Apr 15, 2010.

  1. Dave W.

    Dave W. Guest

    I've subclassed InteractiveInterpreter in an attempt to make it
    friendlier for use in a remote console for a C++ app. What I really
    wanted to do was wrap/adapt the base class's runsource() method to
    return a 2-tuple (more, result) where 'more' is a bool indicating
    whether a complete command was recognized, and 'result' is a string
    representing the results of evaluating the last chunk (which may be
    empty, or may contain an error message).

    The only fool-proof way I found to do this was to redirect stdout/
    stderr
    during calls to the base class's runsource() method. If I only
    provide
    custom sys.displayhook and sys.excepthook replacements, then any print
    statements the user enters go to the host app's stdout:

    >>> print xvis.get_object(17).pos


    instead of being captured and echoed back to the remote console.
    If I omit the print, it does what I expect:

    >>> xvis.get_object(17).pos
    >>> (11.7, 1005.0, 0.0)


    But I'd really like to:

    1) Get rid of this slight difference to the normal python REPL
    2) Not have to redirect stdout/stderr (I'm afraid other [C++]
    threads may compete with python for these streams.)

    I thought I could get away with import print_function from __future__
    (see example code below), but my re-pointed print function never gets
    called.

    I think I'm just naive about how exec() works. Is it hard-wired to
    stdout or something? *Should* this even work?

    Any insight much appreciated!

    - Dave Wolfe

    ----------

    from __future__ import print_function
    ....
    class HookContext(object):

    def __init__(self, printhook, displayhook, excepthook):
    self.printhook = printhook
    self.displayhook = displayhook
    self.excepthook = excepthook

    def __enter__(self):
    print = self.printhook
    sys.displayhook = self.displayhook
    sys.excepthook = self.excepthook

    def __exit__(self, exc_type, exc_value, traceback):
    print = __builtins__.print
    sys.displayhook = sys.__displayhook__
    sys.excepthook = sys.__excepthook__


    class Interpreter(InteractiveInterpreter):

    [ ... lots of stuff omitted ...]

    def do_runsource(self, source):
    self.output.reset()
    self.output.truncate()

    with HookContext(self.printhook
    self.displayhook, self.excepthook):
    try:
    more = InteractiveInterpreter.runsource(self, source)
    result = self.output.getvalue()
    except (EOFError, OverflowError, SyntaxError):
    pass

    return more, result
    Dave W., Apr 15, 2010
    #1
    1. Advertising

  2. Dave W.

    Jerry Hill Guest

    On Wed, Apr 14, 2010 at 9:03 PM, Dave W. <> wrote:
    > I thought I could get away with import print_function from __future__
    > (see example code below), but my re-pointed print function never gets
    > called.


    -snip-

    >    def __enter__(self):
    >        print = self.printhook


    That redefines the print function local to __enter__. You need to
    change the global value of print.

    from __future__ import print_function
    from contextlib import contextmanager

    def printhook(obj, *args, **kwargs):
    __builtins__.print('['+obj+']', *args, **kwargs)

    @contextmanager
    def local_printhook():
    print = printhook
    yield
    print = __builtins__.print

    @contextmanager
    def global_printhook():
    global print
    print = printhook
    yield
    print = __builtins__.print


    with local_printhook():
    print("Test 1")

    with global_printhook():
    print("Test 2")

    >>> ================================ RESTART ================================
    >>>

    Test 1
    [Test 2]
    >>>
    Jerry Hill, Apr 15, 2010
    #2
    1. Advertising

  3. Dave W.

    Dave W. Guest

    > > I thought I could get away with importing print_function
    > > from __future__ ... but my re-pointed print function never gets
    > > called.


    > -snip-


    > > def __enter__(self):
    > > print = self.printhook


    > That redefines the print function local to __enter__. You need to
    > change the global value of print.


    Thanks for the clue. I had actually tried using 'global print' in
    the __enter__ function, but it broke my tests even worse so I
    figured I must not need it. :-} After your reply, I went back and
    discovered that everything was actually working fine with 'global'
    in place---except my automated tests break. So I just have a little
    mystery to solve regarding how execution using nose differs from
    running the app manually.

    > from contextlib import contextmanager


    Hey, thanks for that gem! I didn't know about this clever little
    module---really useful!
    Dave W., Apr 15, 2010
    #3
  4. Dave W.

    Dave W. Guest

    > After your reply, I went back and
    > discovered that everything was actually working fine with 'global'
    > in place <snip>


    Oops, I lied. It still doesn't work. It doesn't seem possible (or
    at least not easy) to *globally* override the built-in print
    function. The best I've managed to do is override it for the
    current module.

    Think I'll start a new post with the subject: "Globally override
    built-in print function?"
    Dave W., Apr 15, 2010
    #4
  5. Dave W.

    Robert Kern Guest

    On 2010-04-15 17:54 PM, Dave W. wrote:
    >> After your reply, I went back and
    >> discovered that everything was actually working fine with 'global'
    >> in place<snip>

    >
    > Oops, I lied. It still doesn't work. It doesn't seem possible (or
    > at least not easy) to *globally* override the built-in print
    > function. The best I've managed to do is override it for the
    > current module.
    >
    > Think I'll start a new post with the subject: "Globally override
    > built-in print function?"


    Don't bother. Replacing sys.stdout is the right thing to do. It won't interfere
    with the C++ streams but it will take care of some potential Python output that
    does not go through the print() function.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Apr 16, 2010
    #5
  6. Dave W.

    Dave W. Guest

    > > Think I'll start a new post with the subject: "Globally override
    > > built-in print function?"


    > Don't bother. Replacing sys.stdout is the right thing to do. It
    > won't interfere with the C++ streams...

    -snip-

    I'm not so certain. Won't the C++ host app share the same
    stdin/stdout/stderr file descriptors with the embedded Python
    interpreter? So there's at least the *potential* for a background
    C++ thread dedicated to processing Python commands (which redirect
    stdout) to interfere with, say, the main thread's log statements to
    stdout. Seems like I could wind up with log statements in my
    interpreter results.

    As it happens, the background thread for processing incoming
    commands from the remote REPL currently just queues those commands
    to be executed in the main thread, so there's no contention. But
    this makes some 'interesting' things possible using the remote
    console:

    c:\pt\celoverus\scripts>remote_console.py 192.168.1.10
    Python 2.6.2 [IG REPL]
    >>> from time import sleep
    >>> sleep(1000000)


    Because this is executed in the main app thread's context, it hangs
    the host app forever. Kind of a perfect DoS attack. Not that
    security is a big deal for this particular app. I'm more worried
    about not being able to perform debugging tasks like this:

    >>> while True:

    .... for obj in engine.get_objects():
    .... if obj.collides_with(engine.get_object(OWNSHIP)
    .... print("'{0}' hit the ownship".format(obj.name))
    .... time.sleep(5.0)

    Currently, this would hang the app forever, too. Moving the
    execution of remote Python commands into a low-priority background
    thread would allow the host app to keep running in this case.

    Anyway, that's why I've got this mild obsession with finding a way
    to capture output from the interpreter *without* redirecting
    stdout...
    Dave W., Apr 16, 2010
    #6
  7. Dave W.

    Robert Kern Guest

    On 2010-04-16 00:42 AM, Dave W. wrote:
    >>> Think I'll start a new post with the subject: "Globally override
    >>> built-in print function?"

    >
    >> Don't bother. Replacing sys.stdout is the right thing to do. It
    >> won't interfere with the C++ streams...

    > -snip-
    >
    > I'm not so certain. Won't the C++ host app share the same
    > stdin/stdout/stderr file descriptors with the embedded Python
    > interpreter? So there's at least the *potential* for a background
    > C++ thread dedicated to processing Python commands (which redirect
    > stdout) to interfere with, say, the main thread's log statements to
    > stdout. Seems like I could wind up with log statements in my
    > interpreter results.


    No. Changing the object that the name sys.stdout refers to does not swap out the
    underlying file descriptors.

    > Anyway, that's why I've got this mild obsession with finding a way
    > to capture output from the interpreter *without* redirecting
    > stdout...


    Not possible. Many things write directly to sys.stdout/sys.stderr.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Apr 16, 2010
    #7
  8. Dave W.

    Dave W. Guest

    >>> Don't bother. Replacing sys.stdout is the right thing to do. It
    >>> won't interfere with the C++ streams...


    >> I'm not so certain. Won't the C++ host app share the same
    >> stdin/stdout/stderr file descriptors with the embedded Python
    >> interpreter?


    > No. Changing the object that the name sys.stdout refers to does
    > not swap out the underlying file descriptors.


    Whoa--that's a bombshell. I guess I shouldn't have assumed that
    this was the case, but it seemed so 'obvious'. Glad I was wrong,
    since it makes like much, much easier. Thank you!

    [xref "Globally override built-in print function?"]
    Dave W., Apr 16, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    2
    Views:
    558
    Jim Sizelove
    Dec 3, 2004
  2. MKoool
    Replies:
    2
    Views:
    312
    MKoool
    May 30, 2005
  3. Replies:
    5
    Views:
    704
    Kenneth Brody
    Nov 6, 2005
  4. Replies:
    37
    Views:
    2,053
    Flash Gordon
    Jan 9, 2006
  5. Colin Howell
    Replies:
    2
    Views:
    569
    John Bokma
    Apr 25, 2010
Loading...

Share This Page