os.system and loggers

Discussion in 'Python' started by Tim, Jan 7, 2011.

  1. Tim

    Tim Guest

    hi, I'm using a 3rd-party python program that uses the python logging
    facility and also makes calls to os.system. I'm trying to capture its
    output to a file.

    In my own code, I've taken control of the loggers that are setup in
    the other program by removing its StreamHandler and replacing with
    FileHander. But when it comes to the call to os.system I'm at a loss.

    I want to capture the stdout from that os.system call in my
    FileHandler. I thought this might work, before I call the other
    program's class/method:
    sys.stdout = getLogger('status').handlers[0].stream

    but no dice. Is there any clean way to get what I want? If not, I
    guess I'll override the other method with my own, but it will
    basically be a bunch of code copied with os.sytem replaced with
    subprocess, using getLogger('status').handlers[0].stream for stdout/
    stderr.

    thanks,
    --Tim Arnold
    Tim, Jan 7, 2011
    #1
    1. Advertising

  2. Tim

    Tim Guest

    On Jan 7, 11:24 am, Tim <> wrote:
    > hi, I'm using a 3rd-party python program that uses the python logging
    > facility and also makes calls to os.system. I'm trying to capture its
    > output to a file.
    >
    > In my own code, I've taken control of the loggers that are setup in
    > the other program by removing its StreamHandler and replacing with
    > FileHander. But when it comes to the call to os.system I'm at a loss.
    >
    > I want to capture the stdout from that os.system call in my
    > FileHandler. I thought this might work, before I call the other
    > program's class/method:
    > sys.stdout = getLogger('status').handlers[0].stream
    >
    > but no dice. Is there any clean way to get what I want? If not, I
    > guess I'll override the other method with my own, but it will
    > basically be a bunch of code copied with os.sytem replaced with
    > subprocess, using getLogger('status').handlers[0].stream for stdout/
    > stderr.
    >
    > thanks,
    > --Tim Arnold


    Replying to my own post....

    I think I may have included too much fluff in my original question.
    The main thing I wonder is whether I can attach a log handler to
    stdout in such a way that os.system calls will write to that handler
    instead of the console.

    thanks,
    --Tim
    Tim, Jan 10, 2011
    #2
    1. Advertising

  3. Tim

    Carl Banks Guest

    On Jan 10, 8:29 am, Tim <> wrote:
    > On Jan 7, 11:24 am, Tim <> wrote:
    >
    >
    >
    >
    >
    > > hi, I'm using a 3rd-party python program that uses the python logging
    > > facility and also makes calls to os.system. I'm trying to capture its
    > > output to a file.

    >
    > > In my own code, I've taken control of the loggers that are setup in
    > > the other program by removing its StreamHandler and replacing with
    > > FileHander. But when it comes to the call to os.system I'm at a loss.

    >
    > > I want to capture the stdout from that os.system call in my
    > > FileHandler. I thought this might work, before I call the other
    > > program's class/method:
    > > sys.stdout = getLogger('status').handlers[0].stream

    >
    > > but no dice. Is there any clean way to get what I want? If not, I
    > > guess I'll override the other method with my own, but it will
    > > basically be a bunch of code copied with os.sytem replaced with
    > > subprocess, using getLogger('status').handlers[0].stream for stdout/
    > > stderr.

    >
    > > thanks,
    > > --Tim Arnold

    >
    > Replying to my own post....
    >
    > I think I may have included too much fluff in my original question.
    > The main thing I wonder is whether I can attach a log handler to
    > stdout in such a way that os.system calls will write to that handler
    > instead of the console.


    No, but you could replace os.system with something that does work.
    (It would replace it globally for all uses, so you may need some logic
    to decide whether to leave the call alone, or to modify it, perhaps by
    inspecting the call stack.)

    The simplest thing to do is to append a shell redirection to the
    command (>/your/log/file), so something like this:

    _real_os_system = os.system

    def my_os_system(cmd):
    if test_log_condition:
    return _real_os_system(cmd + "2> /my/log/file")
    return _real_os_system(cmd)

    os.system = my_os_system

    That could backfire for any number of reasons so you probably should
    only do this if you know that it works with all the commands it
    issues.

    The better way might be to call the subprocess module instead, where
    you can dispatch the command with redirection to any stream. I doubt
    there's a foolproof way to do that given an arbitrary os.system
    command, but the subprocess way is probably safer.


    Carl Banks
    Carl Banks, Jan 10, 2011
    #3
  4. Tim

    Carl Banks Guest

    On Jan 10, 8:29 am, Tim <> wrote:
    > I think I may have included too much fluff in my original question.
    > The main thing I wonder is whether I can attach a log handler to
    > stdout in such a way that os.system calls will write to that handler
    > instead of the console.
    Carl Banks, Jan 10, 2011
    #4
  5. Tim

    Tim Guest

    On Jan 10, 1:01 pm, Carl Banks <> wrote:
    > On Jan 10, 8:29 am, Tim <> wrote:
    >
    >
    >
    >
    >
    >
    >
    >
    >
    > > On Jan 7, 11:24 am, Tim <> wrote:

    >
    > > > hi, I'm using a 3rd-party python program that uses the python logging
    > > > facility and also makes calls to os.system. I'm trying to capture its
    > > > output to a file.

    >
    > > > In my own code, I've taken control of the loggers that are setup in
    > > > the other program by removing its StreamHandler and replacing with
    > > > FileHander. But when it comes to the call to os.system I'm at a loss.

    >
    > > > I want to capture the stdout from that os.system call in my
    > > > FileHandler. I thought this might work, before I call the other
    > > > program's class/method:
    > > > sys.stdout = getLogger('status').handlers[0].stream

    >
    > > > but no dice. Is there any clean way to get what I want? If not, I
    > > > guess I'll override the other method with my own, but it will
    > > > basically be a bunch of code copied with os.sytem replaced with
    > > > subprocess, using getLogger('status').handlers[0].stream for stdout/
    > > > stderr.

    >
    > > > thanks,
    > > > --Tim Arnold

    >
    > > Replying to my own post....

    >
    > > I think I may have included too much fluff in my original question.
    > > The main thing I wonder is whether I can attach a log handler to
    > > stdout in such a way that os.system calls will write to that handler
    > > instead of the console.

    >
    > No, but you could replace os.system with something that does work.
    > (It would replace it globally for all uses, so you may need some logic
    > to decide whether to leave the call alone, or to modify it, perhaps by
    > inspecting the call stack.)
    >
    > The simplest thing to do is to append a shell redirection to the
    > command (>/your/log/file), so something like this:
    >
    > _real_os_system = os.system
    >
    > def my_os_system(cmd):
    >     if test_log_condition:
    >         return _real_os_system(cmd + "2> /my/log/file")
    >     return _real_os_system(cmd)
    >
    > os.system = my_os_system
    >
    > That could backfire for any number of reasons so you probably should
    > only do this if you know that it works with all the commands it
    > issues.
    >
    > The better way might be to call the subprocess module instead, where
    > you can dispatch the command with redirection to any stream.  I doubt
    > there's a foolproof way to do that given an arbitrary os.system
    > command, but the subprocess way is probably safer.
    >
    > Carl Banks


    Thanks Carl. I will use subprocess. I made this little toy example to
    prove to myself that subprocess does handle a filehandler stream, so I
    include it here for completeness' sake:

    import subprocess,logging,shlex

    # create the logger, filehandler and get the stream
    log = logging.getLogger('mytest')
    fh = logging.FileHandler('mytest.log')
    log.addHandler(fh)
    log.setLevel(logging.INFO)
    mylog = logging.getLogger('mytest').handlers[0].stream

    # write a test line to the log
    log.info('my first line')

    # execute the subprocess using the stream as stdout
    cmd = 'ls -l'
    p = subprocess.Popen(shlex.split(cmd),stdout=mylog)

    # if you don't wait(), the output won't be necessarily in
    chronological order.
    r = p.wait()

    # write a last test line to the log
    log.info('done %s'% r)

    and it works as expected.
    thanks,
    --Tim Arnold
    Tim, Jan 11, 2011
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Simo-Pekka Sadeluoto

    loggers with different handler properties

    Simo-Pekka Sadeluoto, Jul 8, 2003, in forum: Java
    Replies:
    0
    Views:
    524
    Simo-Pekka Sadeluoto
    Jul 8, 2003
  2. Francois Bouffard

    Loggers and reloads

    Francois Bouffard, May 26, 2004, in forum: Python
    Replies:
    2
    Views:
    291
    =?ISO-8859-1?Q?Fran=E7ois_Bouffard?=
    May 27, 2004
  3. nico
    Replies:
    3
    Views:
    438
    Arne Vajhøj
    Jan 19, 2008
  4. Rhino
    Replies:
    8
    Views:
    499
    markspace
    Apr 30, 2010
  5. Tomer
    Replies:
    14
    Views:
    271
    Arne Vajhøj
    Jan 5, 2013
Loading...

Share This Page