os.system and loggers

[Tim]

hi, I'm using a 3rd-party python program that uses the python logging
facility and also makes calls to os.system. I'm trying to capture its
output to a file.

In my own code, I've taken control of the loggers that are setup in
the other program by removing its StreamHandler and replacing with
FileHander. But when it comes to the call to os.system I'm at a loss.

I want to capture the stdout from that os.system call in my
FileHandler. I thought this might work, before I call the other
program's class/method:
sys.stdout = getLogger('status').handlers[0].stream

but no dice. Is there any clean way to get what I want? If not, I
guess I'll override the other method with my own, but it will
basically be a bunch of code copied with os.sytem replaced with
subprocess, using getLogger('status').handlers[0].stream for stdout/
stderr.

thanks,
--Tim Arnold

 

[Tim]

hi, I'm using a 3rd-party python program that uses the python logging
facility and also makes calls to os.system. I'm trying to capture its
output to a file.

In my own code, I've taken control of the loggers that are setup in
the other program by removing its StreamHandler and replacing with
FileHander. But when it comes to the call to os.system I'm at a loss.

I want to capture the stdout from that os.system call in my
FileHandler. I thought this might work, before I call the other
program's class/method:
sys.stdout = getLogger('status').handlers[0].stream

but no dice. Is there any clean way to get what I want? If not, I
guess I'll override the other method with my own, but it will
basically be a bunch of code copied with os.sytem replaced with
subprocess, using getLogger('status').handlers[0].stream for stdout/
stderr.

thanks,
--Tim Arnold

Replying to my own post....

I think I may have included too much fluff in my original question.
The main thing I wonder is whether I can attach a log handler to
stdout in such a way that os.system calls will write to that handler
instead of the console.

thanks,
--Tim

 

[Carl Banks]

hi, I'm using a 3rd-party python program that uses the python logging
facility and also makes calls to os.system. I'm trying to capture its
output to a file.

In my own code, I've taken control of the loggers that are setup in
the other program by removing its StreamHandler and replacing with
FileHander. But when it comes to the call to os.system I'm at a loss.

I want to capture the stdout from that os.system call in my
FileHandler. I thought this might work, before I call the other
program's class/method:
sys.stdout = getLogger('status').handlers[0].stream

but no dice. Is there any clean way to get what I want? If not, I
guess I'll override the other method with my own, but it will
basically be a bunch of code copied with os.sytem replaced with
subprocess, using getLogger('status').handlers[0].stream for stdout/
stderr.

thanks,
--Tim Arnold

Replying to my own post....

I think I may have included too much fluff in my original question.
The main thing I wonder is whether I can attach a log handler to
stdout in such a way that os.system calls will write to that handler
instead of the console.

No, but you could replace os.system with something that does work.
(It would replace it globally for all uses, so you may need some logic
to decide whether to leave the call alone, or to modify it, perhaps by
inspecting the call stack.)

The simplest thing to do is to append a shell redirection to the
command (>/your/log/file), so something like this:

_real_os_system = os.system

def my_os_system(cmd):
if test_log_condition:
return _real_os_system(cmd + "2> /my/log/file")
return _real_os_system(cmd)

os.system = my_os_system

That could backfire for any number of reasons so you probably should
only do this if you know that it works with all the commands it
issues.

The better way might be to call the subprocess module instead, where
you can dispatch the command with redirection to any stream. I doubt
there's a foolproof way to do that given an arbitrary os.system
command, but the subprocess way is probably safer.


Carl Banks

 

[Carl Banks]

I think I may have included too much fluff in my original question.
The main thing I wonder is whether I can attach a log handler to
stdout in such a way that os.system calls will write to that handler
instead of the console.

 

[Tim]

hi, I'm using a 3rd-party python program that uses the python logging
facility and also makes calls to os.system. I'm trying to capture its
output to a file.
In my own code, I've taken control of the loggers that are setup in
the other program by removing its StreamHandler and replacing with
FileHander. But when it comes to the call to os.system I'm at a loss.
I want to capture the stdout from that os.system call in my
FileHandler. I thought this might work, before I call the other
program's class/method:
sys.stdout = getLogger('status').handlers[0].stream
but no dice. Is there any clean way to get what I want? If not, I
guess I'll override the other method with my own, but it will
basically be a bunch of code copied with os.sytem replaced with
subprocess, using getLogger('status').handlers[0].stream for stdout/
stderr.
thanks,
--Tim Arnold

Replying to my own post....

I think I may have included too much fluff in my original question.
The main thing I wonder is whether I can attach a log handler to
stdout in such a way that os.system calls will write to that handler
instead of the console.

No, but you could replace os.system with something that does work.
(It would replace it globally for all uses, so you may need some logic
to decide whether to leave the call alone, or to modify it, perhaps by
inspecting the call stack.)

The simplest thing to do is to append a shell redirection to the
command (>/your/log/file), so something like this:

_real_os_system = os.system

def my_os_system(cmd):
    if test_log_condition:
        return _real_os_system(cmd + "2> /my/log/file")
    return _real_os_system(cmd)

os.system = my_os_system

That could backfire for any number of reasons so you probably should
only do this if you know that it works with all the commands it
issues.

The better way might be to call the subprocess module instead, where
you can dispatch the command with redirection to any stream.  I doubt
there's a foolproof way to do that given an arbitrary os.system
command, but the subprocess way is probably safer.

Carl Banks

Thanks Carl. I will use subprocess. I made this little toy example to
prove to myself that subprocess does handle a filehandler stream, so I
include it here for completeness' sake:

import subprocess,logging,shlex

# create the logger, filehandler and get the stream
log = logging.getLogger('mytest')
fh = logging.FileHandler('mytest.log')
log.addHandler(fh)
log.setLevel(logging.INFO)
mylog = logging.getLogger('mytest').handlers[0].stream

# write a test line to the log
log.info('my first line')

# execute the subprocess using the stream as stdout
cmd = 'ls -l'
p = subprocess.Popen(shlex.split(cmd),stdout=mylog)

# if you don't wait(), the output won't be necessarily in
chronological order.
r = p.wait()

# write a last test line to the log
log.info('done %s'% r)

and it works as expected.
thanks,
--Tim Arnold