Redirecting stderr and stdout to syslog

Discussion in 'Python' started by Lincoln Yeoh, Aug 5, 2008.

  1. Lincoln Yeoh

    Lincoln Yeoh Guest

    Hi,

    I've just started to learn python (I've been using perl for some years).

    How do I redirect ALL stderr stuff to syslog, even stderr from
    external programs that don't explicitly change their own stderr?

    Say I have a program called foo:

    #!/usr/bin/python
    import syslog
    import os, sys
    class logstderr:
    def write(self, data):
    syslog.syslog('STDERR: %s' % data)
    syslog.openlog('test[%u]' % os.getpid() )
    sys.stderr=logstderr()
    cmd='ls -al asdfsdf'
    os.system(cmd)
    bar('foo')

    And bar is a nonexistent function.

    If I run test I get:
    ../test
    ls: cannot access asdfsdf: No such file or directory

    And in /var/log/messages I get:
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: Traceback (most recent
    call last):
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: File "./foo", line
    11, in <module>
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: bar('foo')
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: NameError
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: :
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: name 'bar' is not defined
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:


    What I want is the "ls: cannot access asdfsdf: No such file or
    directory" message to go to syslog instead:
    e.g.
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: ls: cannot access
    asdfsdf: No such file or directory
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: Traceback (most recent
    call last):
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: File "./foo", line
    11, in <module>
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: bar('foo')
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: NameError
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: :
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR: name 'bar' is not defined
    Aug 5 21:08:35 linux-9k3z test[2186]: STDERR:

    Explanation:

    I do not normally redirect STDERR and STDOUT to /dev/null for daemons I write.

    Since in _theory_ nothing should be "leaking" out, if stuff does leak
    out in practice, something is not quite right.

    So I want all such "leaks" be redirected to syslog (or my logging
    routines), so that I can see the bugs/warnings - whether in my
    program or other programs/modules I call/use.

    Sorry if this has been dealt with before - I haven't found the
    solution in my searches though.

    I do NOT want to resort to this:

    #!/bin/sh
    /bin/foo 2>&1 | logger -t "test: STDERR/STDOUT"

    :)

    Thanks,

    Link.
     
    Lincoln Yeoh, Aug 5, 2008
    #1
    1. Advertising

  2. In article <>,

    > How do I redirect ALL stderr stuff to syslog, even stderr from
    > external programs that don't explicitly change their own stderr?


    Sending messages to syslog involves more than writing to a file
    descriptor, so there's no way to make this happen without having some
    process read the external programs' output and send it to syslog (which
    is basically the 'logger' method that you said you didn't like).


    > Since in _theory_ nothing should be "leaking" out, if stuff does leak
    > out in practice, something is not quite right.


    > So I want all such "leaks" be redirected to syslog (or my logging
    > routines), so that I can see the bugs/warnings - whether in my
    > program or other programs/modules I call/use.


    One reasonable way to do this is to have a separate 'breakage log' file
    for this kind of message, and point standard error there. This will
    catch output from external programs, and any output to standard error
    which libraries in your main program (or the Python interpreter) decide
    to produce.

    You'd do that with something like this (untested):

    STDERR = 2
    se = os.open("breakage.log", os.O_WRONLY|os.O_APPEND)
    os.dup2(se, STDERR)
    os.close(se)

    You can also use this breakage log for errors from your own program, if
    there are any cases where you think things might be too messed up for
    you to want to rely on your normal logging routines.

    Then since the breakage log should normally be empty, you can write a
    cronjob or something to keep an eye on it and notify you if anything
    appears there.

    -M-
     
    Matthew Woodcraft, Aug 5, 2008
    #2
    1. Advertising

  3. Lincoln Yeoh

    Lincoln Yeoh Guest

    At 03:29 AM 8/6/2008, Matthew Woodcraft wrote:
    >In article <>,
    >
    > > How do I redirect ALL stderr stuff to syslog, even stderr from
    > > external programs that don't explicitly change their own stderr?

    >
    >Sending messages to syslog involves more than writing to a file
    >descriptor, so there's no way to make this happen without having some
    >process read the external programs' output and send it to syslog (which
    >is basically the 'logger' method that you said you didn't like).


    I actually don't mind resorting to logger, I just don't want to wrap
    the python program in a shell script.

    Anyway the following appears to work, not sure if there are any
    gotchas (I'm new to this).

    (a,b)=popen2.popen2('/bin/logger -t "test[%u]: STDERR"' % os.getpid())
    os.dup2(b.fileno(),sys.stderr.fileno())
    (a,b1)=popen2.popen2('/bin/logger -t "test[%u]: STDOUT"' % os.getpid())
    os.dup2(b1.fileno(),sys.stdout.fileno())

    It seems subprocess is to be preferred over popen2. subprocess is
    more verbose for this.

    With perl I normally do (close your eyes if you are allergic to perl :) ):

    open(STDERR,"|/bin/logger -t \"${PROGNAME}[$$]: STDERR\"") or die
    "Error: Unable to redirect STDERR to logger!";
    open(STDOUT,"|/bin/logger -t \"${PROGNAME}[$$]: STDOUT\"") or die
    "Error: Unable to redirect STDOUT to logger!";

    >You'd do that with something like this (untested):
    >
    > STDERR = 2
    > se = os.open("breakage.log", os.O_WRONLY|os.O_APPEND)
    > os.dup2(se, STDERR)
    > os.close(se)
    >
    >You can also use this breakage log for errors from your own program, if
    >there are any cases where you think things might be too messed up for
    >you to want to rely on your normal logging routines.


    I've seen that but rejected it because it will not provide
    timestamps, and the logs end up in different places - some to syslog
    and some to this file.

    Having the logs go to the same place also gives a hint on the order
    of events - it's not 100% in order of course, but usually good
    enough, and much better than not even knowing what hour each line occurred.

    Regards,
    Link.
     
    Lincoln Yeoh, Aug 6, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jon Landenburer

    redirecting stderr and stdout

    Jon Landenburer, May 12, 2004, in forum: Perl
    Replies:
    1
    Views:
    11,253
    Joe Smith
    May 13, 2004
  2. Replies:
    2
    Views:
    6,172
    Chris Uppal
    Jun 20, 2006
  3. leroy isaac
    Replies:
    1
    Views:
    513
    James Willmore
    Oct 29, 2004
  4. it_says_BALLS_on_your forehead
    Replies:
    2
    Views:
    284
    Joe Smith
    Jan 10, 2006
  5. Bad Dog
    Replies:
    0
    Views:
    251
    Bad Dog
    Aug 9, 2007
Loading...

Share This Page