Logging multiple lines

Discussion in 'Python' started by Nikolaus Rath, Jun 17, 2009.

  1. Hi,

    Are there any best practices for handling multi-line log messages?

    For example, the program,

    main = logging.getLogger()
    handler = logging.StreamHandler()
    handler.setFormatter(logging.Formatter('%(asctime)s %(levelname)s %(message)s'))
    main.addHandler(handler)
    main.setLevel(logging.DEBUG)
    main.info("Starting")
    try:
    bla = 42/0
    except:
    main.exception("Oops...")

    generates the log messages

    2009-06-16 22:19:57,515 INFO Starting
    2009-06-16 22:19:57,518 ERROR Oops...
    Traceback (most recent call last):
    File "/home/nikratio/lib/EclipseWorkspace/playground/src/mytests.py", line 27, in <module>
    bla = 42/0
    ZeroDivisionError: integer division or modulo by zero


    which are a mess in any logfile because they make it really difficult to
    parse.


    How do you usually handle multi-line messages? Do you avoid them
    completely (and therefore also the exception logging facilities provided
    by logging)? Or is it possible to tweak the formatter so that it inserts
    the prefix at the beginning of every line?


    Best,


    -Nikolaus

    --
    »Time flies like an arrow, fruit flies like a Banana.«

    PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
     
    Nikolaus Rath, Jun 17, 2009
    #1
    1. Advertising

  2. On Tue, 16 Jun 2009 22:22:31 -0400
    Nikolaus Rath <> wrote:

    > How do you usually handle multi-line messages? Do you avoid them
    > completely (and therefore also the exception logging facilities
    > provided by logging)? Or is it possible to tweak the formatter so
    > that it inserts the prefix at the beginning of every line?


    I'd log exception name and timestamp (or id) only, pushing the full
    message with the same id to another log or facility (like mail it to
    some dedicated bug-report box).

    --
    Mike Kazantsev // fraggod.net

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.11 (GNU/Linux)

    iEYEARECAAYFAko4YeEACgkQASbOZpzyXnFu5ACgzkD9O9eNjQFytPxTElMm525w
    eGMAn2ch6kt5ydE2p+UqlqrPD/GhMAUE
    =1Dhz
    -----END PGP SIGNATURE-----
     
    Mike Kazantsev, Jun 17, 2009
    #2
    1. Advertising

  3. Nikolaus Rath

    Lie Ryan Guest

    Nikolaus Rath wrote:
    > Hi,
    >
    > Are there any best practices for handling multi-line log messages?
    >
    > For example, the program,
    >
    > main = logging.getLogger()
    > handler = logging.StreamHandler()
    > handler.setFormatter(logging.Formatter('%(asctime)s %(levelname)s %(message)s'))
    > main.addHandler(handler)
    > main.setLevel(logging.DEBUG)
    > main.info("Starting")
    > try:
    > bla = 42/0
    > except:
    > main.exception("Oops...")
    >
    > generates the log messages
    >
    > 2009-06-16 22:19:57,515 INFO Starting
    > 2009-06-16 22:19:57,518 ERROR Oops...
    > Traceback (most recent call last):
    > File "/home/nikratio/lib/EclipseWorkspace/playground/src/mytests.py", line 27, in <module>
    > bla = 42/0
    > ZeroDivisionError: integer division or modulo by zero
    >
    >
    > which are a mess in any logfile because they make it really difficult to
    > parse.
    >
    >
    > How do you usually handle multi-line messages? Do you avoid them
    > completely (and therefore also the exception logging facilities provided
    > by logging)? Or is it possible to tweak the formatter so that it inserts
    > the prefix at the beginning of every line?
    >
    >


    The next line that starts with a timestamp is part of the next log.

    If you added timestamp on every line, that would be much more difficult
    to parse. You could do something like:

    import re

    timestamp_re = '...'
    messages = []

    for line in open('log.log'):
    if timestamp_re.match(line):
    messages.append([line])
    else:
    messages[-1].append(line)
     
    Lie Ryan, Jun 17, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Stefan Siegl
    Replies:
    0
    Views:
    960
    Stefan Siegl
    Aug 27, 2003
  2. janne
    Replies:
    0
    Views:
    9,496
    janne
    Sep 10, 2004
  3. Christoph Haas
    Replies:
    0
    Views:
    450
    Christoph Haas
    Jun 12, 2006
  4. Dave

    multiple changes on multiple lines

    Dave, Apr 2, 2005, in forum: Perl Misc
    Replies:
    1
    Views:
    109
    Gunnar Hjalmarsson
    Apr 2, 2005
  5. Cah Sableng
    Replies:
    0
    Views:
    242
    Cah Sableng
    Apr 23, 2007
Loading...

Share This Page