Choosing log file destination in logging configuration file

Discussion in 'Python' started by guybenron@gmail.com, Apr 22, 2008.

  1. Guest

    Hey,

    I have a sort of petty-neurotic question, I'm kinda pedantic and like
    to keep my log directories clean, and this thing is bothering me to
    the point of actually posting a question (couldn't find any post about
    this..).

    My application has two types of components: daemons and regular ones
    that are run every on crontab. Because the log structure is very
    similar, I want both to use the same logging configuration file.

    For clarity, I want all of the files to have the date as part of the
    filename.
    The TimedRotatingFileHandler has it built in (once it rotates), and I
    found a dirty hack to make it work for the FileHandler (see shameful
    code below). Basically I'm all set - have the daemons use the rotating
    handler, since they're, um, daemonic and on all the time, and have the
    regular components use the regular handler, using the dirty hack to
    insert the date into the filename.

    So far so good. In the relevant applications, the code looks something
    like this:
    logging.config.fileConfig('log.ini')
    logger = logging.getLogger('log.regular') <or> logger =
    logging.getLogger('log.daemonic')
    ... and start logging.

    The thorn in my side is that after the fileConfig call, BOTH handlers
    are instantiated, meaning both types of files are created for every
    component, even if I don't need it: component.log (for the rotating
    handler) and compenent.date.log (for the regular file handler).

    ...So finally, here's my question:
    Apart from splitting the logging configuration into two separate
    files, is there any way to NOT create the file until you actually use
    it?

    Here's the logging configuration file for reference.. Thanks !

    [loggers]
    keys=root,regular,daemonic

    [handlers]
    keys=fileHandler,consoleHandler,timedRotatingFileHandler

    [formatters]
    keys=simpleFormatter

    [logger_root]
    level=DEBUG
    handlers=consoleHandler

    [logger_regular]
    level=INFO
    handlers=fileHandler,consoleHandler
    propagate=0
    qualname=log.regular

    [logger_daemonic]
    level=INFO
    handlers=timedRotatingFileHandler,consoleHandler
    propagate=0
    qualname=log.daemonic

    [handler_timedRotatingFileHandler]
    class=handlers.TimedRotatingFileHandler
    level=INFO
    formatter=simpleFormatter
    args=('mylog.log', 'midnight')

    [handler_fileHandler]
    class=FileHandler
    level=INFO
    formatter=simpleFormatter
    ; sorry about the grossness
    args=('mylog.' + str(time.localtime().tm_year) + '_' +
    str(time.localtime().tm_mon).zfill(2) + '_' +
    str(time.localtime().tm_mday).zfill(2) + '.log', 'a')

    [handler_consoleHandler]
    class=StreamHandler
    level=DEBUG
    formatter=simpleFormatter
    args=(sys.stdout,)

    [formatter_simpleFormatter]
    format=%(asctime)s - %(filename)s - %(name)s - %(levelname)s - %
    (message)s
     
    , Apr 22, 2008
    #1
    1. Advertising

  2. Miki Guest

    Hello,

    > So far so good. In the relevant applications, the code looks something
    > like this:
    > logging.config.fileConfig('log.ini')
    > logger = logging.getLogger('log.regular') <or> logger =
    > logging.getLogger('log.daemonic')
    > .. and start logging.
    >
    > The thorn in my side is that after the fileConfig call, BOTH handlers
    > are instantiated, meaning both types of files are created for every
    > component, even if I don't need it: component.log (for the rotating
    > handler) and compenent.date.log (for the regular file handler).
    >
    > ..So finally, here's my question:
    > Apart from splitting the logging configuration into two separate
    > files, is there any way to NOT create the file until you actually use
    > it?

    You can generate the .ini file on the fly and then load it:
    >>> log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
    >>> atexit.register(lambda: remove(log_ini))
    >>> logging.config.fileConfig(log_ini)


    HTH,
    --
    Miki <>
    http://pythonwise.blogspot.com
     
    Miki, Apr 22, 2008
    #2
    1. Advertising

  3. Guest

    On Apr 22, 12:57 pm, Miki <> wrote:
    > Hello,
    >
    >
    >
    > > So far so good. In the relevant applications, the code looks something
    > > like this:
    > > logging.config.fileConfig('log.ini')
    > > logger = logging.getLogger('log.regular') <or> logger =
    > > logging.getLogger('log.daemonic')
    > > .. and start logging.

    >
    > > The thorn in my side is that after the fileConfig call, BOTH handlers
    > > are instantiated, meaning both types of files are created for every
    > > component, even if I don't need it: component.log (for the rotating
    > > handler) and compenent.date.log (for the regular file handler).

    >
    > > ..So finally, here's my question:
    > > Apart from splitting the logging configuration into two separate
    > > files, is there any way to NOT create the file until you actually use
    > > it?

    >
    > You can generate the .ini file on the fly and then load it:
    >
    > >>> log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
    > >>> atexit.register(lambda: remove(log_ini))
    > >>> logging.config.fileConfig(log_ini)

    >
    > HTH,
    > --
    > Miki <>http://pythonwise.blogspot.com



    I think it misses the point of having a file to config..
    It looks as if there's no solution. A peek at the logging module shows
    this:
    klass = cp.get(sectname, "class")
    ...
    klass = eval(klass, vars(logging))
    args = cp.get(sectname, "args")
    args = eval(args, vars(logging))
    h = apply(klass, args)

    Bah. I'll have to split them into two configuration files (or subclass
    and have the respective file and rotating handlers lazy evaluate until
    the actual log call is made).
    Thanks though.
     
    , Apr 22, 2008
    #3
  4. Vinay Sajip Guest

    On Apr 22, 9:48 pm, wrote:
    > On Apr 22, 12:57 pm, Miki <> wrote:
    >
    >
    >
    > > Hello,

    >
    > > > So far so good. In the relevant applications, the code looks something
    > > > like this:
    > > >logging.config.fileConfig('log.ini')
    > > > logger =logging.getLogger('log.regular') <or> logger =
    > > >logging.getLogger('log.daemonic')
    > > > .. and startlogging.

    >
    > > > The thorn in my side is that after the fileConfig call, BOTH handlers
    > > > are instantiated, meaning both types of files are created for every
    > > > component, even if I don't need it: component.log (for the rotating
    > > > handler) and compenent.date.log (for the regular file handler).

    >
    > > > ..So finally, here's my question:
    > > > Apart from splitting theloggingconfiguration into two separate
    > > > files, is there any way to NOT create the file until you actually use
    > > > it?

    >
    > > You can generate the .ini file on the fly and then load it:

    >
    > > >>> log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
    > > >>> atexit.register(lambda: remove(log_ini))
    > > >>>logging.config.fileConfig(log_ini)

    >
    > > HTH,
    > > --
    > > Miki <>http://pythonwise.blogspot.com

    >
    > I think it misses the point of having a file to config..
    > It looks as if there's no solution. A peek at theloggingmodule shows
    > this:
    > klass = cp.get(sectname, "class")
    > ...
    > klass = eval(klass, vars(logging))
    > args = cp.get(sectname, "args")
    > args = eval(args, vars(logging))
    > h = apply(klass, args)
    >
    > Bah. I'll have to split them into two configuration files (or subclass
    > and have the respective file and rotating handlers lazy evaluate until
    > the actual log call is made).
    > Thanks though.


    Not sure if it's any help - the SVN version contains a new optional
    "delay" parameter for FileHandler and subclasses (including the
    rotating handlers) which delays opening the file until there's a need
    to, i.e. when an emit() call occurs.

    Regards,

    Vinay Sajip
     
    Vinay Sajip, Apr 23, 2008
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John
    Replies:
    1
    Views:
    373
    Joerg Jooss
    Jun 26, 2005
  2. David Collier
    Replies:
    0
    Views:
    364
    David Collier
    Jul 26, 2004
  3. Royan
    Replies:
    1
    Views:
    1,452
    Andrea Francia
    Apr 1, 2008
  4. Niv (KP)
    Replies:
    2
    Views:
    410
  5. John
    Replies:
    1
    Views:
    104
    Patrice
    May 3, 2006
Loading...

Share This Page