A simpler logging configuration file format?

Discussion in 'Python' started by geoff.bache@gmail.com, Jun 5, 2009.

  1. Guest

    Hi all,

    I wonder if there are others out there who like me have tried to use
    the logging module's configuration file and found it bloated and over-
    complex for simple usage (i.e. a collection of loggers writing to
    files)

    At the moment, if I want to add a new logger "foo" writing to its own
    file "foo" to my config file, I have to repeat the name 7(!) times,
    editing 3 different sections in the process:

    1) Add "foo" to [loggers] and [handlers]. (These seem completely
    pointless and were apparently added in anticipation of new
    functionality that never happened).

    2) Create the section
    [logger_foo]
    handler:foo
    qualname:foo
    level:INFO

    3) Create the section
    [handler_foo]
    class: FileHandler
    args:("foo", "a")

    Wondering how it got like this, I found this blog from soon after it
    was released in 2004:

    http://www.mechanicalcat.net/richard/log/Python/Simple_usage_of_Python_s_logging_module

    Some of the verdicts are "full of dead chickens and error prone",
    "horribly complicated" , "over-engineered". So along comes the
    "basicConfig" method in 2005 which is supposed to address thes
    concerns. But this can only be applied globally, not even to
    individual loggers, so is only helpful for extremely basic usage
    (everything to one target) as far as I can see. The config file is
    still much as it was then.

    I'd really like to see the concept extended to encompass multiple
    loggers and the config file. Allowing Logger.basicConfig should be
    trivial. Writing a configuration format which took a section and
    passed all the options in it to basicConfig would in my view lead to a
    much friendlier syntax for normal usage.
     
    , Jun 5, 2009
    #1
    1. Advertising

  2. In message
    <>,
    wrote:

    > At the moment, if I want to add a new logger "foo" writing to its own
    > file "foo" to my config file, I have to repeat the name 7(!) times,
    > editing 3 different sections in the process:


    I suppose there's always the Sendmail solution: stick an m4 wrapper on top
    to hide the details of the process. :)
     
    Lawrence D'Oliveiro, Jun 6, 2009
    #2
    1. Advertising

  3. Vinay Sajip Guest

    On Jun 5, 9:07 pm, wrote:
    > Hi all,


    Some discussion about this has already taken place on the issue
    tracker:

    http://bugs.python.org/issue6136

    and I thought I would summarise my comments there on this thread, for
    people who have no need to look at that issue.

    > At the moment, if I want to add a new logger "foo" writing to its own
    > file "foo" to my config file, I have to repeat the name 7(!) times,
    > editing 3 different sections in the process:


    Strictly speaking, you don't need to repeat the name "foo" 7 times.

    > 1) Add "foo" to [loggers] and [handlers]. (These seem completely
    > pointless and were apparently added in anticipation of new
    > functionality that never happened).


    You don't have to call the logger "foo" and the handler "foo". The
    names in the [loggers] and [handlers] section are just to allow the
    system to cross-refer different entities (loggers, handlers,
    formatters) declared in the config file. You could give the logger a
    name of "L1" and the handler a name of "H1", for example.

    > 2) Create the section
    > [logger_foo]
    > handler:foo
    > qualname:foo
    > level:INFO


    Here, if you named your logger L1, the section would be [logger_L1],
    not [logger_foo]. If you called your handler H1, then the handler line
    would be handlers=H1. The qualname is "foo" because you choose to log
    events to the logger named "foo". It could be more specific, e.g.
    "myapp.mymodule.myfunctionalarea".

    > 3) Create the section
    > [handler_foo]
    > class: FileHandler
    > args:("foo", "a")


    Here, if you named your handler H1, the section would be [handler_H1],
    not [handler_foo]. You've also called the log file "foo", which more
    conventionally you might name "foo.log".

    In your simple example where everything is named foo, and you've
    chosen to call cross-referencing handles within the file "foo", then
    naturally you repeat "foo" a lot of times. In an equivalent file

    [loggers]
    keys=root,L1

    [handlers]
    keys=H1

    [logger_L1]
    level=INFO
    handlers=H1
    qualname=foo

    [handler_H1]
    class=FileHandler
    args=('foo.log', 'w')


    "foo" only occurs twice.

    > Wondering how it got like this, I found this blog from soon after it
    > was released in 2004:
    >
    > http://www.mechanicalcat.net/richard/log/Python/Simple_usage_of_Pytho...
    >
    > Some of the verdicts are "full of dead chickens and error prone",
    > "horribly complicated" , "over-engineered". So along comes the
    > "basicConfig" method in 2005 which is supposed to address thes
    > concerns. But this can only be applied globally, not even to
    > individual loggers, so is only helpful for extremely basic usage
    > (everything to one target) as far as I can see. The config file is
    > still much as it was then.


    Some of this is just a consequence of using ConfigParser. The existing
    file format is not perfect or even complete (for example - no support
    for filters), and was proposed on python-dev for review before the
    logging package went into Python. The general consensus was that it
    was OK for basic use, and IIRC several people considered it as +0 or
    -0 because they envisaged doing their own thing (e.g. having the
    logging configuration as just part of an application's overall
    configuration). This is the reason configuration was put in a separate
    sub-module - if it's not needed, it needn't be loaded. AFAICT since
    basicConfig was introduced and later refined, there has not been much
    feedback about the configuration file format being a problem - and
    while that could be because no-one is using it, that's not borne out
    by the evidence. There have been numerous instances on c.l.py where
    questions have been raised about how to specify custom handlers in the
    configuration file, so at least some people are using it or have used
    it (the configuration functionality, I mean).

    > I'd really like to see the concept extended to encompass multiple
    > loggers and the config file. Allowing Logger.basicConfig should be
    > trivial. Writing a configuration format which took a section and
    > passed all the options in it to basicConfig would in my view lead to a
    > much friendlier syntax for normal usage.


    Well, basicConfig is for basic configuration of the logging package as
    a whole, not for configuring individual loggers or handlers. It's best
    not to conflate loggers and handlers - basicConfig might appear to do
    this because its arguments are partly specifying handler configuration
    and partly logger configuration. This is because it's the simplest
    case - one logger (the root logger) and one handler (according to the
    relevant arguments passed in). The handler-related arguments are used
    to create a default handler and/or formatter, the logging level is
    specified by the level argument, and it's applied to the root logger.

    It doesn't make sense to use the same approach if you are going to
    configure multiple loggers and handlers; if the existing format is not
    palatable, it's relatively easy to create your own format and write
    code to parse it and load it, and then call the programmatic API to
    configure logging appropriately. You mentioned on the bug tracker that
    you are coming from a background of using log4py - in which I believe
    loggers provide the functionality which is found in handlers in
    Python's logging.

    If you believe you have an idea for a much simpler configuration
    format (perhaps for simpler use cases, or perhaps for general use) I
    would encouraging writing an implementation and putting it up on PyPI
    so that people can play around with it and give feedback. ISTM that
    there is a general feeling that any sizeable new or changed
    functionality intended for the stdlib should be proven in the
    community through wide use before being seriously considered for
    inclusion. For example, I wrote a hierarchical configuration module,
    just one of several configuration-related modules on PyPI:

    http://pypi.python.org/pypi?:action=search&term=config&submit=search

    However, I am not proposing it for inclusion in the stdlib, as I don't
    believe it has had enough feedback yet from the community to be able
    to accurately gauge its quality or utility.
     
    Vinay Sajip, Jun 17, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tobias Besch
    Replies:
    0
    Views:
    339
    Tobias Besch
    Oct 18, 2004
  2. Tobias Besch
    Replies:
    4
    Views:
    4,921
    Tobias Besch
    Oct 27, 2004
  3. Alessandro Bottoni

    Is there a better/simpler logging module?

    Alessandro Bottoni, Aug 8, 2005, in forum: Python
    Replies:
    2
    Views:
    249
    Michael Hoffman
    Aug 8, 2005
  4. Replies:
    3
    Views:
    338
    Vinay Sajip
    Apr 23, 2008
  5. Lie Ryan

    Making logging.getLogger() simpler

    Lie Ryan, Sep 18, 2010, in forum: Python
    Replies:
    3
    Views:
    877
    Vinay Sajip
    Sep 19, 2010
Loading...

Share This Page