python logging filters

[Grimsqueaker]

I have found some strange behaviour when using the Python 2.6 logging
module, maybe someone here can explain what's happening?

When I add a Filter to a Handler, everything works as expected (ie.
all messages sent from Loggers below the Filter's level are allowed
through), but when I add the Filter directly on to the Logger, only
that Logger is blocked, regardless of the contents of the Filter.

The code below will show the expected and desired output when run. To
see the inexplicable output, use the commented line in place of the
line above it.

code:

import logging


root_logger = logging.getLogger()
loggerA = logging.getLogger('A')

loggerA1 = logging.getLogger('A.1')

loggerA2 = logging.getLogger('A.2')

loggerA11 = logging.getLogger('A.1.1')

loggerA12 = logging.getLogger('A.1.2')

loggerB = logging.getLogger('B')

loggerB1 = logging.getLogger('B.1')

loggerB11 = logging.getLogger('B.1.1')


root_logger.addHandler(logging.StreamHandler())

f = logging.Filter(loggerB.name)

root_logger.handlers[0].addFilter(f)

#root_logger.addFilter(f)

for l in [root_logger, loggerA, loggerB, loggerA1, loggerA2, loggerB1,
loggerA11, loggerA12, loggerB11]:

l.critical(l.name)

correct output:

B
B.1
B.1.1

weird output:

A
B
A.1
A.2
B.1
A.1.1
A.1.2
B.1.1


Thanks

 

[Vinay Sajip]

When I add a Filter to a Handler, everything works as expected (ie.
all messages sent from Loggers below the Filter's level are allowed
through), but when I add the Filter directly on to the Logger, only
that Logger is blocked, regardless of the contents of the Filter.

The key thing to remember is that when a logger processes an event,
handlers attached to it *and all its parents* are offered the event
for handling. In the case where you have just one handler and it has
the filter attached, filtering works as you expected. By attaching the
filter to the root logger, you are not filtering its handler; this
handler is invoked for events logged to all the other loggers, and so
(apart from the event at the root logger) those events are not
filtered.

For some examples of filter usage, see this post:

http://groups.google.com/group/comp.lang.python/msg/2eb4cf8f879c6451

Regards,

Vinay Sajip

 

[Grimsqueaker]

The key thing to remember is that when a logger processes an event,
handlers attached to it *and all its parents* are offered the event
for handling. In the case where you have just one handler and it has
the filter attached, filtering works as you expected. By attaching the
filter to the root logger, you are not filtering its handler; this
handler is invoked for events logged to all the other loggers, and so
(apart from the event at the root logger) those events are not
filtered.

For some examples of filter usage, see this post:

http://groups.google.com/group/comp.lang.python/msg/2eb4cf8f879c6451

Regards,

Vinay Sajip

OK, that makes sense, but does this mean that I am unable to block a
logging path from one point? ie. in my example, I would have to add
Filter(loggerB.name) to every Logger in the 'A' path to block the A
path from showing at my root Handler? Is there no way I can just block
the whole 'A' path from one point? I was under the impression that
Filters worked hierarchically and that a message would be passed up
the chain of Loggers until it was stopped by a Filter (or its loglevel
was not allowed).

Thanks for your help

 

[Grimsqueaker]

OK, that makes sense, but does this mean that I am unable to block a
logging path from one point? ie. in my example, I would have to add
Filter(loggerB.name) to every Logger in the 'A' path to block the A
path from showing at my root Handler? Is there no way I can just block
the whole 'A' path from one point? I was under the impression that
Filters worked hierarchically and that a message would be passed up
the chain of Loggers until it was stopped by a Filter (or its loglevel
was not allowed).

Thanks for your help

So would I be correct in saying that Filters apply only the the object
they are attached to and have no effect on how messages are propagated
through the logging hierarchy? If this is the case my next question
would be: How can I affect whether or not a message is propagated
further up the tree?

Sorry to post again so quickly.

 

[Grimsqueaker]

OK, that makes sense, but does this mean that I am unable to block a
logging path from one point? ie. in my example, I would have to add
Filter(loggerB.name) to every Logger in the 'A' path to block the A
path from showing at my root Handler? Is there no way I can just block
the whole 'A' path from one point? I was under the impression that
Filters worked hierarchically and that a message would be passed up
the chain of Loggers until it was stopped by a Filter (or its loglevel
was not allowed).

Thanks for your help

So would I be correct in saying that Filters apply only the the object
they are attached to and have no effect on how messages are propagated
through the logging hierarchy? If this is the case my next question
would be: How can I affect whether or not a message is propagated
further up the tree?

Sorry to post again so quickly.

 

[Vinay Sajip]

So would I be correct in saying that Filters apply only the the object
they are attached to and have no effect on how messages are propagated
through thelogginghierarchy? If this is the case my next question
would be: How can I affect whether or not a message is propagated
further up the tree?

You are correct about how Filters work. To prevent propagation, use
the propagate attribute, documented here:

http://docs.python.org/library/logging.html#logger-objects

Most times, you don't need to use this. For example, if you want
events in the A hierarchy to go to one place and events in the B
hierarchy to go to another place (e.g. two different log files), you
just attach different handlers (e.g. two different FileHandlers) to
loggers A and B. If you want a combined log as well, add an
appropriate handler to the root logger.

I find that things usually work OK if I just determine what handlers I
need and attach them to the appropriate loggers, setting the levels on
handlers and loggers appropriately. If I need filtering logic for a
logger or handler which is more involved than just an integer
threshold (which happens less often), I use Filters. Even less often,
if I need to block events from a part of the logger hierarchy from
bubbling upwards, then I use the propagate attribute.

Regards,

Vinay Sajip

 

[Grimsqueaker]

My logging behaves as I expect now and I have a better understanding
of how the module functions.
Thank you for taking the time to explain those points to me.

 

[Grimsqueaker]

My logging behaves as I expect now and I have a better understanding
of how the module functions.
Thank you for taking the time to explain those points to me.

 

[Grimsqueaker]

My logging behaves as I expect now and I have a better understanding
of how the module functions.
Thank you for taking the time to explain those points to me.