Efficient way of looging in python

[maitreymishra23]

Hi,

I need an efficient way of logging using python.
My problem statemnt:
1. I have multiple processes using the same logging file.
I need solutions to the following:
a) If multiple processes are trying to write to the same file, I need to prevent that. Otherwise, the logging messages will be puzzling to the user as he would see logging messages from one module than may be some other following it.

Please let me know if there is an efficient way to do it.

Thanks!

Br,
Maitrey

 

[Skip Montanaro]

a) If multiple processes are trying to write to the same file, I need to prevent that.

Two things: Use some sort of file locking. You can get the lockfile
module from PyPI. Include at least the process id as one of the
logging fields in your formatter. I haven't don't enough with the
logging module to know it it supports some sort of "name" field, which
would make the different messages more easily distinguished, but if
so, that would be a better alternative to the process id.

Skip

 

[Roozbeh]

You can also add several handlers to your logger with different levels, so that one Filehandler passes the errors to a log file and StreamHandler passes messages with a different level to the output stream.
Also by setting the basicConfig of the logger you can format the log messages so that you would see where the messages are originating from.