line duplication using logging to file

Discussion in 'Python' started by seb, Jan 9, 2007.

  1. seb

    seb Guest

    Hi, I am writing to a file some basic information using the logging
    module. It is working but in the log file some line are printed
    several time. I had put some print debugging messages in the logging
    function (so they appear on the consile) and they are called once only.
    Obviously there is some understantding of the logging module that I am
    missing.

    My simple logging program (see below) is called by several processes.
    In this way I can collect the information from various sources (and not
    use the network enabled logging module)

    I am using python 2.4 on WinXP SP2.

    Do you have any idea ? Thanks in advance.

    Seb.

    *****************************************
    The very simple "logging program ":
    *****************************************
    import logging, logging.handlers
    import time

    def write_log(level, message):
    # Utilisation de l'API pour le Handler global
    print "time.asctime()",time.asctime(),"received level=",level,"message
    =",message
    nom_logger="main_log_file"
    logger=logging.getLogger(nom_logger)
    logger.setLevel(logging.DEBUG)
    prefix = "pix_main_log_file"
    #fh=logging.handlers.RotatingFileHandler(prefix + "_log.txt", 'a',
    1000000,10)
    fh=logging.FileHandler("main_log.txt")
    fh.setLevel(logging.DEBUG)
    #formater = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s
    - %(message)s")
    formater = logging.Formatter("%(asctime)s - %(levelname)s -
    %(message)s")
    fh.setFormatter(formater)
    logger.addHandler(fh)
    #logger.info("***************************** debut")


    #message = main(url,conf_file,logger)
    #message="hello seb"
    if str(level).lower() == "info" :
    print "logger info"
    logger.info(str(message))
    elif str(level).lower() =="error":
    print "logger error"
    logger.error(str(message))
    elif str(level).lower()=="warning" :
    print "logger warning"
    logger.warning(str(message))
    elif str(level).lower() =="critical":
    print "logger critical"
    logger.critical(str(message))
    elif str(level).lower() == "exception":
    print "logger exception"
    logger.exception(str(message))
    else :
    logger.info("niveau inconnu "+str(message))

    print "_________",message
    #print dir(logger)
    return

    *************************************************
    Example of the log file with duplicated line.
    **************************************************

    2007-01-08 18:26:19,578 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version
    simple\thread_RS232.py___init_rs232initrs232_openCOM1
    2007-01-08 18:26:19,578 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version
    simple\thread_RS232.py___run____thread lance
    2007-01-08 18:26:19,578 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version
    simple\thread_RS232.py___run____thread lance
    2007-01-08 18:26:32,015 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:32,015 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:32,015 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:42,483 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:42,483 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:42,483 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:42,483 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:53,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:53,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:53,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:53,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:26:53,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:03,092 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:03,092 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:03,092 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:03,092 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:03,092 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:03,092 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:13,671 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,796 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-08 18:27:14,890 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:51:26,562 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version
    simple\thread_RS232.py___init_rs232initrs232_openCOM1
    2007-01-09 08:51:26,733 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version
    simple\thread_RS232.py___run____thread lance
    2007-01-09 08:51:26,733 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version
    simple\thread_RS232.py___run____thread lance
    2007-01-09 08:51:39,453 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:39,453 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:39,453 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:48,280 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:48,280 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:48,280 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:48,280 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:58,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:58,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:58,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:58,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:51:58,750 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:09,812 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:09,812 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:09,812 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:09,812 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:09,812 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:09,812 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:19,078 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
    1 = OK
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,078 - WARNING - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
    dans le run car continue = 0
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret
    2007-01-09 08:52:22,125 - INFO - C:\Documents and
    Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
    demande d'arret


    **************************
    Program calling the log :
    ***************************
    import appel_log
    ......

    message = "something"
    nom_function_actuelle= str(sys._getframe().f_code.co_filename)
    +"___"+str(sys._getframe().f_code.co_name)
    appel_log.write_log("info",nom_function_actuelle+message)
    .................
    seb, Jan 9, 2007
    #1
    1. Advertising

  2. seb

    Vinay Sajip Guest

    seb wrote:

    > Hi, I am writing to a file some basic information using the logging
    > module. It is working but in the log file some line are printed
    > several time. I had put some print debugging messages in the logging
    > function (so they appear on the consile) and they are called once only.
    > Obviously there is some understantding of the logging module that I am
    > missing.
    >
    > My simple logging program (see below) is called by several processes.
    > In this way I can collect the information from various sources (and not
    > use the network enabled logging module)
    >
    > I am using python 2.4 on WinXP SP2.
    >
    > Do you have any idea ? Thanks in advance.
    >


    I copied your two scripts into "appel_main.py" and "appel_log.py" and
    when I run, I get a single line in main_log.txt:

    2007-01-09 10:01:42,766 - INFO -C:\temp\appel_main.py___?something

    On the console is printed:

    time.asctime() Tue Jan 09 10:04:25 2007 received level= info message=
    C:\temp\appel_main.py___?something
    logger info
    _________ C:\temp\appel_main.py___?something

    I didn't see any repetitions. Perhaps it's to do with multiple
    processes in your environment?

    If you call write_log several times you will add a handler several
    times - this is not a normal pattern of use. You should separate setup
    of Loggers/Handlers/Formatters from actually logging events via
    debug(), info() etc.

    Best regards,

    Vinay Sajip
    Vinay Sajip, Jan 9, 2007
    #2
    1. Advertising

  3. seb

    Paul McGuire Guest

    "seb" <> wrote in message
    news:...
    > Hi, I am writing to a file some basic information using the logging
    > module. It is working but in the log file some line are printed
    > several time. I had put some print debugging messages in the logging
    > function (so they appear on the consile) and they are called once only.
    > Obviously there is some understantding of the logging module that I am
    > missing.
    >
    > My simple logging program (see below) is called by several processes.
    > In this way I can collect the information from various sources (and not
    > use the network enabled logging module)
    >
    > I am using python 2.4 on WinXP SP2.
    >
    > Do you have any idea ? Thanks in advance.
    >
    > Seb.
    >


    A quick tally of log messages by timestamp and comment gives this data:

    ('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
    ('2007-01-08 18:26:19,578', '___run____thread lance') : 2
    ('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
    ('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
    ('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
    ('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
    ('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
    ('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
    ('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
    ('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
    ('2007-01-09 08:51:26,733', '___run____thread lance') : 2
    ('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
    ('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
    ('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
    ('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
    ('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
    ('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
    ('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
    ('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

    Does this suggest anything to you?

    -- Paul


    (BTW, here is the pyparsing program I used to do this analysis)

    from pyparsing import
    Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine

    # create pyparsing grammar definition for a log line
    date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
    time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
    ','+Word(nums,exact=3)
    timestamp = Combine(date + ' ' + time)
    severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
    data
    backslash = Literal("\\")
    fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
    ".py")
    logline = ( timestamp.setResultsName("timestamp") + "-" +
    severity.setResultsName("severity") + "-" +
    fileref.setResultsName("fileref") +
    restOfLine.setResultsName("comment") )

    # create list of ParseResults, with addressable log line elements by results
    name
    logEntries = [ logline.parseString(line) for line in logdata ]

    # tally up log lines by timestamp and comment
    tallyByTimestamp = {}
    for entry in logEntries:
    tallyKey = (entry.timestamp, entry.comment)
    tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

    for ts in sorted( tallyByTimestamp.items() ):
    print "%s : %d" % ts
    Paul McGuire, Jan 9, 2007
    #3
  4. seb

    seb Guest

    Hi,

    Thanks for the help.
    Meanwhile I have written the logging function from scratch and it works
    without the multiple lines.

    This means that the multiple line copy is not due to the multiple
    processes (or thread) trying to access the log file but to something
    else.

    Thanks.
    Sebastien.

    the new function :


    import logging
    import time

    def write_log(level, message):
    nom_logger="main_log_file.txt"
    logging.basicConfig(level=logging.DEBUG,
    format='%(asctime)s %(levelname)s %(message)s',
    filename=nom_logger,
    filemode='w')

    if str(level).lower() == "info" :
    logging.info(str(message))
    elif str(level).lower() =="error":
    logging.error(str(message))
    elif str(level).lower()=="warning" :
    logging.warning(str(message))
    elif str(level).lower() =="critical":
    logging.critical(str(message))
    elif str(level).lower() == "exception":
    logging.exception(str(message))
    else :
    logging.INFO(str(message))

    return







    Paul McGuire a écrit :
    > "seb" <> wrote in message
    > news:...
    > > Hi, I am writing to a file some basic information using the logging
    > > module. It is working but in the log file some line are printed
    > > several time. I had put some print debugging messages in the logging
    > > function (so they appear on the consile) and they are called once only.
    > > Obviously there is some understantding of the logging module that I am
    > > missing.
    > >
    > > My simple logging program (see below) is called by several processes.
    > > In this way I can collect the information from various sources (and not
    > > use the network enabled logging module)
    > >
    > > I am using python 2.4 on WinXP SP2.
    > >
    > > Do you have any idea ? Thanks in advance.
    > >
    > > Seb.
    > >

    >
    > A quick tally of log messages by timestamp and comment gives this data:
    >
    > ('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
    > ('2007-01-08 18:26:19,578', '___run____thread lance') : 2
    > ('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
    > ('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
    > ('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
    > ('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
    > ('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
    > ('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
    > ('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
    > ('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
    > ('2007-01-09 08:51:26,733', '___run____thread lance') : 2
    > ('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
    > ('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
    > ('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
    > ('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
    > ('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
    > ('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
    > ('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
    > ('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1
    >
    > Does this suggest anything to you?
    >
    > -- Paul
    >
    >
    > (BTW, here is the pyparsing program I used to do this analysis)
    >
    > from pyparsing import
    > Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine
    >
    > # create pyparsing grammar definition for a log line
    > date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
    > time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
    > ','+Word(nums,exact=3)
    > timestamp = Combine(date + ' ' + time)
    > severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
    > data
    > backslash = Literal("\\")
    > fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
    > ".py")
    > logline = ( timestamp.setResultsName("timestamp") + "-" +
    > severity.setResultsName("severity") + "-" +
    > fileref.setResultsName("fileref") +
    > restOfLine.setResultsName("comment") )
    >
    > # create list of ParseResults, with addressable log line elements by results
    > name
    > logEntries = [ logline.parseString(line) for line in logdata ]
    >
    > # tally up log lines by timestamp and comment
    > tallyByTimestamp = {}
    > for entry in logEntries:
    > tallyKey = (entry.timestamp, entry.comment)
    > tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1
    >
    > for ts in sorted( tallyByTimestamp.items() ):
    > print "%s : %d" % ts
    seb, Jan 9, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Stefan Siegl
    Replies:
    0
    Views:
    949
    Stefan Siegl
    Aug 27, 2003
  2. Albretch
    Replies:
    0
    Views:
    280
    Albretch
    Dec 24, 2004
  3. simon.alexandre
    Replies:
    3
    Views:
    671
    simon.alexandre
    Jan 11, 2005
  4. scad
    Replies:
    23
    Views:
    1,157
    Alf P. Steinbach
    May 17, 2009
  5. Andrew Poulos

    Minimising duplication using ((){})();

    Andrew Poulos, Jul 16, 2008, in forum: Javascript
    Replies:
    1
    Views:
    90
    Lasse Reichstein Nielsen
    Jul 16, 2008
Loading...

Share This Page