line duplication using logging to file

S

seb

Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.

*****************************************
The very simple "logging program ":
*****************************************
import logging, logging.handlers
import time

def write_log(level, message):
# Utilisation de l'API pour le Handler global
print "time.asctime()",time.asctime(),"received level=",level,"message
=",message
nom_logger="main_log_file"
logger=logging.getLogger(nom_logger)
logger.setLevel(logging.DEBUG)
prefix = "pix_main_log_file"
#fh=logging.handlers.RotatingFileHandler(prefix + "_log.txt", 'a',
1000000,10)
fh=logging.FileHandler("main_log.txt")
fh.setLevel(logging.DEBUG)
#formater = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s
- %(message)s")
formater = logging.Formatter("%(asctime)s - %(levelname)s -
%(message)s")
fh.setFormatter(formater)
logger.addHandler(fh)
#logger.info("***************************** debut")


#message = main(url,conf_file,logger)
#message="hello seb"
if str(level).lower() == "info" :
print "logger info"
logger.info(str(message))
elif str(level).lower() =="error":
print "logger error"
logger.error(str(message))
elif str(level).lower()=="warning" :
print "logger warning"
logger.warning(str(message))
elif str(level).lower() =="critical":
print "logger critical"
logger.critical(str(message))
elif str(level).lower() == "exception":
print "logger exception"
logger.exception(str(message))
else :
logger.info("niveau inconnu "+str(message))

print "_________",message
#print dir(logger)
return

*************************************************
Example of the log file with duplicated line.
**************************************************

2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openCOM1
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:51:26,562 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openCOM1
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret


**************************
Program calling the log :
***************************
import appel_log
......

message = "something"
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
appel_log.write_log("info",nom_function_actuelle+message)
.................
 
V

Vinay Sajip

seb said:
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.
Click to expand...

I copied your two scripts into "appel_main.py" and "appel_log.py" and
when I run, I get a single line in main_log.txt:

2007-01-09 10:01:42,766 - INFO -C:\temp\appel_main.py___?something

On the console is printed:

time.asctime() Tue Jan 09 10:04:25 2007 received level= info message=
C:\temp\appel_main.py___?something
logger info
_________ C:\temp\appel_main.py___?something

I didn't see any repetitions. Perhaps it's to do with multiple
processes in your environment?

If you call write_log several times you will add a handler several
times - this is not a normal pattern of use. You should separate setup
of Loggers/Handlers/Formatters from actually logging events via
debug(), info() etc.

Best regards,

Vinay Sajip
 
P

Paul McGuire

seb said:
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.
Click to expand...

A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul


(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts
 
S

seb

Hi,

Thanks for the help.
Meanwhile I have written the logging function from scratch and it works
without the multiple lines.

This means that the multiple line copy is not due to the multiple
processes (or thread) trying to access the log file but to something
else.

Thanks.
Sebastien.

the new function :


import logging
import time

def write_log(level, message):
nom_logger="main_log_file.txt"
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename=nom_logger,
filemode='w')

if str(level).lower() == "info" :
logging.info(str(message))
elif str(level).lower() =="error":
logging.error(str(message))
elif str(level).lower()=="warning" :
logging.warning(str(message))
elif str(level).lower() =="critical":
logging.critical(str(message))
elif str(level).lower() == "exception":
logging.exception(str(message))
else :
logging.INFO(str(message))

return







Paul McGuire a écrit :
seb said:
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.
Click to expand...

A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul


(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts
Click to expand...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Importing package on Windows XP 5
Traceback in Logging 7
Custom Minecraft launcher client error; I think regarding java 0
How to use ufixed when it involves multiplication a number of times?(VHDL question) 0
seeking to improve Python skills 0
Logging: how to suppress default output when adding handlers? 3
Emacs Time Line, Graphical Chart by Jamie Zawinski - Valuable 0
Help with code 0

Members online

Total: 37 (members: 2, guests: 35)
Robots: 320

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,011
Latest member
AjaUqq1950

Latest Threads

Top