Search log for string and display hourly/daily report

S

spek06

I need to search a log file for a specific string (Successfully Sent) and report the number of instances in the last hour (from when executed) and total for the day so far (midnight till the time executed). Can anyone provideany examples of such a program or get me started?
 
T

Tim Chase

I need to search a log file for a specific string (Successfully
Sent) and report the number of instances in the last hour (from
when executed) and total for the day so far (midnight till the
time executed). Can anyone provide any examples of such a program
or get me started?

You'd have to specify additional details on how the log-file is
formatted, presumably how it's delimited, and what the format of the
corresponding time-stamp is. Something like

"I have a tab-delimited log file where the 4th column is the
timestamp in the format 'YYYY-MM-DD HH:mm:ss' and the 18th column is
the status. I want to search for items where the status contains
'Successfully Sent' and then further filter them by (1) events in
the last hour, and (2) all events today"

-tkc
 
J

John Gordon

In said:
I need to search a log file for a specific string (Successfully Sent) and
report the number of instances in the last hour (from when executed) and
total for the day so far (midnight till the time executed). Can anyone
provide any examples of such a program or get me started?

from datetime import datetime, timedelta
from time import mktime, strptime

now = datetime.now()
midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
one_hour_ago = now - timedelta(hours=1)

daily_instances = 0
hourly_instances = 0

with open('myfile.log') as logfile:
for line in logfile:
if 'Successfully Sent' in line:
time_string = line[0:19]
struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
log_time = datetime.fromtimestamp(mktime(struct))

if log_time > midnight:
daily_instances += 1

if log_time > one_hour_ago:
hourly_instances += 1

print "Instances in the last hour: ", hourly_instances
print "Instances since midnight: ", daily_instances


This code assumes that log lines begin with a timestamp similar to
"2013-01-23T09:27:01". If the timestamp is in a different format, or
occurs elsewhere in the line, you'll have to adjust for that.
 
M

Michael Torrie

I need to search a log file for a specific string (Successfully Sent)
and report the number of instances in the last hour (from when
executed) and total for the day so far (midnight till the time
executed). Can anyone provide any examples of such a program or get
me started?

Take a look at this very interesting presentation/document:

http://www.dabeaz.com/generators/
 
S

spek06

I need to search a log file for a specific string (Successfully Sent) and
report the number of instances in the last hour (from when executed) and
total for the day so far (midnight till the time executed). Can anyone
provide any examples of such a program or get me started?



from datetime import datetime, timedelta

from time import mktime, strptime



now = datetime.now()

midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)

one_hour_ago = now - timedelta(hours=1)



daily_instances = 0

hourly_instances = 0



with open('myfile.log') as logfile:

for line in logfile:

if 'Successfully Sent' in line:

time_string = line[0:19]

struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")

log_time = datetime.fromtimestamp(mktime(struct))



if log_time > midnight:

daily_instances += 1



if log_time > one_hour_ago:

hourly_instances += 1



print "Instances in the last hour: ", hourly_instances

print "Instances since midnight: ", daily_instances





This code assumes that log lines begin with a timestamp similar to

"2013-01-23T09:27:01". If the timestamp is in a different format, or

occurs elsewhere in the line, you'll have to adjust for that.



--

John Gordon A is for Amy, who fell down the stairs

B is for Basil, assaulted by bears

-- Edward Gorey, "The Gashlycrumb Tinies"

Thanks John, I think this will definitely help get me started!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top