Search log for string and display hourly/daily report

Discussion in 'Python' started by spek06@gmail.com, Jan 23, 2013.

  1. Guest

    I need to search a log file for a specific string (Successfully Sent) and report the number of instances in the last hour (from when executed) and total for the day so far (midnight till the time executed). Can anyone provideany examples of such a program or get me started?
    , Jan 23, 2013
    #1
    1. Advertising

  2. Tim Chase Guest

    On 01/23/13 13:05, wrote:
    > I need to search a log file for a specific string (Successfully
    > Sent) and report the number of instances in the last hour (from
    > when executed) and total for the day so far (midnight till the
    > time executed). Can anyone provide any examples of such a program
    > or get me started?


    You'd have to specify additional details on how the log-file is
    formatted, presumably how it's delimited, and what the format of the
    corresponding time-stamp is. Something like

    "I have a tab-delimited log file where the 4th column is the
    timestamp in the format 'YYYY-MM-DD HH:mm:ss' and the 18th column is
    the status. I want to search for items where the status contains
    'Successfully Sent' and then further filter them by (1) events in
    the last hour, and (2) all events today"

    -tkc
    Tim Chase, Jan 23, 2013
    #2
    1. Advertising

  3. John Gordon Guest

    In <> writes:

    > I need to search a log file for a specific string (Successfully Sent) and
    > report the number of instances in the last hour (from when executed) and
    > total for the day so far (midnight till the time executed). Can anyone
    > provide any examples of such a program or get me started?


    from datetime import datetime, timedelta
    from time import mktime, strptime

    now = datetime.now()
    midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
    one_hour_ago = now - timedelta(hours=1)

    daily_instances = 0
    hourly_instances = 0

    with open('myfile.log') as logfile:
    for line in logfile:
    if 'Successfully Sent' in line:
    time_string = line[0:19]
    struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
    log_time = datetime.fromtimestamp(mktime(struct))

    if log_time > midnight:
    daily_instances += 1

    if log_time > one_hour_ago:
    hourly_instances += 1

    print "Instances in the last hour: ", hourly_instances
    print "Instances since midnight: ", daily_instances


    This code assumes that log lines begin with a timestamp similar to
    "2013-01-23T09:27:01". If the timestamp is in a different format, or
    occurs elsewhere in the line, you'll have to adjust for that.

    --
    John Gordon A is for Amy, who fell down the stairs
    B is for Basil, assaulted by bears
    -- Edward Gorey, "The Gashlycrumb Tinies"
    John Gordon, Jan 23, 2013
    #3
  4. On 01/23/2013 12:05 PM, wrote:
    > I need to search a log file for a specific string (Successfully Sent)
    > and report the number of instances in the last hour (from when
    > executed) and total for the day so far (midnight till the time
    > executed). Can anyone provide any examples of such a program or get
    > me started?


    Take a look at this very interesting presentation/document:

    http://www.dabeaz.com/generators/
    Michael Torrie, Jan 23, 2013
    #4
  5. Guest

    On Wednesday, January 23, 2013 4:25:59 PM UTC-5, John Gordon wrote:
    > In <> spek06 writes:
    >
    >
    >
    > > I need to search a log file for a specific string (Successfully Sent) and

    >
    > > report the number of instances in the last hour (from when executed) and

    >
    > > total for the day so far (midnight till the time executed). Can anyone

    >
    > > provide any examples of such a program or get me started?

    >
    >
    >
    > from datetime import datetime, timedelta
    >
    > from time import mktime, strptime
    >
    >
    >
    > now = datetime.now()
    >
    > midnight = now.replace(hour=0, minute=0, second=0, microsecond=0)
    >
    > one_hour_ago = now - timedelta(hours=1)
    >
    >
    >
    > daily_instances = 0
    >
    > hourly_instances = 0
    >
    >
    >
    > with open('myfile.log') as logfile:
    >
    > for line in logfile:
    >
    > if 'Successfully Sent' in line:
    >
    > time_string = line[0:19]
    >
    > struct = strptime(time_string, "%Y-%m-%dT%H:%M:%S")
    >
    > log_time = datetime.fromtimestamp(mktime(struct))
    >
    >
    >
    > if log_time > midnight:
    >
    > daily_instances += 1
    >
    >
    >
    > if log_time > one_hour_ago:
    >
    > hourly_instances += 1
    >
    >
    >
    > print "Instances in the last hour: ", hourly_instances
    >
    > print "Instances since midnight: ", daily_instances
    >
    >
    >
    >
    >
    > This code assumes that log lines begin with a timestamp similar to
    >
    > "2013-01-23T09:27:01". If the timestamp is in a different format, or
    >
    > occurs elsewhere in the line, you'll have to adjust for that.
    >
    >
    >
    > --
    >
    > John Gordon A is for Amy, who fell down the stairs
    >
    > B is for Basil, assaulted by bears
    >
    > -- Edward Gorey, "The Gashlycrumb Tinies"


    Thanks John, I think this will definitely help get me started!
    , Jan 24, 2013
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Creating a log on daily basis

    , Sep 8, 2006, in forum: C Programming
    Replies:
    3
    Views:
    284
    Dave Thompson
    Sep 21, 2006
  2. AAaron123
    Replies:
    1
    Views:
    266
    Cowboy \(Gregory A. Beamer\)
    Oct 14, 2008
  3. Alex
    Replies:
    1
    Views:
    356
  4. Abby Lee
    Replies:
    5
    Views:
    375
    Abby Lee
    Aug 2, 2004
  5. Tony Kemp

    [ANN] sa-daily-report

    Tony Kemp, Mar 24, 2006, in forum: Ruby
    Replies:
    1
    Views:
    77
    Tony Kemp
    Mar 25, 2006
Loading...

Share This Page