Perl module for analyzing log files?

[D. Alvarado]

Can anyone recommend a module that I can use to break apart Apache log
files? We are doing out log file analysis on a separate machine and I
would like to find a Perl module to help me code a primitive log file
analysis program that will analyze log files as they are dumped to the
machine. Thanks - Dave

 

[J. Gleixner]

Can anyone recommend a module that I can use to break apart Apache log
files? We are doing out log file analysis on a separate machine and I
would like to find a Perl module to help me code a primitive log file
analysis program that will analyze log files as they are dumped to the
machine. Thanks - Dave

To find modules, start with CPAN:

http://search.cpan.org/

Apache:arseLog is probably what you're after.

 

[Massi]

More specifically, what would be the right module to generically parse
timestamps in any of these log files and do date arithmetic (ie. find
all lines from a specified time period)?

I'd like to put proper date parsing in the logwatch package
(http://www.logwatch.org/) and I'd like to use a date parser that's
generally available on most OS's that run syslog/Apache/BIND.

Try using Date:arse
I used in the past to generate a filtered "messages".
For example: I told the script to output me the messages generated in the
last xxx hours.
Here is a snippet of code:

#!/bin/perl -w
use Date:arse;
my $HoursBefore=24;
$HoursBefore=time()-($HoursBefore*3600); # transform in second
open(MESSFILE, "</var/log/syslog");
while (<MESSFILE>) {
my $date = /^(.+?\s.+?\s.+?\s)/
# transform in second ???? maybe. But I'm sure there is a way to do it.
# See POD file of the module.
my $dateNum = str2time($date);

if ($dateNum > $HoursBefore) {
.... positive command
} else {
.... negative commands
}
}
close MESSFILE;

This is something I resume from an old (and maybe not functioning)
script.

I never used to parse Apache/BIND logs ... but I think you can with some
trials.

Massi

 

[Massi]

Thanks. How does that compare to Date::Manip or Time:arseDate?

Sorry ...
I never used those modules...

M.