Multiple File Grep

[Blake]

I'm trying to figure out how to grep a bunch of log files into one
file.

Basically I have virtual hosts set up like this

/home/user/site/log

There's about 85 like that. Within /log/ there is an access_log file

So what I want to do is to be able to grep out all the hits in a
certain hour to see who's killing the server.

So what I'd like to do is do something like

find /home/ -name access_log
while { there's a log }
grep July 27 10pm
send that output to a single file

Then I can grep out the hits from that one file to see who's kiling
me.

What's the best what to do that?

 

[Josef Moellers]

I'm trying to figure out how to grep a bunch of log files into one
file.

Basically I have virtual hosts set up like this

/home/user/site/log

There's about 85 like that. Within /log/ there is an access_log file

So what I want to do is to be able to grep out all the hits in a
certain hour to see who's killing the server.

So what I'd like to do is do something like

find /home/ -name access_log
while { there's a log }
grep July 27 10pm
send that output to a single file

Then I can grep out the hits from that one file to see who's kiling
me.

What's the best what to do that?

I'd use File::Find.

 

[ctcgag]

I'm trying to figure out how to grep a bunch of log files into one
file.

Basically I have virtual hosts set up like this

/home/user/site/log

There's about 85 like that. Within /log/ there is an access_log file

So what I want to do is to be able to grep out all the hits in a
certain hour to see who's killing the server.

So what I'd like to do is do something like

find /home/ -name access_log
while { there's a log }
grep July 27 10pm
send that output to a single file

Then I can grep out the hits from that one file to see who's kiling
me.

What's the best what to do that?

system q{
grep 'July 27 10pm' /home/*/site/log/access_log > a_single_file
}q



Xho

 

[Greg Bacon]

: [...]
: So what I want to do is to be able to grep out all the hits in a
: certain hour to see who's killing the server.
:
: So what I'd like to do is do something like
:
: find /home/ -name access_log
: while { there's a log }
: grep July 27 10pm
: send that output to a single file
:
: Then I can grep out the hits from that one file to see who's kiling
: me.

Consider the code below. Example usage:

% ghrp /home 27/Jul/2004:22

#! /usr/local/bin/perl

# ghrp: search for an hour and print

use warnings;
use strict;

sub usage { "Usage: $0 root-dir time-pattern\n" }

my %mon = (
Jan => 1, Feb => 2, Mar => 3, Apr => 4, May => 5, Jun => 6,
Jul => 7, Aug => 8, Sep => 9, Oct => 10, Nov => 11, Dec => 12,
);

sub date {
my $date = shift;

my($d,$m,$y,$hr,$min,$sec);

# e.g., 03/Feb/1998:17:42:15 -0500
if (m!(\d+)/(\w+)/(\d+)\d+)\d+)\d+)!) {
($d,$m,$y,$hr,$min,$sec) = ($1,$2,$3,$4,$5,$6);

$m = $mon{$m} || 0;
}
else {
$d = $m = $y = $hr = $min = $sec = 0;
}

($d,$m,$y,$hr,$min,$sec);
}

sub date_asc {
$a->[2] <=> $b->[2] # year
||
$a->[1] <=> $b->[1] # month
||
$a->[0] <=> $b->[0] # day
||
$a->[3] <=> $b->[3] # hour
||
$a->[4] <=> $b->[4] # min
||
$a->[5] <=> $b->[5] # sec
}

## main
die usage unless @ARGV == 2;

my $root = shift;
die "$0: '$root' is not a directory!\n" . usage unless -d $root;

(my $time = shift) =~ s,/,\\/,g;
my $pat = eval "qr/" . $time . "/";

unless (defined $pat) {
die "$0: bad time pattern\n";
}

# from http://stein.cshl.org/WWW/docs/handout.html#Log_Parsing
my $line = qr/^\S+ \S+ \S+ \[($pat[^]]+)\] "\w+ \S+.*" \d+ \S+/;

my @hits;
for (`find $root -name access_log 2>&1`) {
chomp;

# assume this line is a warning if it's not a filename
unless (-f $_) {
warn $_ . "\n";
next;
}

my $fh;
unless (open $fh, "<", $_) {
warn "$0: open $_: $!\n";
next;
}

while (<$fh>) {
push @hits, [ date($1), $_ ] if /$line/;
}
}

print $_ for map $_->[6], sort date_asc @hits;

__END__

Hope this helps,
Greg