Multiple File Grep

Discussion in 'Perl Misc' started by Blake, Jul 27, 2004.

  1. Blake

    Blake Guest

    I'm trying to figure out how to grep a bunch of log files into one
    file.

    Basically I have virtual hosts set up like this

    /home/user/site/log

    There's about 85 like that. Within /log/ there is an access_log file

    So what I want to do is to be able to grep out all the hits in a
    certain hour to see who's killing the server.

    So what I'd like to do is do something like

    find /home/ -name access_log
    while { there's a log }
    grep July 27 10pm
    send that output to a single file

    Then I can grep out the hits from that one file to see who's kiling
    me.

    What's the best what to do that?
     
    Blake, Jul 27, 2004
    #1
    1. Advertising

  2. Blake wrote:
    > I'm trying to figure out how to grep a bunch of log files into one
    > file.
    >
    > Basically I have virtual hosts set up like this
    >
    > /home/user/site/log
    >
    > There's about 85 like that. Within /log/ there is an access_log file
    >
    > So what I want to do is to be able to grep out all the hits in a
    > certain hour to see who's killing the server.
    >
    > So what I'd like to do is do something like
    >
    > find /home/ -name access_log
    > while { there's a log }
    > grep July 27 10pm
    > send that output to a single file
    >
    > Then I can grep out the hits from that one file to see who's kiling
    > me.
    >
    > What's the best what to do that?


    I'd use File::Find.

    --
    Josef Möllers (Pinguinpfleger bei FSC)
    If failure had no penalty success would not be a prize
    -- T. Pratchett
     
    Josef Moellers, Jul 27, 2004
    #2
    1. Advertising

  3. Blake

    Guest

    (Blake) wrote:
    > I'm trying to figure out how to grep a bunch of log files into one
    > file.
    >
    > Basically I have virtual hosts set up like this
    >
    > /home/user/site/log
    >
    > There's about 85 like that. Within /log/ there is an access_log file
    >
    > So what I want to do is to be able to grep out all the hits in a
    > certain hour to see who's killing the server.
    >
    > So what I'd like to do is do something like
    >
    > find /home/ -name access_log
    > while { there's a log }
    > grep July 27 10pm
    > send that output to a single file
    >
    > Then I can grep out the hits from that one file to see who's kiling
    > me.
    >
    > What's the best what to do that?



    system q{
    grep 'July 27 10pm' /home/*/site/log/access_log > a_single_file
    }q



    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Jul 27, 2004
    #3
  4. Blake

    Greg Bacon Guest

    In article <>,
    Blake <> wrote:

    : [...]
    : So what I want to do is to be able to grep out all the hits in a
    : certain hour to see who's killing the server.
    :
    : So what I'd like to do is do something like
    :
    : find /home/ -name access_log
    : while { there's a log }
    : grep July 27 10pm
    : send that output to a single file
    :
    : Then I can grep out the hits from that one file to see who's kiling
    : me.

    Consider the code below. Example usage:

    % ghrp /home 27/Jul/2004:22

    #! /usr/local/bin/perl

    # ghrp: search for an hour and print

    use warnings;
    use strict;

    sub usage { "Usage: $0 root-dir time-pattern\n" }

    my %mon = (
    Jan => 1, Feb => 2, Mar => 3, Apr => 4, May => 5, Jun => 6,
    Jul => 7, Aug => 8, Sep => 9, Oct => 10, Nov => 11, Dec => 12,
    );

    sub date {
    my $date = shift;

    my($d,$m,$y,$hr,$min,$sec);

    # e.g., 03/Feb/1998:17:42:15 -0500
    if (m!(\d+)/(\w+)/(\d+):(\d+):(\d+):(\d+)!) {
    ($d,$m,$y,$hr,$min,$sec) = ($1,$2,$3,$4,$5,$6);

    $m = $mon{$m} || 0;
    }
    else {
    $d = $m = $y = $hr = $min = $sec = 0;
    }

    ($d,$m,$y,$hr,$min,$sec);
    }

    sub date_asc {
    $a->[2] <=> $b->[2] # year
    ||
    $a->[1] <=> $b->[1] # month
    ||
    $a->[0] <=> $b->[0] # day
    ||
    $a->[3] <=> $b->[3] # hour
    ||
    $a->[4] <=> $b->[4] # min
    ||
    $a->[5] <=> $b->[5] # sec
    }

    ## main
    die usage unless @ARGV == 2;

    my $root = shift;
    die "$0: '$root' is not a directory!\n" . usage unless -d $root;

    (my $time = shift) =~ s,/,\\/,g;
    my $pat = eval "qr/" . $time . "/";

    unless (defined $pat) {
    die "$0: bad time pattern\n";
    }

    # from http://stein.cshl.org/WWW/docs/handout.html#Log_Parsing
    my $line = qr/^\S+ \S+ \S+ \[($pat[^]]+)\] "\w+ \S+.*" \d+ \S+/;

    my @hits;
    for (`find $root -name access_log 2>&1`) {
    chomp;

    # assume this line is a warning if it's not a filename
    unless (-f $_) {
    warn $_ . "\n";
    next;
    }

    my $fh;
    unless (open $fh, "<", $_) {
    warn "$0: open $_: $!\n";
    next;
    }

    while (<$fh>) {
    push @hits, [ date($1), $_ ] if /$line/;
    }
    }

    print $_ for map $_->[6], sort date_asc @hits;

    __END__

    Hope this helps,
    Greg
    --
    The logic is flawless: when a private business *accidentally* kills 146
    people, we need to increase the power of the government, an entity that
    *deliberately* kills millions.
    -- Gene Callahan
     
    Greg Bacon, Jul 27, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. danpres2k
    Replies:
    3
    Views:
    7,475
    danpres2k
    Aug 25, 2003
  2. Abhijeet Dharmapurikar

    file grep

    Abhijeet Dharmapurikar, Aug 16, 2007, in forum: Ruby
    Replies:
    6
    Views:
    117
    Alexander Mcconaughey
    May 18, 2009
  3. qanda
    Replies:
    9
    Views:
    163
    qanda
    Sep 11, 2003
  4. qanda
    Replies:
    1
    Views:
    113
    Tassilo v. Parseval
    Sep 15, 2003
  5. Chris L.

    Can I Grep multiple lines??

    Chris L., Feb 10, 2006, in forum: Perl Misc
    Replies:
    16
    Views:
    357
    Anno Siegel
    Feb 13, 2006
Loading...

Share This Page