Sorting non-compresses Unix logs

Discussion in 'Perl Misc' started by sunckell, Jan 3, 2006.

  1. sunckell

    sunckell Guest

    Greetings,
    I am having a little difficulty sorting unix non-compressed logs
    files that are in an array.

    The log files ( as usual ) end in a number ranging from nothing (such
    as mail.debug) to 30 (such as mail.debug.30).

    My sort is mostly working ( thanks to the help of perldoc -f sort )
    but it spits a warning because my first log file doesn't have a
    numberical ending.

    Warning message: Use of uninitialized value in numeric comparison (<=>)
    at
    $script line number...

    Can anyone offer a better way to sort the array of log files?

    -------------------------------------------------------------------------------------------
    use strict;
    use warnings;
    use sort 'stable';
    use diagnostics;
    use POSIX qw(fstat);
    use Sys::Hostname;
    $|++;

    use vars qw(
    $log_dir $hostname $maillog_dir

    @log_files @mail_logs_unsorted @mail_logs_sorted
    );

    $0 = $1 if $0 =~ m:/([^/]+)$:;
    $hostname = hostname();
    $maillog_dir = '/var/log';

    # get list of log_files in $log_dir
    $log_dir = POSIX::eek:pendir( $maillog_dir );
    @log_files = POSIX::readdir( $log_dir );
    POSIX::closedir( $log_dir );

    #cut the list down to only mail logs
    foreach my $log ( @log_files ){
    if($log =~ /^mail\.debug/){
    push( @mail_logs_unsorted, "$log" );
    }
    }
    undef @log_files; #don't need that anymore

    @mail_logs_sorted = sort { substr($a, 11) <=> substr($b, 11) }
    @mail_logs_unsorted;

    foreach my $log ( @mail_logs_sorted ) {
    print "$log\n";
    }


    Thanks for the help,
    SunCkell
    sunckell, Jan 3, 2006
    #1
    1. Advertising

  2. sunckell

    Paul Lalli Guest

    sunckell wrote:
    > I am having a little difficulty sorting unix non-compressed logs
    > files that are in an array.
    >
    > The log files ( as usual ) end in a number ranging from nothing (such
    > as mail.debug) to 30 (such as mail.debug.30).
    >
    > My sort is mostly working ( thanks to the help of perldoc -f sort )
    > but it spits a warning because my first log file doesn't have a
    > numberical ending.
    >
    > Warning message: Use of uninitialized value in numeric comparison (<=>)
    > at
    > $script line number...
    >
    > Can anyone offer a better way to sort the array of log files?
    >
    > -------------------------------------------------------------------------------------------
    > use strict;
    > use warnings;
    > use sort 'stable';
    > use diagnostics;
    > use POSIX qw(fstat);
    > use Sys::Hostname;
    > $|++;
    >
    > use vars qw(
    > $log_dir $hostname $maillog_dir
    >
    > @log_files @mail_logs_unsorted @mail_logs_sorted
    > );


    Why are you using global variables? And why are you using the old
    style of declaring global variables? If you have a Perl recent enough
    to use warnings.pm, you should replace all instances of
    use vars qw(...);
    with
    our ...;

    However, there is no reason to make any of these globals. Make them
    lexical (using my), and declare them in the smallest scope possible
    (ie, as you need them, not all at once).

    > $0 = $1 if $0 =~ m:/([^/]+)$:;


    Yeesh. perldoc File::Basename

    > $hostname = hostname();
    > $maillog_dir = '/var/log';
    >
    > # get list of log_files in $log_dir
    > $log_dir = POSIX::eek:pendir( $maillog_dir );
    > @log_files = POSIX::readdir( $log_dir );
    > POSIX::closedir( $log_dir );


    Why are you using the POSIX equivalents? These are built-in perl
    functions. Also, always check the return value of all system calls:

    opendir my $log_dh, $maillog_dir or die "Could not open directory
    $maillog_dir: $!";
    my @log_files = readdir($log_dh);
    closedir $log_dh or die "Could not close directory $maillog_dir: $!";

    > #cut the list down to only mail logs
    > foreach my $log ( @log_files ){
    > if($log =~ /^mail\.debug/){
    > push( @mail_logs_unsorted, "$log" );
    > }
    > }


    Learn to love grep:
    my @mail_logs_unsorted = grep /^mail\.debug/ @log_files;

    > undef @log_files; #don't need that anymore


    You didn't *need* it to begin with:

    my @mail_logs_unsorted = grep /^mail\.debug/ readdir($log_dh);

    > @mail_logs_sorted = sort { substr($a, 11) <=> substr($b, 11) }
    > @mail_logs_unsorted;


    Gahhh. Horribly inefficient. Use a Schwartzian Transform so you only
    have to do the substr() once per filename:

    my @mail_logs_sorted = map { $_->[0] }
    sort { $a->[1] <=> $b->[1] }
    map { [ $_, substr($_, 11) ] };


    Now, if you want to get rid of the warning, I see two options:
    (1) Specifically turn off the warning in the sort subroutine:
    sort { no warnings 'numeric'; $a->[1] <=> $b->[1] }
    (2) in the initial map, store a 0 instead of undef if the file is not
    11 chars long:
    map { [$_, substr($_, 11) || 0 ] }

    Paul Lalli
    Paul Lalli, Jan 3, 2006
    #2
    1. Advertising

  3. sunckell wrote:
    > I am having a little difficulty sorting unix non-compressed logs
    > files that are in an array.
    >
    > The log files ( as usual ) end in a number ranging from nothing (such
    > as mail.debug) to 30 (such as mail.debug.30).
    >
    > My sort is mostly working ( thanks to the help of perldoc -f sort )
    > but it spits a warning because my first log file doesn't have a
    > numberical ending.
    >
    > Warning message: Use of uninitialized value in numeric comparison (<=>)
    > at
    > $script line number...


    If you know that's the reason, disable warnings like this:

    {
    no warnings 'uninitialized';
    @mail_logs_sorted = sort { substr($a, 11) <=> substr($b, 11) }
    @mail_logs_unsorted;
    }

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
    Gunnar Hjalmarsson, Jan 3, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    12
    Views:
    1,639
    Dave Thompson
    Jan 10, 2005
  2. Sorting Unix mailboxes

    , Sep 13, 2005, in forum: Python
    Replies:
    3
    Views:
    368
    Gregory K. Johnson
    Sep 16, 2005
  3. Replies:
    18
    Views:
    619
    Dave Thompson
    Jan 10, 2005
  4. Robert Wallace

    my own perl "dos->unix"/"unix->dos"

    Robert Wallace, Jan 21, 2004, in forum: Perl Misc
    Replies:
    7
    Views:
    277
    Michele Dondi
    Jan 22, 2004
  5. a cgi for view unix logs?

    , Feb 13, 2007, in forum: Perl Misc
    Replies:
    6
    Views:
    478
    Andrew DeFaria
    Feb 13, 2007
Loading...

Share This Page