Sorting non-compresses Unix logs

[sunckell]

Greetings,
I am having a little difficulty sorting unix non-compressed logs
files that are in an array.

The log files ( as usual ) end in a number ranging from nothing (such
as mail.debug) to 30 (such as mail.debug.30).

My sort is mostly working ( thanks to the help of perldoc -f sort )
but it spits a warning because my first log file doesn't have a
numberical ending.

Warning message: Use of uninitialized value in numeric comparison (<=>)
at
$script line number...

Can anyone offer a better way to sort the array of log files?

-------------------------------------------------------------------------------------------
use strict;
use warnings;
use sort 'stable';
use diagnostics;
use POSIX qw(fstat);
use Sys::Hostname;
$|++;

use vars qw(
$log_dir $hostname $maillog_dir

@log_files @mail_logs_unsorted @mail_logs_sorted
);

$0 = $1 if $0 =~ m:/([^/]+)$:;
$hostname = hostname();
$maillog_dir = '/var/log';

# get list of log_files in $log_dir
$log_dir = POSIX:pendir( $maillog_dir );
@log_files = POSIX::readdir( $log_dir );
POSIX::closedir( $log_dir );

#cut the list down to only mail logs
foreach my $log ( @log_files ){
if($log =~ /^mail\.debug/){
push( @mail_logs_unsorted, "$log" );
}
}
undef @log_files; #don't need that anymore

@mail_logs_sorted = sort { substr($a, 11) <=> substr($b, 11) }
@mail_logs_unsorted;

foreach my $log ( @mail_logs_sorted ) {
print "$log\n";
}


Thanks for the help,
SunCkell

 

[Paul Lalli]

I am having a little difficulty sorting unix non-compressed logs
files that are in an array.

The log files ( as usual ) end in a number ranging from nothing (such
as mail.debug) to 30 (such as mail.debug.30).

My sort is mostly working ( thanks to the help of perldoc -f sort )
but it spits a warning because my first log file doesn't have a
numberical ending.

Warning message: Use of uninitialized value in numeric comparison (<=>)
at
$script line number...

Can anyone offer a better way to sort the array of log files?

-------------------------------------------------------------------------------------------
use strict;
use warnings;
use sort 'stable';
use diagnostics;
use POSIX qw(fstat);
use Sys::Hostname;
$|++;

use vars qw(
$log_dir $hostname $maillog_dir

@log_files @mail_logs_unsorted @mail_logs_sorted
);

Why are you using global variables? And why are you using the old
style of declaring global variables? If you have a Perl recent enough
to use warnings.pm, you should replace all instances of
use vars qw(...);
with
our ...;

However, there is no reason to make any of these globals. Make them
lexical (using my), and declare them in the smallest scope possible
(ie, as you need them, not all at once).

$0 = $1 if $0 =~ m:/([^/]+)$:;

Yeesh. perldoc File::Basename

$hostname = hostname();
$maillog_dir = '/var/log';

# get list of log_files in $log_dir
$log_dir = POSIX:pendir( $maillog_dir );
@log_files = POSIX::readdir( $log_dir );
POSIX::closedir( $log_dir );

Why are you using the POSIX equivalents? These are built-in perl
functions. Also, always check the return value of all system calls:

opendir my $log_dh, $maillog_dir or die "Could not open directory
$maillog_dir: $!";
my @log_files = readdir($log_dh);
closedir $log_dh or die "Could not close directory $maillog_dir: $!";

#cut the list down to only mail logs
foreach my $log ( @log_files ){
if($log =~ /^mail\.debug/){
push( @mail_logs_unsorted, "$log" );
}
}

Learn to love grep:
my @mail_logs_unsorted = grep /^mail\.debug/ @log_files;

undef @log_files; #don't need that anymore

You didn't *need* it to begin with:

my @mail_logs_unsorted = grep /^mail\.debug/ readdir($log_dh);

@mail_logs_sorted = sort { substr($a, 11) <=> substr($b, 11) }
@mail_logs_unsorted;

Gahhh. Horribly inefficient. Use a Schwartzian Transform so you only
have to do the substr() once per filename:

my @mail_logs_sorted = map { $_->[0] }
sort { $a->[1] <=> $b->[1] }
map { [ $_, substr($_, 11) ] };


Now, if you want to get rid of the warning, I see two options:
(1) Specifically turn off the warning in the sort subroutine:
sort { no warnings 'numeric'; $a->[1] <=> $b->[1] }
(2) in the initial map, store a 0 instead of undef if the file is not
11 chars long:
map { [$_, substr($_, 11) || 0 ] }

Paul Lalli

 

[Gunnar Hjalmarsson]

I am having a little difficulty sorting unix non-compressed logs
files that are in an array.

The log files ( as usual ) end in a number ranging from nothing (such
as mail.debug) to 30 (such as mail.debug.30).

My sort is mostly working ( thanks to the help of perldoc -f sort )
but it spits a warning because my first log file doesn't have a
numberical ending.

Warning message: Use of uninitialized value in numeric comparison (<=>)
at
$script line number...

If you know that's the reason, disable warnings like this:

{
no warnings 'uninitialized';
@mail_logs_sorted = sort { substr($a, 11) <=> substr($b, 11) }
@mail_logs_unsorted;
}