Novice::Help>Split IP Address(port) + count

[RobO]

Greetings,

Anybody willing to lend me a helping hand?

I am trying to split IP addresses from ports and count the destination
ports with the most hits and source IP addresses with the most hits
from log files.
For example the line below shows a log entry:

##################
#2005-06-13 00:19:03 Local5.Info 10.3.1.1 %SEC-6-IPACCESSLOGP: list
INGRESS1 denied udp 202.97.175.52(4864) -> xxx.xxx.xxx.xxx(1434), 1
packet
##################

As you can see port 1434/udp is being logged as destination port, is it
possible(sure it is) to count the matching ports and return the most
hit ports in the logfile?

The log file is tab seperated and I can count the matching IP addresses
when it is an ICMP log and the IP addresses are on their own, as below.

##################
#2005-06-13 00:17:27 Local5.Info 10.3.1.1 %SEC-6-IPACCESSLOGDP: list
INGRESS1 denied icmp 81.139.2.31 -> xxx.xxx.xxx.xxx (3/13), 1 packet
##################

But with the IP_ADDRESS(port) log entry I am stupified as to how to
separate them and count.
Hope this makes sense!
Any help will be greatly appreciated and thanks in advance.

Rob

 

[Jim Keenan]

I am trying to split IP addresses from ports and count the destination
ports with the most hits and source IP addresses with the most hits
from log files.
For example the line below shows a log entry:

##################
#2005-06-13 00:19:03 Local5.Info 10.3.1.1 %SEC-6-IPACCESSLOGP: list
INGRESS1 denied udp 202.97.175.52(4864) -> xxx.xxx.xxx.xxx(1434), 1
packet
##################

As you can see port 1434/udp is being logged as destination port, is it
possible(sure it is) to count the matching ports and return the most
hit ports in the logfile?

The log file is tab seperated and I can count the matching IP addresses
when it is an ICMP log and the IP addresses are on their own, as below.

##################
#2005-06-13 00:17:27 Local5.Info 10.3.1.1 %SEC-6-IPACCESSLOGDP: list
INGRESS1 denied icmp 81.139.2.31 -> xxx.xxx.xxx.xxx (3/13), 1 packet
##################

But with the IP_ADDRESS(port) log entry I am stupified as to how to
separate them and count.

AFAICT, the only difference between the two critical lines is that the
top one has two data within parentheses while the bottom one has only
one. Since you imply that you have successfully analyzed the bottom
line, you should be able to modify the code you used to do that without
much trouble. Could you please show that code so we can suggest how to
modify it? Thanks.

jimk

 

[Paul Lalli]

I am trying to split IP addresses from ports and count the destination
ports with the most hits and source IP addresses with the most hits
from log files.
For example the line below shows a log entry:

##################
#2005-06-13 00:19:03 Local5.Info 10.3.1.1 %SEC-6-IPACCESSLOGP: list
INGRESS1 denied udp 202.97.175.52(4864) -> xxx.xxx.xxx.xxx(1434), 1
packet
##################

As you can see port 1434/udp is being logged as destination port, is it
possible(sure it is) to count the matching ports and return the most
hit ports in the logfile?

But with the IP_ADDRESS(port) log entry I am stupified as to how to
separate them and count.

What have you tried so far? It seems like a fairly easy task for a
regular expression to parse out the port number. You could then store
each port number as the key to a hash, incrementing the associated hash
value for each port number found.

perldoc perlrequick
perldoc perlretut
perldoc perlre

Let us know what problems you're encountering with your attempt, and we
can help you fix your code.

Paul Lalli

 

[RobO]

Jim,

Thanks!

I have been messing around with this so some lines are commented at
this time of posting so its probably very messy and possibly very
wrong.....

I have a feeling that I'm not splitting the lines correctly or I need
to split them seperately possibly in different "if" statements.???
As mentioned an ICMP count is fine I get the results I want but when
its a UDP count as in the other log line the format is
ip_address(port).

I have only been able to analyze the IP addresses but not anything
within parenthesis.

###start###newbie-dont laugh
open (INFILE, $log_file) || die "Could not open $log_file";
foreach (<INFILE>) {
chomp($log_file);
($date_stmp, $time_stmp, $facility, $logsrc, $loglevel, $restof) =
split (/\s+/, $_, 6);
($left, $right) = split (/\-\>/, $restof, 2);
($list, $listname, $action, $protocol, $src_ip) = split (/\s+/, $left,
5);
($dest_ip, $icmp_type, $packets) = split (/\,/, $right, 3);
# ($src_ip, $dest_ip) = split (/[()]/, $_, 2);

if ($protocol =~ /icmp/) {
$icmp_hits++;
$_ = $src_ip;
/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/;
$icmp_ip_count{$src_ip}++;
}
# if ($src_ip =~ /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/) {
# $denied_hits++;
# $_ = $src_ip;
# (/\(\d{1,5}\)/);
# /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/;
# $deny_ip_count{$src_ip}++;
# }
close (INFILE);


sub icmp_count_ip_byval { $icmp_ip_count{$b}-$icmp_ip_count{$a}; }
$icmp_ip_count = 0;
foreach (sort icmp_count_ip_byval keys %icmp_ip_count) {
# if ($icmp_ip_count{$_} > 1) {
print "$icmp_ip_count{$_}\t$_\n";
# }

sub deny_count_ip_byval { $deny_ip_count{$b}-$deny_ip_count{$a}; }
$deny_ip_count = 0;
foreach (sort deny_count_ip_byval keys %deny_ip_count) {
print "$_\n";
}
}
###end###

Cheers,

Rob

 

[Jim Keenan]

[snip]
I have a feeling that I'm not splitting the lines correctly or I need
to split them seperately possibly in different "if" statements.???
As mentioned an ICMP count is fine I get the results I want but when
its a UDP count as in the other log line the format is
ip_address(port).

I have only been able to analyze the IP addresses but not anything
within parenthesis.

###start###newbie-dont laugh
open (INFILE, $log_file) || die "Could not open $log_file";
foreach (<INFILE>) {
chomp($log_file);
($date_stmp, $time_stmp, $facility, $logsrc, $loglevel, $restof) =
split (/\s+/, $_, 6);
($left, $right) = split (/\-\>/, $restof, 2);
($list, $listname, $action, $protocol, $src_ip) = split (/\s+/, $left,
5);
($dest_ip, $icmp_type, $packets) = split (/\,/, $right, 3);
# ($src_ip, $dest_ip) = split (/[()]/, $_, 2);

if ($protocol =~ /icmp/) {
$icmp_hits++;
$_ = $src_ip;
/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/;
$icmp_ip_count{$src_ip}++;
}
# if ($src_ip =~ /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/) {
# $denied_hits++;
# $_ = $src_ip;
# (/\(\d{1,5}\)/);
# /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/;
# $deny_ip_count{$src_ip}++;
# }
close (INFILE);


sub icmp_count_ip_byval { $icmp_ip_count{$b}-$icmp_ip_count{$a}; }
$icmp_ip_count = 0;
foreach (sort icmp_count_ip_byval keys %icmp_ip_count) {
# if ($icmp_ip_count{$_} > 1) {
print "$icmp_ip_count{$_}\t$_\n";
# }

sub deny_count_ip_byval { $deny_ip_count{$b}-$deny_ip_count{$a}; }
$deny_ip_count = 0;
foreach (sort deny_count_ip_byval keys %deny_ip_count) {
print "$_\n";
}
}
###end###

You need to focus your analysis more, for two reasons.

The less important of the two is that, when posting on this or any
other list, you will elicit more offers of assistance if the people
reading your post don't have to wade through irrelevant material to get
to the heart of the problem. (Cf the Posting Guidelines).

The more important is that the *very process of focusing your analysis*
will often help you diagnose the problem yourself.

The problem you originally described has to do with parsing a line in a
log file: What is the best combination of 'split' and regexes to do
so? So far, so good.

But the code you posted starts with opening and reading the file --
which is not your stated problem. You also provide a lot of code
dealing with printing the results of your line parsing to STDOUT --
again, not your stated problem.

My hunch is that your problem lies in this area:

($date_stmp, $time_stmp, $facility, $logsrc, $loglevel, $restof) =
split (/\s+/, $_, 6);
($left, $right) = split (/\-\>/, $restof, 2);
($list, $listname, $action, $protocol, $src_ip) = split (/\s+/, $left,
5);
($dest_ip, $icmp_type, $packets) = split (/\,/, $right, 3);

Can you narrow the focus further? In particular, figure out what you
need to do with this line:

INGRESS1 denied udp 202.97.175.52(4864) -> xxx.xxx.xxx.xxx(1434), 1
packet

HTH

jimk

 

[John W. Krahn]

I have been messing around with this so some lines are commented at
this time of posting so its probably very messy and possibly very
wrong.....

I have a feeling that I'm not splitting the lines correctly or I need
to split them seperately possibly in different "if" statements.???
As mentioned an ICMP count is fine I get the results I want but when
its a UDP count as in the other log line the format is
ip_address(port).

I have only been able to analyze the IP addresses but not anything
within parenthesis.

###start###newbie-dont laugh

use warnings;
use strict;

open (INFILE, $log_file) || die "Could not open $log_file";

You should include the $! variable in the error message so that you know why
it died.

open( INFILE, $log_file ) || die "Could not open $log_file: $!";

foreach (<INFILE>) {

There is no need to slurp the entire file into memory as you are only working
with one line at a time.

chomp($log_file);

You are chomp()ing the _file name_, you want to chomp() the current line which
is in the $_ variable.

chomp;

($date_stmp, $time_stmp, $facility, $logsrc, $loglevel, $restof) =
split (/\s+/, $_, 6);
($left, $right) = split (/\-\>/, $restof, 2);
($list, $listname, $action, $protocol, $src_ip) = split (/\s+/, $left,
5);
($dest_ip, $icmp_type, $packets) = split (/\,/, $right, 3);
# ($src_ip, $dest_ip) = split (/[()]/, $_, 2);

Do you really need all those variables? It looks like you only use $protocol
and $src_ip?

if ($protocol =~ /icmp/) {

Do you really need a regular expression for that?

if ( $protocol eq 'icmp' ) {

$icmp_hits++;
$_ = $src_ip;
Why??

/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/;

You are doing a match in void context because?? This has no effect at all on
whether the next line works or not.

$icmp_ip_count{$src_ip}++;
}
# if ($src_ip =~ /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/) {
# $denied_hits++;
# $_ = $src_ip;
# (/\(\d{1,5}\)/);
# /\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/;
# $deny_ip_count{$src_ip}++;
# }

It looks like you are only working with four fields so:

my %stats;
while ( <INFILE> ) {
my ( $proto, $source, $dest, $port ) =
/(icmp|udp|tcp)\s+(\S+)\s+->\s+(\S+)[\s(]+([^)]+/ or next;
$stats{ $proto }++;
$stats{ $source }++;
$stats{ $dest }++;
$stats{ $port }++;
}

close (INFILE);


sub icmp_count_ip_byval { $icmp_ip_count{$b}-$icmp_ip_count{$a}; }

Are you trying to save characters by using - instead of <=>? Just hope that
your numbers don't get large enough to overflow.



John