Sort two dimensional array with multiple keys

M

Milkweed

I am trying sort an multi-dimensional array with three elements in
each "row".

Here's a look at a small section of input data:

207.28.198.86 10.36.1.121 0310291642
207.28.198.86 10.36.1.121 0310291753
207.28.201.113 10.77.2.241 0310291642
207.28.200.113 10.75.2.87 0310291642
207.28.199.86 10.76.1.80 0310291642
207.28.198.104 10.1.3.153 0310291642
207.28.198.86 10.36.1.121 0310291324
207.28.199.104 10.2.2.77 0310291642
207.28.195.33 10.2.4.111 0310291642

These are timestamped NAT translations from a firewall.
What I want to do is sort this data by element 0 (global IP) as the
primary key and element 2 (timestamp) as the secondary key so I can
keep a historical record of NAT translations for a given global IP.

I can't seem to get the sort to work. Here is a copy of what I have
tried so far:

#!/usr/bin/perl
use strict;
use warnings;
use Net::Telnet::Cisco;
use Data::Sorting qw( :basics :arrays );

....

foreach (@xlate) { # @xlate is populated by a SNMP call to the
firewall
next unless /^Global/;
my ($global, $local) = (split /\s/)[1,3];
push @entries, [$global,$local,$timestamp];
# each row of @entries now looks like the sample data above
}

#Sort entries by IP (primary key) and date (secondary key) and print
output
my @ordered = sorted_array( @entries,
{ engine=> 'packed', sortkey=>$_[0][0] },
sortkey=>$_[0][2] );
#my $i = 0;
#keys my %h = @history;
#@h{ sort map pack('C4 A x N' => $->[0] =~
/(\d+)\.(\d+)\.(\d+)\.(\d+)/,
# $_->[2], $i++) => @history } = @history;
#my @ordered = @h{ sort keys %h };
open NEW, ">xlate-ordered"
or die "Could not open xlate-ordered for writing: $!\n";
for my $row (@ordered) {
print NEW "@$row\n";
}

As you can see I have tried using the "indexed sort" method from "A
Fresh Look at Efficient Perl Sorting" as well as the Data::Sorting
module with no success. Help with either option would be much
appreciated!

Thanks,
Chris
 
B

Ben Morrow

I am trying sort an multi-dimensional array with three elements in
each "row".

Here's a look at a small section of input data:

207.28.198.86 10.36.1.121 0310291642
207.28.198.86 10.36.1.121 0310291753
207.28.201.113 10.77.2.241 0310291642
207.28.200.113 10.75.2.87 0310291642
207.28.199.86 10.76.1.80 0310291642
207.28.198.104 10.1.3.153 0310291642
207.28.198.86 10.36.1.121 0310291324
207.28.199.104 10.2.2.77 0310291642
207.28.195.33 10.2.4.111 0310291642

These are timestamped NAT translations from a firewall.
What I want to do is sort this data by element 0 (global IP) as the
primary key and element 2 (timestamp) as the secondary key so I can
keep a historical record of NAT translations for a given global IP.

I can't seem to get the sort to work. Here is a copy of what I have
tried so far:

#!/usr/bin/perl
use strict;
use warnings;
use Net::Telnet::Cisco;
use Data::Sorting qw( :basics :arrays );

...

foreach (@xlate) { # @xlate is populated by a SNMP call to the
firewall
next unless /^Global/;
my ($global, $local) = (split /\s/)[1,3];
push @entries, [$global,$local,$timestamp];
# each row of @entries now looks like the sample data above
}
Click to expand...

Try this (the technique is known as the Schwartzian Transform after
Randal Schwartz):

my @sorted = map { $_->[0] }
sort { $b->[1] cmp $a->[1] }
map { [ $_, $_->[0].$_->[2] ] }
@entries;

This will sort in ASCIIbetical order, first of global IP and then of
timestamp. If you want the IPs in a better order you could use
map { [ $_, inet_aton($_->[0]).$_->[2] ] }
for the first map instead (after using Socket, obviously).

I found out about this from http://perl.plover.com/: a very useful
resource.

Ben
 
A

Anno Siegel

Milkweed said:
I am trying sort an multi-dimensional array with three elements in
each "row".

Here's a look at a small section of input data:

207.28.198.86 10.36.1.121 0310291642
207.28.198.86 10.36.1.121 0310291753
207.28.201.113 10.77.2.241 0310291642
207.28.200.113 10.75.2.87 0310291642
207.28.199.86 10.76.1.80 0310291642
207.28.198.104 10.1.3.153 0310291642
207.28.198.86 10.36.1.121 0310291324
207.28.199.104 10.2.2.77 0310291642
207.28.195.33 10.2.4.111 0310291642

These are timestamped NAT translations from a firewall.
What I want to do is sort this data by element 0 (global IP) as the
primary key and element 2 (timestamp) as the secondary key so I can
keep a historical record of NAT translations for a given global IP.

I can't seem to get the sort to work. Here is a copy of what I have
tried so far:

#!/usr/bin/perl
use strict;
use warnings;
use Net::Telnet::Cisco;
use Data::Sorting qw( :basics :arrays );
Click to expand...

I'm not acquainted with that module, but your sort problem is so
straight-forward, I'd first try a direct solution.
...

foreach (@xlate) { # @xlate is populated by a SNMP call to the
firewall
next unless /^Global/;
my ($global, $local) = (split /\s/)[1,3];
push @entries, [$global,$local,$timestamp];
# each row of @entries now looks like the sample data above
Click to expand...

Unlikely. The variable $timestamp isn't assigned a value.
}

#Sort entries by IP (primary key) and date (secondary key) and print
output
my @ordered = sorted_array( @entries,
{ engine=> 'packed', sortkey=>$_[0][0] },
sortkey=>$_[0][2] );
#my $i = 0;
#keys my %h = @history;
#@h{ sort map pack('C4 A x N' => $->[0] =~
/(\d+)\.(\d+)\.(\d+)\.(\d+)/,
# $_->[2], $i++) => @history } = @history;
#my @ordered = @h{ sort keys %h };
open NEW, ">xlate-ordered"
or die "Could not open xlate-ordered for writing: $!\n";
for my $row (@ordered) {
print NEW "@$row\n";
}

As you can see I have tried using the "indexed sort" method from "A
Fresh Look at Efficient Perl Sorting" as well as the Data::Sorting
module with no success. Help with either option would be much
appreciated!
Click to expand...

As noted, you don't *need* any special methods for this sort. You
may be able to speed things up a bit, but the first step should be
to get it right.

Assume your data is given like this (what it should be, but isn't, after
the loop over @xlate):

my @entries = (
[ qw( 207.28.198.86 10.36.1.121 0310291642)],
[ qw( 207.28.198.86 10.36.1.121 0310291753)],
[ qw( 207.28.201.113 10.77.2.241 0310291642)],
[ qw( 207.28.200.113 10.75.2.87 0310291642)],
[ qw( 207.28.199.86 10.76.1.80 0310291642)],
[ qw( 207.28.198.104 10.1.3.153 0310291642)],
[ qw( 207.28.198.86 10.36.1.121 0310291324)],
[ qw( 207.28.199.104 10.2.2.77 0310291642)],
[ qw( 207.28.195.33 10.2.4.111 0310291642)],
);

For sorting, compare any two IPs as strings. If they are equal,
compare the timestamps as numbers. The result is your sorted list:

my @sorted = sort
{ $a->[ 0] cmp $b->[ 0] or $a->[ 2] <=> $b->[ 2] }
@entries;

print "@$_\n" for @sorted;

Anno
 
M

Milkweed

Unlikely. The variable $timestamp isn't assigned a value.

I forgot to mention the $timestamp variable is assigned using the
localtime function further up in the code. Sorry for not explaining
this in my previous post.

Assume your data is given like this (what it should be, but isn't, after
the loop over @xlate):
Click to expand...

This is what is looks like.
my @entries = (
[ qw( 207.28.198.86 10.36.1.121 0310291642)],
[ qw( 207.28.198.86 10.36.1.121 0310291753)],
[ qw( 207.28.201.113 10.77.2.241 0310291642)],
[ qw( 207.28.200.113 10.75.2.87 0310291642)],
[ qw( 207.28.199.86 10.76.1.80 0310291642)],
[ qw( 207.28.198.104 10.1.3.153 0310291642)],
[ qw( 207.28.198.86 10.36.1.121 0310291324)],
[ qw( 207.28.199.104 10.2.2.77 0310291642)],
[ qw( 207.28.195.33 10.2.4.111 0310291642)],
);

For sorting, compare any two IPs as strings. If they are equal,
compare the timestamps as numbers. The result is your sorted list:

my @sorted = sort
{ $a->[ 0] cmp $b->[ 0] or $a->[ 2] <=> $b->[ 2] }
@entries;

print "@$_\n" for @sorted;

Anno
Click to expand...

The above code works. Thanks! I'm worried about how fast the two
subsorts will perform. I could potentially have hundreds of thousands
of entries, which is why I was trying to do something with an indexed
search in the first place. Thanks again for your help.

Chris
 
M

Milkweed

Try this (the technique is known as the Schwartzian Transform after
Randal Schwartz):

my @sorted = map { $_->[0] }
sort { $b->[1] cmp $a->[1] }
map { [ $_, $_->[0].$_->[2] ] }
@entries;

This will sort in ASCIIbetical order, first of global IP and then of
timestamp. If you want the IPs in a better order you could use
map { [ $_, inet_aton($_->[0]).$_->[2] ] }
for the first map instead (after using Socket, obviously).
Click to expand...

When I try this:
my @sorted =
map { $_->[0] }
sort { $b->[1] cmp $a->[1] }
map { $_, inet_aton($_->[0]).$_->[2] }
@entries;

I get the following error:

Can't use string ("ÏÄÈ0310301117") as an ARRAY ref while "strict refs"
in use at ./NAT.pl line 73.

Any idea how to get past this?

Thanks,
Chris
 
G

Greg Bacon

: When I try this:
: my @sorted =
: map { $_->[0] }
: sort { $b->[1] cmp $a->[1] }
: map { $_, inet_aton($_->[0]).$_->[2] }
: @entries;
:
: I get the following error:
:
: Can't use string ("ÏÄÈ0310301117") as an ARRAY ref while "strict refs"
: in use at ./NAT.pl line 73.

Your first map should produce a list of array references:

...
map { [ $_, inet_aton($_->[0]) . $_->[2] ] }
@entries;

Greg
 
M

Milkweed

Try this (the technique is known as the Schwartzian Transform after
Randal Schwartz):

my @sorted = map { $_->[0] }
sort { $b->[1] cmp $a->[1] }
map { [ $_, $_->[0].$_->[2] ] }
@entries;

This will sort in ASCIIbetical order, first of global IP and then of
timestamp. If you want the IPs in a better order you could use
map { [ $_, inet_aton($_->[0]).$_->[2] ] }
for the first map instead (after using Socket, obviously).

I found out about this from http://perl.plover.com/: a very useful
resource.

Ben
Click to expand...

Forget my last reply. I'm a dork. I forgot to put the outside [ ] in
the first map statement. Once I put those in it worked perfectly.
Thanks for your help!

Chris
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Sort by number of characters 1
Sort by number of characters 0
Some sort questions - especially hashes 4
FAQ 4.60 How do I sort a hash (optionally by value instead of key)? 0
Comparing values of multiple hash keys 8
FAQ 4.52 How do I sort an array by (anything)? 0
problem with hash & sort array 7
Error 'this.$event[...].keys' is 5

Members online

No members online now.
Total: 39 (members: 0, guests: 39)
Robots: 198

Forum statistics

Threads
473,763
Messages
2,569,562
Members
45,038
Latest member
OrderProperKetocapsules

Latest Threads

Top