Perl DNS reverse lookups -- multiple IP addresses per line

[Maynard]

Hello all,

I'm attempting to take a log file with several thousand IP addresses
and convert them into their DNS named equivalent -- Not so hard. But
what's stumping me is some basic regex syntax. I know that the 'g'
operator should allow me to continue from where I left off, but I seem
to be missing the correct way to implement this... my script is
picking up the first IP address in the line, but not the second.
Perhaps someone can show me the error of my ways and get me back on
track.

Input log file is formatted like this:

TCP out 65.198.222.70:43679 in 192.168.0.3:80 idle 0:00:08
TCP out 209.73.24.2:29042 in 192.168.0.3:80 idle 0:00:08
TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00
....


Complete script:
----------------------------------

use Socket;

my ($ip, $hostname);
my ($SourceFile, $DestFile) = ("log1.txt", "log2.txt");

open(INPUTFILE, "<$SourceFile") || die("ERROR: Cannot open
$SourceFile, $!");
open(OUTPUTFILE, ">$DestFile") || die("ERROR: Cannot open $DestFile,
$!");

binmode INPUTFILE;
binmode OUTPUTFILE;

while(<INPUTFILE>) {
my $line = $_;

if ($line =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {

$ip = $1;
$hostname = lc gethostbyaddr(inet_aton($ip),AF_INET);

if ($hostname ne "") {
$line =~ s/$g_ip/$g_hostname/;
print " IP address $ip = $hostname\n";
}
else {
print " IP address $ip = *NO DNS RECORD FOUND*\n";
}
}

print OUTPUTFILE $line;
}

close(INPUTFILE);
close(OUTPUTFILE);

 

[Sam Holden]

Hello all,

I'm attempting to take a log file with several thousand IP addresses
and convert them into their DNS named equivalent -- Not so hard. But
what's stumping me is some basic regex syntax. I know that the 'g'
operator should allow me to continue from where I left off, but I seem
to be missing the correct way to implement this... my script is
picking up the first IP address in the line, but not the second.
Perhaps someone can show me the error of my ways and get me back on
track.

Input log file is formatted like this:

TCP out 65.198.222.70:43679 in 192.168.0.3:80 idle 0:00:08
TCP out 209.73.24.2:29042 in 192.168.0.3:80 idle 0:00:08
TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00
...


Complete script:
----------------------------------

use Socket;

my ($ip, $hostname);
my ($SourceFile, $DestFile) = ("log1.txt", "log2.txt");

open(INPUTFILE, "<$SourceFile") || die("ERROR: Cannot open
$SourceFile, $!");
open(OUTPUTFILE, ">$DestFile") || die("ERROR: Cannot open $DestFile,
$!");

binmode INPUTFILE;
binmode OUTPUTFILE;

while(<INPUTFILE>) {
my $line = $_;

if ($line =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {

You need to loop for the g to have any effect. so s/if/while/.

However, you do a s// on $line inside the body - I don't know if that will
break the /g pickup point or not. Read the documentation and check I guess.

Or you could do something like:

while (<INPUTFILE>) {
s/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/
lc gethostbyaddr(inet_aton($1),AF_INET)||$1/ge;
print OUTPUTFILE;
}

 

[John W. Krahn]

I'm attempting to take a log file with several thousand IP addresses
and convert them into their DNS named equivalent -- Not so hard. But
what's stumping me is some basic regex syntax. I know that the 'g'
operator should allow me to continue from where I left off, but I seem
to be missing the correct way to implement this... my script is
picking up the first IP address in the line, but not the second.
Perhaps someone can show me the error of my ways and get me back on
track.

Input log file is formatted like this:

TCP out 65.198.222.70:43679 in 192.168.0.3:80 idle 0:00:08 ^^^^^^^^^^^
TCP out 209.73.24.2:29042 in 192.168.0.3:80 idle 0:00:08 ^^^^^^^^^^^
TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00

^^^^^^^^^^^
It is not going to be very useful to do a DNS lookup on 192.168/16
addresses. See RFC1918 for the reason why.

use Socket;

my ($ip, $hostname);
my ($SourceFile, $DestFile) = ("log1.txt", "log2.txt");

open(INPUTFILE, "<$SourceFile") || die("ERROR: Cannot open
$SourceFile, $!");
open(OUTPUTFILE, ">$DestFile") || die("ERROR: Cannot open $DestFile,
$!");

binmode INPUTFILE;
binmode OUTPUTFILE;

while(<INPUTFILE>) {
my $line = $_;

if ($line =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {
$ip = $1;

To capture all IP addresses in the line you could use a for loop:

for my $ip ( $line =~ /\d{1,3}(?:\.\d{1,3}){3}/g ) {

Or a while loop:

while ( $line =~ /(\d{1,3}(?:\.\d{1,3}){3})/g ) {
my $ip = $1;

Or assign them to an array:

my @ips = $line =~ /\d{1,3}(?:\.\d{1,3}){3}/g;

$hostname = lc gethostbyaddr(inet_aton($ip),AF_INET);

If the string in $ip is not a valid IP address then inet_aton() will
return undef. Perhaps you should test for that?

if ($hostname ne "") {

It looks like you don't have warnings enabled as that will warn "Use of
uninitialized value in string ne" when gethostbyaddr() can't find a
hostname.

$line =~ s/$g_ip/$g_hostname/;

Where did the variables $g_ip and $g_hostname come from?

print " IP address $ip = $hostname\n";
}
else {
print " IP address $ip = *NO DNS RECORD FOUND*\n";
}
}

print OUTPUTFILE $line;
}

close(INPUTFILE);
close(OUTPUTFILE);

John

 

[Petri]

Maynard wrote:
^^^^^^^^^^^
It is not going to be very useful to do a DNS lookup on
192.168/16 addresses. See RFC1918 for the reason why.

It may be useful at the OP's network.
He might be running split-DNS, with an internal reverse zone for
0.0.168.192.in-addr.arpa.
In that case he would catch names of internal systems, instead of their
IP-addresses.


Petri

 

[Vetle Roeim]

Which is true in my case. Those get translated to host names on my
network, which is always nice.

Thanks everyone for your responses and suggestions; I took some of
them and implemented a couple extras... like a hash file that "caches"
names/IPs, so that only new ones get looked up (saves tons of time
when there's a 2 second delay when a host doesn't have a DNS record),
and just for you John, I added a switch that skips non-routable IP
addrs, just in case you want to do that.

For maximum speed I recommend using Net:NS::Resolver and the
bgsend method, to send multiple requests before waiting for
answers, and then using IO::Select to read the answers as they
become available.

Incidentally, there's already a script that does this:
<URL: http://jdrowell.com/projects/jdresolve >

It may not be neccessary to do it like this, but that depends on
the amount of data. I've created a script that has to resolve
millions of IP addresses in a reasonable amount of time, and
Net:NS::Resolver in combination with IO::Select seems to be
the way to go.


[...]