Perl DNS reverse lookups -- multiple IP addresses per line

Discussion in 'Perl Misc' started by Maynard, Jun 21, 2004.

  1. Maynard

    Maynard Guest

    Hello all,

    I'm attempting to take a log file with several thousand IP addresses
    and convert them into their DNS named equivalent -- Not so hard. But
    what's stumping me is some basic regex syntax. I know that the 'g'
    operator should allow me to continue from where I left off, but I seem
    to be missing the correct way to implement this... my script is
    picking up the first IP address in the line, but not the second.
    Perhaps someone can show me the error of my ways and get me back on
    track.

    Input log file is formatted like this:

    TCP out 65.198.222.70:43679 in 192.168.0.3:80 idle 0:00:08
    TCP out 209.73.24.2:29042 in 192.168.0.3:80 idle 0:00:08
    TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00
    ....


    Complete script:
    ----------------------------------

    use Socket;

    my ($ip, $hostname);
    my ($SourceFile, $DestFile) = ("log1.txt", "log2.txt");

    open(INPUTFILE, "<$SourceFile") || die("ERROR: Cannot open
    $SourceFile, $!");
    open(OUTPUTFILE, ">$DestFile") || die("ERROR: Cannot open $DestFile,
    $!");

    binmode INPUTFILE;
    binmode OUTPUTFILE;

    while(<INPUTFILE>) {
    my $line = $_;

    if ($line =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {

    $ip = $1;
    $hostname = lc gethostbyaddr(inet_aton($ip),AF_INET);

    if ($hostname ne "") {
    $line =~ s/$g_ip/$g_hostname/;
    print " IP address $ip = $hostname\n";
    }
    else {
    print " IP address $ip = *NO DNS RECORD FOUND*\n";
    }
    }

    print OUTPUTFILE $line;
    }

    close(INPUTFILE);
    close(OUTPUTFILE);

    ----------------------------------


    Thanks very much for any help.
     
    Maynard, Jun 21, 2004
    #1
    1. Advertising

  2. Maynard

    Sam Holden Guest

    On 20 Jun 2004 22:33:56 -0700, Maynard <> wrote:
    > Hello all,
    >
    > I'm attempting to take a log file with several thousand IP addresses
    > and convert them into their DNS named equivalent -- Not so hard. But
    > what's stumping me is some basic regex syntax. I know that the 'g'
    > operator should allow me to continue from where I left off, but I seem
    > to be missing the correct way to implement this... my script is
    > picking up the first IP address in the line, but not the second.
    > Perhaps someone can show me the error of my ways and get me back on
    > track.
    >
    > Input log file is formatted like this:
    >
    > TCP out 65.198.222.70:43679 in 192.168.0.3:80 idle 0:00:08
    > TCP out 209.73.24.2:29042 in 192.168.0.3:80 idle 0:00:08
    > TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00
    > ...
    >
    >
    > Complete script:
    > ----------------------------------
    >
    > use Socket;
    >
    > my ($ip, $hostname);
    > my ($SourceFile, $DestFile) = ("log1.txt", "log2.txt");
    >
    > open(INPUTFILE, "<$SourceFile") || die("ERROR: Cannot open
    > $SourceFile, $!");
    > open(OUTPUTFILE, ">$DestFile") || die("ERROR: Cannot open $DestFile,
    > $!");
    >
    > binmode INPUTFILE;
    > binmode OUTPUTFILE;
    >
    > while(<INPUTFILE>) {
    > my $line = $_;
    >
    > if ($line =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {


    You need to loop for the g to have any effect. so s/if/while/.

    However, you do a s// on $line inside the body - I don't know if that will
    break the /g pickup point or not. Read the documentation and check I guess.

    Or you could do something like:

    while (<INPUTFILE>) {
    s/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/
    lc gethostbyaddr(inet_aton($1),AF_INET)||$1/ge;
    print OUTPUTFILE;
    }

    --
    Sam Holden
     
    Sam Holden, Jun 21, 2004
    #2
    1. Advertising

  3. Maynard wrote:
    >
    > I'm attempting to take a log file with several thousand IP addresses
    > and convert them into their DNS named equivalent -- Not so hard. But
    > what's stumping me is some basic regex syntax. I know that the 'g'
    > operator should allow me to continue from where I left off, but I seem
    > to be missing the correct way to implement this... my script is
    > picking up the first IP address in the line, but not the second.
    > Perhaps someone can show me the error of my ways and get me back on
    > track.
    >
    > Input log file is formatted like this:
    >
    > TCP out 65.198.222.70:43679 in 192.168.0.3:80 idle 0:00:08

    ^^^^^^^^^^^
    > TCP out 209.73.24.2:29042 in 192.168.0.3:80 idle 0:00:08

    ^^^^^^^^^^^
    > TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00

    ^^^^^^^^^^^
    It is not going to be very useful to do a DNS lookup on 192.168/16
    addresses. See RFC1918 for the reason why.


    > use Socket;
    >
    > my ($ip, $hostname);
    > my ($SourceFile, $DestFile) = ("log1.txt", "log2.txt");
    >
    > open(INPUTFILE, "<$SourceFile") || die("ERROR: Cannot open
    > $SourceFile, $!");
    > open(OUTPUTFILE, ">$DestFile") || die("ERROR: Cannot open $DestFile,
    > $!");
    >
    > binmode INPUTFILE;
    > binmode OUTPUTFILE;
    >
    > while(<INPUTFILE>) {
    > my $line = $_;


    while ( my $line = <INPUTFILE> ) {


    > if ($line =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {
    > $ip = $1;


    To capture all IP addresses in the line you could use a for loop:

    for my $ip ( $line =~ /\d{1,3}(?:\.\d{1,3}){3}/g ) {

    Or a while loop:

    while ( $line =~ /(\d{1,3}(?:\.\d{1,3}){3})/g ) {
    my $ip = $1;

    Or assign them to an array:

    my @ips = $line =~ /\d{1,3}(?:\.\d{1,3}){3}/g;


    > $hostname = lc gethostbyaddr(inet_aton($ip),AF_INET);


    If the string in $ip is not a valid IP address then inet_aton() will
    return undef. Perhaps you should test for that?


    > if ($hostname ne "") {


    It looks like you don't have warnings enabled as that will warn "Use of
    uninitialized value in string ne" when gethostbyaddr() can't find a
    hostname.


    > $line =~ s/$g_ip/$g_hostname/;


    Where did the variables $g_ip and $g_hostname come from?


    > print " IP address $ip = $hostname\n";
    > }
    > else {
    > print " IP address $ip = *NO DNS RECORD FOUND*\n";
    > }
    > }
    >
    > print OUTPUTFILE $line;
    > }
    >
    > close(INPUTFILE);
    > close(OUTPUTFILE);



    John
    --
    use Perl;
    program
    fulfillment
     
    John W. Krahn, Jun 22, 2004
    #3
  4. Maynard

    Petri Guest

    In article <>, John W. Krahn says...

    > Maynard wrote:


    >> TCP out 65.198.222.70:43685 in 192.168.0.3:80 idle 0:00:00

    > ^^^^^^^^^^^
    > It is not going to be very useful to do a DNS lookup on
    > 192.168/16 addresses. See RFC1918 for the reason why.


    It may be useful at the OP's network.
    He might be running split-DNS, with an internal reverse zone for
    0.0.168.192.in-addr.arpa.
    In that case he would catch names of internal systems, instead of their
    IP-addresses.


    Petri
     
    Petri, Jun 22, 2004
    #4
  5. Maynard

    Vetle Roeim Guest

    On 22 Jun 2004 15:05:12 -0700, Maynard <> wrote:

    > Petri <> wrote...
    >
    >> It may be useful at the OP's network.
    >> He might be running split-DNS, with an internal reverse zone for
    >> 0.0.168.192.in-addr.arpa.
    >> In that case he would catch names of internal systems, instead of their
    >> IP-addresses.

    >
    >
    > Which is true in my case. Those get translated to host names on my
    > network, which is always nice.
    >
    > Thanks everyone for your responses and suggestions; I took some of
    > them and implemented a couple extras... like a hash file that "caches"
    > names/IPs, so that only new ones get looked up (saves tons of time
    > when there's a 2 second delay when a host doesn't have a DNS record),
    > and just for you John, I added a switch that skips non-routable IP
    > addrs, just in case you want to do that.


    For maximum speed I recommend using Net::DNS::Resolver and the
    bgsend method, to send multiple requests before waiting for
    answers, and then using IO::Select to read the answers as they
    become available.

    Incidentally, there's already a script that does this:
    <URL: http://jdrowell.com/projects/jdresolve >

    It may not be neccessary to do it like this, but that depends on
    the amount of data. I've created a script that has to resolve
    millions of IP addresses in a reasonable amount of time, and
    Net::DNS::Resolver in combination with IO::Select seems to be
    the way to go.


    [...]
    --
    Touch eyeballs to screen for cheap laser surgery!
     
    Vetle Roeim, Jun 23, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hugo
    Replies:
    10
    Views:
    1,385
    Matt Humphrey
    Oct 18, 2004
  2. Rogan Dawes
    Replies:
    4
    Views:
    428
    Rogan Dawes
    May 17, 2006
  3. Tim Ward
    Replies:
    4
    Views:
    5,693
    Tim Ward
    Jun 12, 2006
  4. Sam Roberts
    Replies:
    0
    Views:
    245
    Sam Roberts
    Mar 20, 2005
  5. gavino
    Replies:
    2
    Views:
    190
    Ted Zlatanov
    Jul 27, 2006
Loading...

Share This Page