Perl Hacker, Python Initiate

[Gary Chambers]

All,

Given the following Perl script:

#!/usr/bin/perl

%dig = (
solaris => "/usr/sbin/dig",
linux => "/usr/bin/dig",
darwin => "/usr/bin/dig"
);

$DIG = $dig{"$^O"};
$DOMAIN = "example.com";
$DNS = "ns.example.com";
$DIGCMD = qq/$DIG \@$DNS $DOMAIN axfr/;

open DIG, "$DIGCMD|" or die "$DIG: $!\n";
while (<DIG>) {
next if (/^;/); # Skip any comments
# If we match a CNAME record, we have an alias to something.
# $1 = alias (CNAME), $2 = canonical hostname
if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*CNAME\s+(\S+)\.${DOMAIN}\.$/) {
# Push an alias (CNAME) onto an array indexed on canonical hostname
push(@{$cnames{$2}}, $1);
}
# Here's a standard A (canonical hostname) record
# $1 = canonical hostname, $2 = IPv4 address
if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*A\s+(\S+)$/) {
$ip{$1} = $2;
}
}
close DIG;

# Format and display it like niscat hosts:
# canonicalHostname alias1 [alias2 aliasN] ipAddress
for $host (sort keys %ip) {
print "$host ";
if (defined(@{$cnames{$host}})) {
print join(' ', @{$cnames{$host}});
print " ";
}
print "$ip{$host}\n";
}
exit 0;

Will someone please provide some insight on how to accomplish that task in
Python? I am unable to continually (i.e. it stops after displaying a single
line) loop through the output while testing for the matches on the two
regular expressions. Thank you.

-- Gary Chambers

 

[Carl Banks]

open DIG, "$DIGCMD|" or die "$DIG: $!\n";
while (<DIG>) {

Will someone please provide some insight on how to accomplish that task in
Python?  I am unable to continually (i.e. it stops after displaying a single
line) loop through the output while testing for the matches on the two
regular expressions.  Thank you.

You may have called read() instead of readlines().


Carl Banks

 

[Steven D'Aprano]

All,

Given the following Perl script:

[snip line noise]

Will someone please provide some insight on how to accomplish that task
in Python?

No idea, I can't read Perl, and you shouldn't assume that people will be
able to.

Can you simplify your problem to the smallest sub-task that you cannot
perform? Focus on this part of the problem:

I am unable to continually (i.e. it stops after displaying a
single line) loop through the output while testing for the matches on
the two regular expressions. Thank you.

What Python code are you using to loop? Simplify the problem to focus on
the fundamental problem, not the irrelevant details of looking up CNAME
and A Records using dig.

 

[Alain Ketterlin]

Given the following Perl script:

[41 lines of Perl removed]

Sorry, I'm lucky enough to be able to completely ignore Perl.

Will someone please provide some insight on how to accomplish that task in
Python?

From what I understood in the comments of your script, here is a possible
python scriptlet:

import sys
import socket
canon,aliases,ipaddrs = socket.gethostbyname_ex(sys.argv[1])
print canon,",".join(aliases),",".join(ipaddrs)

See also getaddrinfo(). Note that a canonical name may have several ip
addresses (try with www.google.com if you doubt).

(BTW, this is a direct interface to gethostbyname(), and there is no
real need to use a tool and parse its output.)

I am unable to continually (i.e. it stops after displaying a single
line) loop through the output while testing for the matches on the two
regular expressions. Thank you.

It is hard to guess what you've tried. See the subprocess package
documentation.

-- Alain.

 

[Tom Boland]

if you want to do dns lookups on a large number of hosts, then try
looking at gnu adns, or if you don't mind each request blocking until
it's complete, then see Alain's response below. I have written some
scripts myself which do massively parallel dns lookups quickly using
twisted.

If this is an excercise in just trying to do a straight port of your
program, and you're not interested in doing it in a nicer fashion, then
I would see the simple response from Carl Banks, however, you should
really have posted the python code you're trying instead!

Cheers. Tom.

Given the following Perl script:

[41 lines of Perl removed]

Sorry, I'm lucky enough to be able to completely ignore Perl.

Will someone please provide some insight on how to accomplish that task in
Python?

From what I understood in the comments of your script, here is a possible

python scriptlet:

import sys
import socket
canon,aliases,ipaddrs = socket.gethostbyname_ex(sys.argv[1])
print canon,",".join(aliases),",".join(ipaddrs)

See also getaddrinfo(). Note that a canonical name may have several ip
addresses (try with www.google.com if you doubt).

(BTW, this is a direct interface to gethostbyname(), and there is no
real need to use a tool and parse its output.)

I am unable to continually (i.e. it stops after displaying a single
line) loop through the output while testing for the matches on the two
regular expressions. Thank you.

It is hard to guess what you've tried. See the subprocess package
documentation.

-- Alain.

 

[Gary Chambers]

All,

Insight will be easier to provide once we see your Python code.

Thanks to all of you who replied to my original request for assistance. All
points are valid and well-taken.

I'm afraid that I can no longer locate the original Python code where I was
encountering the problem I described in my request. Unfortunately, I chose
to teach myself Python by first porting that script directly (for better or
worse, just trying to make it work). Time constraints, a bit of
frustration, and simply being comfortable and efficient in Perl have
relegated the task to a far back burner. Since then, I've forwarded the
Perl code to a few others to see if they could provide a solution, but have
heard nothing in reply -- which is why I presented it to a world of real
Python experts.

If you can't make heads or tails of the Perl code, all I'm trying to do is
loop through some dig output of a DNS zone transfer. There are two
conditions I need to test: whether it's an A or CNAME record. Ultimately, I
need to assemble the data into a single line listing first the canonical
hostname (A), followed by the aliases (CNAME), and finally the IP address.
For example:

were-on-vacation wov vacation vaca holiday 192.168.110.121

Finally, the problem I encountered in the Python code was having the script
fail when it encountered a line that didn't match either of the two regular
expressions. What I'm seeking is either some Python code that mimics what
my Perl script is doing, or to be shown the Python way of how to accomplish
something like that. Surprisingly, there's no mention of regular
expressions in the Perl Phrasebook at
http://wiki.python.org/moin/PerlPhrasebook.

-- Gary Chambers

 

[Carl Banks]

All,


Thanks to all of you who replied to my original request for assistance.  All
points are valid and well-taken.

I'm afraid that I can no longer locate the original Python code where I was
encountering the problem I described in my request.  Unfortunately, I chose
to teach myself Python by first porting that script directly (for better or
worse, just trying to make it work).  Time constraints, a bit of
frustration, and simply being comfortable and efficient in Perl have
relegated the task to a far back burner.

So use Perl then, if that's what you're familiar with and you don't
have time to learn Python.

 Since then, I've forwarded the
Perl code to a few others to see if they could provide a solution, but have
heard nothing in reply -- which is why I presented it to a world of real
Python experts.

How much money did you offer them to translate your script for you?
Maybe they were unsatisfied with your offer.

If you can't make heads or tails of the Perl code, all I'm trying to do is
loop through some dig output of a DNS zone transfer.  There are two
conditions I need to test: whether it's an A or CNAME record.  Ultimately, I
need to assemble the data into a single line listing first the canonical
hostname (A), followed by the aliases (CNAME), and finally the IP address..

I'll do it for US$150.


Carl Banks

 

[Steven D'Aprano]

Finally, the problem I encountered in the Python code was having the
script fail when it encountered a line that didn't match either of the
two regular expressions. What I'm seeking is either some Python code
that mimics what my Perl script is doing, or to be shown the Python way
of how to accomplish something like that.

I'd start with something like this:

lines = ... # wherever you are getting your input from
for line in lines:
if condition1(line):
process1(line)
elif condition2(line):
process2(line)


If neither condition1 nor condition2 are true, the loop will just go onto
the next line. Obviously the condition* and process* are just place-
holders.

Surprisingly, there's no
mention of regular expressions in the Perl Phrasebook at
http://wiki.python.org/moin/PerlPhrasebook.

Perhaps your browser's "find" command is broken, because I count nine
matches. I don't know if any of them are useful or not.

The basic technique for using regular expressions in Python is:


import re # regexes are not built-in
mo = re.search(r"\b\w*ish\b", "Nobody expects the Spanish Inquisition!")
if mo is not None:
mo.group()

=> 'Spanish'

You don't have to use raw string r"", you can manually escape the
backslashes:

r"\b\w*ish\b" <=> "\\b\\w*ish\\b"

 

[Hrvoje Niksic]

Will someone please provide some insight on how to accomplish that
task in Python? I am unable to continually (i.e. it stops after
displaying a single line) loop through the output while testing for
the matches on the two regular expressions. Thank you.

If I understand you correctly, here is the relevant part (untested):

import subprocess, collections

dig = subprocess.Popen(["dig", "ns.example.com", "example.com", "axfr"],
stdout=subprocess.PIPE).stdout

# defaultdict allows the equivalent of push @{$x{$y}}, $z
cnames = collections.defaultdict(list)
ip = {}

for line in dig:
if line.startswith(';'):
continue # Skip any comments
m = re.search(r'regexp1', line)
if m:
cnames[m.group(2)].append(m.group(1)) # push ...
m = re.search(r'regexp2', line)
if m:
ip[m.group(1)] = m.group(2)

 

[sturlamolden]

Given the following Perl script:

(...)

Let me quote the deceased Norwegian lisp hacker Erik Naggum:

"Excuse me while I barf in Larry Wall's general direction."


Sturla

 

[waku]

you've already got a hint on how to do it using library functions in
python. below is a more literal suggestion.

All,

Given the following Perl script:

#!/usr/bin/perl

%dig = (
     solaris => "/usr/sbin/dig",
     linux   => "/usr/bin/dig",
     darwin  => "/usr/bin/dig"
);

dig = {"solaris":"/usr/sbin/dig", "linux":"/usr/bin/dig", "darwin":"/
usr/bin/dig"}

$DIG = $dig{"$^O"};

dig = dig[os.uname()[0].lower()]

$DOMAIN = "example.com";
$DNS = "ns.example.com";

domain, dns = ['%sexample.com'%p for p in ('', 'ns.')] #

$DIGCMD = qq/$DIG \@$DNS $DOMAIN axfr/;

digcmd = '%s @%s %s axfr' % (dig, dns, domain)

open DIG, "$DIGCMD|" or die "$DIG: $!\n";
while (<DIG>) {
     next if (/^;/); # Skip any comments
     # If we match a CNAME record, we have an alias to something.
     # $1 = alias (CNAME), $2 = canonical hostname
     if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*CNAME\s+(\S+)\.${DOMAIN}\.$/) {
         # Push an alias (CNAME) onto an array indexed on canonical hostname
         push(@{$cnames{$2}}, $1);
     }
     # Here's a standard A (canonical hostname) record
     # $1 = canonical hostname, $2 = IPv4 address
     if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*A\s+(\S+)$/) {
         $ip{$1} = $2;
     }}

close DIG;

lines = [line for line in os.popen(digcmd) if not re.match(';', line)]
cname, ip = [re.compile(s.format(domain))
for s in (r'(\S+)\.{0}\.\s+\d+\s+IN\s*CNAME\s+(\S+)\.{0}\.$', r'(\S
+)\.{0}\.\s+\d+\s+IN\s*A\s+(\S+)$')]
cnames, ips = [dict(m.groups() for m in (p.match(l) for l in lines) if
m) for p in cname, ip)]

the rest is left as an exercise. i did not test this exact code
because i don't have your data, but a modified version works on
different data.

vQ

# Format and display it like niscat hosts:
# canonicalHostname alias1 [alias2 aliasN] ipAddress
for $host (sort keys %ip) {
     print "$host ";
     if (defined(@{$cnames{$host}})) {
         print join(' ', @{$cnames{$host}});
         print " ";
     }
     print "$ip{$host}\n";}

exit 0;

Will someone please provide some insight on how to accomplish that task in
Python?  I am unable to continually (i.e. it stops after displaying a single
line) loop through the output while testing for the matches on the two
regular expressions.  Thank you.

-- Gary Chambers

 

[Jorgen Grahn]

All,

Given the following Perl script:

#!/usr/bin/perl

I'm a Perl user, but I generally refuse to read Perl code which
doesn't utilize 'use warnings/-w' and 'use strict'. There are just too
many crazy bugs and 1980s constructs which go unnoticed without them.

%dig = (
solaris => "/usr/sbin/dig",
linux => "/usr/bin/dig",
darwin => "/usr/bin/dig"
);

Not related to your question, except that you'll have to deal with
this in Python too:

I really suggest letting the user's $PATH decide which dig to call.
/usr/bin is always in the path. /usr/sbin may not be, but if that's a
problem for your users, just let your script start by appending it to
the pre-existing $PATH. You don't even have to do OS detection on
that one -- it's safe to do everywhere.

/Jorgen