Perl Hacker, Python Initiate

Discussion in 'Python' started by Gary Chambers, Feb 2, 2011.

  1. All,

    Given the following Perl script:

    #!/usr/bin/perl

    %dig = (
    solaris => "/usr/sbin/dig",
    linux => "/usr/bin/dig",
    darwin => "/usr/bin/dig"
    );

    $DIG = $dig{"$^O"};
    $DOMAIN = "example.com";
    $DNS = "ns.example.com";
    $DIGCMD = qq/$DIG \@$DNS $DOMAIN axfr/;

    open DIG, "$DIGCMD|" or die "$DIG: $!\n";
    while (<DIG>) {
    next if (/^;/); # Skip any comments
    # If we match a CNAME record, we have an alias to something.
    # $1 = alias (CNAME), $2 = canonical hostname
    if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*CNAME\s+(\S+)\.${DOMAIN}\.$/) {
    # Push an alias (CNAME) onto an array indexed on canonical hostname
    push(@{$cnames{$2}}, $1);
    }
    # Here's a standard A (canonical hostname) record
    # $1 = canonical hostname, $2 = IPv4 address
    if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*A\s+(\S+)$/) {
    $ip{$1} = $2;
    }
    }
    close DIG;

    # Format and display it like niscat hosts:
    # canonicalHostname alias1 [alias2 aliasN] ipAddress
    for $host (sort keys %ip) {
    print "$host ";
    if (defined(@{$cnames{$host}})) {
    print join(' ', @{$cnames{$host}});
    print " ";
    }
    print "$ip{$host}\n";
    }
    exit 0;

    Will someone please provide some insight on how to accomplish that task in
    Python? I am unable to continually (i.e. it stops after displaying a single
    line) loop through the output while testing for the matches on the two
    regular expressions. Thank you.

    -- Gary Chambers
    Gary Chambers, Feb 2, 2011
    #1
    1. Advertising

  2. Gary Chambers

    Carl Banks Guest

    On Feb 1, 8:36 pm, Gary Chambers <> wrote:

    > open DIG, "$DIGCMD|" or die "$DIG: $!\n";
    > while (<DIG>) {



    > Will someone please provide some insight on how to accomplish that task in
    > Python?  I am unable to continually (i.e. it stops after displaying a single
    > line) loop through the output while testing for the matches on the two
    > regular expressions.  Thank you.


    You may have called read() instead of readlines().


    Carl Banks
    Carl Banks, Feb 2, 2011
    #2
    1. Advertising

  3. On Tue, 01 Feb 2011 23:36:27 -0500, Gary Chambers wrote:

    > All,
    >
    > Given the following Perl script:


    [snip line noise]

    > Will someone please provide some insight on how to accomplish that task
    > in Python?


    No idea, I can't read Perl, and you shouldn't assume that people will be
    able to.

    Can you simplify your problem to the smallest sub-task that you cannot
    perform? Focus on this part of the problem:

    > I am unable to continually (i.e. it stops after displaying a
    > single line) loop through the output while testing for the matches on
    > the two regular expressions. Thank you.


    What Python code are you using to loop? Simplify the problem to focus on
    the fundamental problem, not the irrelevant details of looking up CNAME
    and A Records using dig.


    --
    Steven
    Steven D'Aprano, Feb 2, 2011
    #3
  4. Gary Chambers <> writes:

    > Given the following Perl script:


    [41 lines of Perl removed]

    Sorry, I'm lucky enough to be able to completely ignore Perl.

    > Will someone please provide some insight on how to accomplish that task in
    > Python?


    From what I understood in the comments of your script, here is a possible
    python scriptlet:

    import sys
    import socket
    canon,aliases,ipaddrs = socket.gethostbyname_ex(sys.argv[1])
    print canon,",".join(aliases),",".join(ipaddrs)

    See also getaddrinfo(). Note that a canonical name may have several ip
    addresses (try with www.google.com if you doubt).

    (BTW, this is a direct interface to gethostbyname(), and there is no
    real need to use a tool and parse its output.)

    > I am unable to continually (i.e. it stops after displaying a single
    > line) loop through the output while testing for the matches on the two
    > regular expressions. Thank you.


    It is hard to guess what you've tried. See the subprocess package
    documentation.

    -- Alain.
    Alain Ketterlin, Feb 2, 2011
    #4
  5. Gary Chambers

    Tom Boland Guest

    if you want to do dns lookups on a large number of hosts, then try
    looking at gnu adns, or if you don't mind each request blocking until
    it's complete, then see Alain's response below. I have written some
    scripts myself which do massively parallel dns lookups quickly using
    twisted.

    If this is an excercise in just trying to do a straight port of your
    program, and you're not interested in doing it in a nicer fashion, then
    I would see the simple response from Carl Banks, however, you should
    really have posted the python code you're trying instead!

    Cheers. Tom.

    On 02/02/11 10:24, Alain Ketterlin wrote:
    > Gary Chambers<> writes:
    >
    >
    >> Given the following Perl script:
    >>

    > [41 lines of Perl removed]
    >
    > Sorry, I'm lucky enough to be able to completely ignore Perl.
    >
    >
    >> Will someone please provide some insight on how to accomplish that task in
    >> Python?
    >>
    > > From what I understood in the comments of your script, here is a possible

    > python scriptlet:
    >
    > import sys
    > import socket
    > canon,aliases,ipaddrs = socket.gethostbyname_ex(sys.argv[1])
    > print canon,",".join(aliases),",".join(ipaddrs)
    >
    > See also getaddrinfo(). Note that a canonical name may have several ip
    > addresses (try with www.google.com if you doubt).
    >
    > (BTW, this is a direct interface to gethostbyname(), and there is no
    > real need to use a tool and parse its output.)
    >
    >
    >> I am unable to continually (i.e. it stops after displaying a single
    >> line) loop through the output while testing for the matches on the two
    >> regular expressions. Thank you.
    >>

    > It is hard to guess what you've tried. See the subprocess package
    > documentation.
    >
    > -- Alain.
    >
    Tom Boland, Feb 2, 2011
    #5
  6. All,

    > Insight will be easier to provide once we see your Python code.


    Thanks to all of you who replied to my original request for assistance. All
    points are valid and well-taken.

    I'm afraid that I can no longer locate the original Python code where I was
    encountering the problem I described in my request. Unfortunately, I chose
    to teach myself Python by first porting that script directly (for better or
    worse, just trying to make it work). Time constraints, a bit of
    frustration, and simply being comfortable and efficient in Perl have
    relegated the task to a far back burner. Since then, I've forwarded the
    Perl code to a few others to see if they could provide a solution, but have
    heard nothing in reply -- which is why I presented it to a world of real
    Python experts.

    If you can't make heads or tails of the Perl code, all I'm trying to do is
    loop through some dig output of a DNS zone transfer. There are two
    conditions I need to test: whether it's an A or CNAME record. Ultimately, I
    need to assemble the data into a single line listing first the canonical
    hostname (A), followed by the aliases (CNAME), and finally the IP address.
    For example:

    were-on-vacation wov vacation vaca holiday 192.168.110.121

    Finally, the problem I encountered in the Python code was having the script
    fail when it encountered a line that didn't match either of the two regular
    expressions. What I'm seeking is either some Python code that mimics what
    my Perl script is doing, or to be shown the Python way of how to accomplish
    something like that. Surprisingly, there's no mention of regular
    expressions in the Perl Phrasebook at
    http://wiki.python.org/moin/PerlPhrasebook.

    -- Gary Chambers
    Gary Chambers, Feb 2, 2011
    #6
  7. Gary Chambers

    Carl Banks Guest

    On Feb 2, 8:00 am, Gary Chambers <> wrote:
    > All,
    >
    > > Insight will be easier to provide once we see your Python code.

    >
    > Thanks to all of you who replied to my original request for assistance.  All
    > points are valid and well-taken.
    >
    > I'm afraid that I can no longer locate the original Python code where I was
    > encountering the problem I described in my request.  Unfortunately, I chose
    > to teach myself Python by first porting that script directly (for better or
    > worse, just trying to make it work).  Time constraints, a bit of
    > frustration, and simply being comfortable and efficient in Perl have
    > relegated the task to a far back burner.


    So use Perl then, if that's what you're familiar with and you don't
    have time to learn Python.

    >  Since then, I've forwarded the
    > Perl code to a few others to see if they could provide a solution, but have
    > heard nothing in reply -- which is why I presented it to a world of real
    > Python experts.


    How much money did you offer them to translate your script for you?
    Maybe they were unsatisfied with your offer.

    > If you can't make heads or tails of the Perl code, all I'm trying to do is
    > loop through some dig output of a DNS zone transfer.  There are two
    > conditions I need to test: whether it's an A or CNAME record.  Ultimately, I
    > need to assemble the data into a single line listing first the canonical
    > hostname (A), followed by the aliases (CNAME), and finally the IP address..


    I'll do it for US$150.


    Carl Banks
    Carl Banks, Feb 2, 2011
    #7
  8. On Wed, 02 Feb 2011 11:00:11 -0500, Gary Chambers wrote:

    > Finally, the problem I encountered in the Python code was having the
    > script fail when it encountered a line that didn't match either of the
    > two regular expressions. What I'm seeking is either some Python code
    > that mimics what my Perl script is doing, or to be shown the Python way
    > of how to accomplish something like that.



    I'd start with something like this:

    lines = ... # wherever you are getting your input from
    for line in lines:
    if condition1(line):
    process1(line)
    elif condition2(line):
    process2(line)


    If neither condition1 nor condition2 are true, the loop will just go onto
    the next line. Obviously the condition* and process* are just place-
    holders.


    > Surprisingly, there's no
    > mention of regular expressions in the Perl Phrasebook at
    > http://wiki.python.org/moin/PerlPhrasebook.


    Perhaps your browser's "find" command is broken, because I count nine
    matches. I don't know if any of them are useful or not.

    The basic technique for using regular expressions in Python is:


    import re # regexes are not built-in
    mo = re.search(r"\b\w*ish\b", "Nobody expects the Spanish Inquisition!")
    if mo is not None:
    mo.group()

    => 'Spanish'

    You don't have to use raw string r"", you can manually escape the
    backslashes:

    r"\b\w*ish\b" <=> "\\b\\w*ish\\b"


    --
    Steven
    Steven D'Aprano, Feb 3, 2011
    #8
  9. Gary Chambers <> writes:

    > Will someone please provide some insight on how to accomplish that
    > task in Python? I am unable to continually (i.e. it stops after
    > displaying a single line) loop through the output while testing for
    > the matches on the two regular expressions. Thank you.


    If I understand you correctly, here is the relevant part (untested):

    import subprocess, collections

    dig = subprocess.Popen(["dig", "ns.example.com", "example.com", "axfr"],
    stdout=subprocess.PIPE).stdout

    # defaultdict allows the equivalent of push @{$x{$y}}, $z
    cnames = collections.defaultdict(list)
    ip = {}

    for line in dig:
    if line.startswith(';'):
    continue # Skip any comments
    m = re.search(r'regexp1', line)
    if m:
    cnames[m.group(2)].append(m.group(1)) # push ...
    m = re.search(r'regexp2', line)
    if m:
    ip[m.group(1)] = m.group(2)
    Hrvoje Niksic, Feb 3, 2011
    #9
  10. Gary Chambers

    sturlamolden Guest

    On 2 Feb, 05:36, Gary Chambers <> wrote:

    > Given the following Perl script:


    (...)

    Let me quote the deceased Norwegian lisp hacker Erik Naggum:

    "Excuse me while I barf in Larry Wall's general direction."


    Sturla
    sturlamolden, Feb 3, 2011
    #10
  11. Gary Chambers

    waku Guest

    you've already got a hint on how to do it using library functions in
    python. below is a more literal suggestion.

    On Feb 1, 10:36 pm, Gary Chambers <> wrote:
    > All,
    >
    > Given the following Perl script:
    >
    > #!/usr/bin/perl
    >
    > %dig = (
    >      solaris => "/usr/sbin/dig",
    >      linux   => "/usr/bin/dig",
    >      darwin  => "/usr/bin/dig"
    > );



    dig = {"solaris":"/usr/sbin/dig", "linux":"/usr/bin/dig", "darwin":"/
    usr/bin/dig"}


    >
    > $DIG = $dig{"$^O"};


    dig = dig[os.uname()[0].lower()]


    > $DOMAIN = "example.com";
    > $DNS = "ns.example.com";


    domain, dns = ['%sexample.com'%p for p in ('', 'ns.')] # ;)

    > $DIGCMD = qq/$DIG \@$DNS $DOMAIN axfr/;


    digcmd = '%s @%s %s axfr' % (dig, dns, domain)

    >
    > open DIG, "$DIGCMD|" or die "$DIG: $!\n";
    > while (<DIG>) {
    >      next if (/^;/); # Skip any comments
    >      # If we match a CNAME record, we have an alias to something.
    >      # $1 = alias (CNAME), $2 = canonical hostname
    >      if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*CNAME\s+(\S+)\.${DOMAIN}\.$/) {
    >          # Push an alias (CNAME) onto an array indexed on canonical hostname
    >          push(@{$cnames{$2}}, $1);
    >      }
    >      # Here's a standard A (canonical hostname) record
    >      # $1 = canonical hostname, $2 = IPv4 address
    >      if (/^(\S+)\.${DOMAIN}\.\s+\d+\s+IN\s*A\s+(\S+)$/) {
    >          $ip{$1} = $2;
    >      }}
    >
    > close DIG;


    lines = [line for line in os.popen(digcmd) if not re.match(';', line)]
    cname, ip = [re.compile(s.format(domain))
    for s in (r'(\S+)\.{0}\.\s+\d+\s+IN\s*CNAME\s+(\S+)\.{0}\.$', r'(\S
    +)\.{0}\.\s+\d+\s+IN\s*A\s+(\S+)$')]
    cnames, ips = [dict(m.groups() for m in (p.match(l) for l in lines) if
    m) for p in cname, ip)]

    the rest is left as an exercise. i did not test this exact code
    because i don't have your data, but a modified version works on
    different data.

    vQ

    >
    > # Format and display it like niscat hosts:
    > # canonicalHostname alias1 [alias2 aliasN] ipAddress
    > for $host (sort keys %ip) {
    >      print "$host ";
    >      if (defined(@{$cnames{$host}})) {
    >          print join(' ', @{$cnames{$host}});
    >          print " ";
    >      }
    >      print "$ip{$host}\n";}
    >
    > exit 0;
    >
    > Will someone please provide some insight on how to accomplish that task in
    > Python?  I am unable to continually (i.e. it stops after displaying a single
    > line) loop through the output while testing for the matches on the two
    > regular expressions.  Thank you.
    >
    > -- Gary Chambers
    waku, Feb 3, 2011
    #11
  12. Gary Chambers

    Jorgen Grahn Guest

    On Wed, 2011-02-02, Gary Chambers wrote:
    > All,
    >
    > Given the following Perl script:
    >
    > #!/usr/bin/perl


    I'm a Perl user, but I generally refuse to read Perl code which
    doesn't utilize 'use warnings/-w' and 'use strict'. There are just too
    many crazy bugs and 1980s constructs which go unnoticed without them.

    > %dig = (
    > solaris => "/usr/sbin/dig",
    > linux => "/usr/bin/dig",
    > darwin => "/usr/bin/dig"
    > );


    Not related to your question, except that you'll have to deal with
    this in Python too:

    I really suggest letting the user's $PATH decide which dig to call.
    /usr/bin is always in the path. /usr/sbin may not be, but if that's a
    problem for your users, just let your script start by appending it to
    the pre-existing $PATH. You don't even have to do OS detection on
    that one -- it's safe to do everywhere.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
    Jorgen Grahn, Feb 3, 2011
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. D E
    Replies:
    4
    Views:
    397
    Rowland Banks
    Jun 30, 2004
  2. b wreath
    Replies:
    3
    Views:
    164
    Ben Morrow
    Jun 2, 2004
  3. doug

    how to become a perl hacker?

    doug, Oct 22, 2004, in forum: Perl Misc
    Replies:
    0
    Views:
    94
  4. doug

    How to become a perl hacker?

    doug, Oct 22, 2004, in forum: Perl Misc
    Replies:
    2
    Views:
    126
    Aaron Sherman
    Oct 23, 2004
  5. NovasTaylor
    Replies:
    1
    Views:
    123
    NovasTaylor
    Aug 16, 2005
Loading...

Share This Page