Screen Scrapping

Discussion in 'Perl Misc' started by Anna, Mar 17, 2007.

  1. Anna

    Anna Guest

    Hi: This is the site which gives information about social security
    Death index search. Is there any way i can pass the social number
    directly thru Perl program in the web site and get the three records
    Name, Birth, Death in my computer. I heard some thing about screen
    scrapping but no idea how to i use it

    Anna, Mar 17, 2007
    1. Advertisements

  2. Anna

    Mirco Wahab Guest

    This sound like a question for
    a programmers job assignement.

    Added: "you have ten minutes from now" ;-)

    You could simply pull the stuff by LWP::UserAgent
    and parse the resulting HTML response, like
    (quick & dirty):

    use strict;
    use warnings;
    use LWP::UserAgent;

    my %lookup = (
    firstname => 'George', middlename => 'W', lastname => 'Bush',
    ssn =>'', nt => 'exact', stat => ''

    my $url = '';
    my $rgx = qr{^<tr><td>(.*?)</td><td>(.*?)</td><td>(.*?)</td><td>(.*?)</td></tr>$};
    my $uag = LWP::UserAgent->new( agent=>'MSIE/6.0' );
    my @TxT = map { /$rgx/ ? [$1, $2, $3] : () } split /\n/,
    $uag->post('http://'.$url, \%lookup)->as_string;

    # stuff is now in $_->[0], $_->[1],$_->[2],
    print "@$_\n", for @TxT;

    Just enter the search term in %lookup and
    parse the response fields.


    Mirco Wahab, Mar 17, 2007
    1. Advertisements

  3. Start with the FAQ answer to your question:

    How do I automate an HTML form submission?
    Tad McClellan, Mar 18, 2007
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.