LWP Post

Discussion in 'Perl Misc' started by Dale Gerdemann, Oct 21, 2004.

  1. I'm trying to figure out how use LWP::UserAgent to get dynamically
    generated Web pages using the POST method. I've followed the examples
    in the O'Reilly book: Perl & LWP by Sean M. Burke. There's an example
    there of posting to the AltaVista Babelfish translator, and this
    example works fine for me. But when I try to do someting similar on
    another page, it fails.

    For example, I tried to get morphological analysis (word structure
    analysis) from http://www.uni-plovdiv.bg/dcs/morphe.htm, and even
    though it's a very simple minded form, I had no success. Sean Burke
    has a program in his book called formpairs, which allows you to test
    what posted info a form is expecting. I used this and checked very
    carefully, but still, no success.

    Here's what the form looks like:

    function FrontPage_Form1_Validator(theForm)
    {

    if (theForm.D1.selectedIndex < 0)
    {
    alert("Please select one of the \"D1\" options.");
    theForm.D1.focus();
    return (false);
    }
    return (true);
    }
    //--></script><!--webbot BOT="GeneratedScript" endspan --><form
    action="http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe" method="POST"
    onsubmit="return FrontPage_Form1_Validator(this)"
    name="FrontPage_Form1">
    <p><textarea name="S1" rows="5" cols="82"></textarea></p>
    <p><!--webbot bot="Validation" b-value-required="TRUE" --><select
    name="D1" size="3">
    <option>1.Windows ( àáâãäåæçèéêëìíîïðñòóôõö÷øùúüþÿ )</option>
    <option>2.Macintosh ( abvgde'zijklmnoprstufxchw]=;_q )</option>
    <option>3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )</option>
    </select></p>

    <p><input type="submit" name="B1" value="Analysis"></p>
    </form>


    You see that there's a bit of Javascript getting in the way, but I
    don't think thats a problem here. [You also see that the author of the
    page mixed up is terminology and referred to ASCII as QWERTY.
    But that's beside the point.] I tried to access this page with the
    following:

    use strict;
    use LWP;
    my $browser;
    sub do_POST {
    # Parameters:
    # the URL,
    # an arrayref or hashref for the key/value pairs,
    # and then, optionally, any header lines: (key,value, key,value)
    $browser = LWP::UserAgent->new( ) unless $browser;
    my $resp = $browser->post(@_);
    return ($resp->content, $resp->status_line, $resp->is_success, $resp)
    if wantarray;
    return unless $resp->is_success;
    return $resp->content;
    }



    my ($content, $message, $is_success) = do_POST(
    'http://www.uni-plovdiv.bg/dcs/morphe.htm',
    [ 'S1' => 'ne', 'B1' => "Analysis",
    'D1'=>'3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )' ],
    );

    print "$content\n";



    Anybody have any ideas???


    Dale Gerdemann
    Dale Gerdemann, Oct 21, 2004
    #1
    1. Advertising

  2. Dale Gerdemann

    Tim Heaney Guest

    -tuebingen.de (Dale Gerdemann) writes:

    > my ($content, $message, $is_success) = do_POST(
    > 'http://www.uni-plovdiv.bg/dcs/morphe.htm',
    > [ 'S1' => 'ne', 'B1' => "Analysis",
    > 'D1'=>'3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )' ],
    > );
    >
    > print "$content\n";
    >
    >
    >
    > Anybody have any ideas???


    You don't want to post to the HTML page with the form on it, you want
    to post to their program. Try something like

    my ($content, $message, $is_success) = do_POST(
    'http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe',
    [ 'S1' => 'ne',
    'D1' => '3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )',
    'B1' => "Analysis",
    ],
    );

    instead (note the different URL). When I do that, I get

    <HTML><HEAD><TITLE>DELPHI CGI TEST</TITLE></HEAD>
    <BODY bgcolor="#FFFFFF">
    <STRONG>Result:<BR><BR>
    <P>ne :particle, :189 1 base form:ne</P>
    </B></P>
    </BODY>

    I hope this helps,

    Tim
    Tim Heaney, Oct 21, 2004
    #2
    1. Advertising

  3. Dale Gerdemann <-tuebingen.de> wrote:

    > I'm trying to figure out how use LWP::UserAgent to get dynamically
    > generated Web pages using the POST method.


    > For example, I tried to get morphological analysis (word structure
    > analysis) from http://www.uni-plovdiv.bg/dcs/morphe.htm, and even
    > though it's a very simple minded form, I had no success.



    > Here's what the form looks like:


    [snip]

    > <form
    > action="http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe" method="POST"

    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


    There is where the message should be posted to...


    > my ($content, $message, $is_success) = do_POST(
    > 'http://www.uni-plovdiv.bg/dcs/morphe.htm',

    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


    .... but that isn't where you are actually posting it!


    > Anybody have any ideas???



    Post to the correct URL. :)



    BTW, the "web scraping proxy" is a wildly valuable tool for
    reverse-engineering such things:

    http://www.research.att.com/~hpk/wsp/

    It wrote this Perl code for me when I tried it:

    $request = POST "http://rdesc.pu.acad.bg/cgi-bin/cgi_morf.exe" , [
    'S1' => "a Bulgarian sentence",
    'D1' => "3.QWERTY ( abwgdevzijklmnoprstufhc`[]yx\q )",
    'B1' => "Analysis",
    ] ;


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Oct 21, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. La Jesus
    Replies:
    9
    Views:
    1,298
    Gunnar Hjalmarsson
    Oct 27, 2003
  2. Karra
    Replies:
    2
    Views:
    757
    Octavian Rasnita
    Dec 29, 2010
  3. Mark Mykkanen

    LWP - Upload via HTTP POST

    Mark Mykkanen, Jul 2, 2003, in forum: Perl Misc
    Replies:
    1
    Views:
    359
    J. Gleixner
    Jul 2, 2003
  4. Robert
    Replies:
    5
    Views:
    106
    Anno Siegel
    Sep 23, 2003
  5. Zapanaz
    Replies:
    4
    Views:
    119
    Tad McClellan
    Oct 12, 2005
Loading...

Share This Page