LWP POST Not Working

Discussion in 'Perl Misc' started by James Thornton, Jul 1, 2004.

  1. I'm trying to write an LWP script that POSTs to
    https://lending.etrade.com/e/t/mortgage/expressratequote and returns
    mortgage rates, but I cannot get it to work. Instead of returning
    rates, it returns the same URL (as it should), but the page contains
    the form autopopulated with the supplied values.

    The page uses a session ID, and I think I'm managing it properly. To
    rule out JavaScript interference, I turned off JavaScript in my
    browser, submitted the form, and it returned the rates so JavaScript
    isn't required. What am I missing? Thanks.

    ###################################

    #!/usr/bin/perl -w

    # Perl 5.8.0
    # libwww-perl-5.79

    use LWP::Debug qw(+);
    use LWP::UserAgent;
    use HTTP::Cookies;
    use URI::URL;

    my %tags = ();
    #$tags{'frm'} = '/e/t/mortgage/expressratequote';
    $tags{'loanpurpose'} = 'P';
    $tags{'_formtarget'} = 'expressratequote';
    $tags{'verifiedincome'} = 'y';
    $tags{'cashout'} = 'none';
    $tags{'genericloanprogram'} = '30-year fixed';
    $tags{'amount'} = '200000';
    $tags{'propertytype'} = 'SFR';
    $tags{'estval'} = '300000';
    $tags{'state'} = 'CA';
    $tags{'purpose'} = '0';
    $tags{'County'} = 'nothing';
    $tags{'county1'} = 'nothing';
    $tags{'getrates'} = 'submit';
    my $curl = url("http:");
    $curl->query_form(%tags);

    my $ua = LWP::UserAgent->new(keep_alive => 50,
    timeout => 30,
    requests_redirectable => ['GET', 'HEAD', 'POST']);

    $ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
    autosave => 1,
    ignore_discard => 0));

    my $cookie_jar = HTTP::Cookies->new;

    #
    # First request to init session ID
    #
    my $req = HTTP::Request->new(GET =>
    'https://lending.etrade.com/e/t/mortgage/expressratequote');
    $req->user_agent('Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)');

    my $res = $ua->request($req);

    # Grab the request object that generated the response
    # (it could have redirected in which case the request object
    # would be different than the one we created)
    $req = $res->request();

    $cookie_jar->extract_cookies($res);
    &dump($req,$res);

    #
    # Actual Form Request
    #
    $req = HTTP::Request->new(POST =>
    'https://lending.etrade.com/e/t/mortgage/expressratequote');
    $req->content_type('application/x-www-form-urlencoded');
    $req->content($curl->equery);
    $req->referer('https://lending.etrade.com/e/t/mortgage/expressratequote');
    $req->user_agent('Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)');
    $cookie_jar->add_cookie_header($req);

    $res = $ua->request($req);

    # Grab the request object that generated the response
    # (it could have redirected in which case the request object
    # would be different than the one we created)
    $req = $res->request();

    $cookie_jar->extract_cookies($res);
    &dump($req,$res);

    print $res->content();
    print "This is libwww-perl-$LWP::VERSION\n";


    #
    # dump sub for debugging
    #
    sub dump {
    ($req,$res) = @_;
    print "\nREQUEST-HEADERS\n";
    print $req->headers_as_string();
    print "\nREQUEST-CONTENT\n";
    print $req->content;
    if ($res->is_success) {
    print "\nRESPONSE-HEADERS\n";
    print $res->headers_as_string();
    print "\nRESPONSE-CONTENT\n";
    #print $res->content;
    } else {
    print "\nRESPONSE-ERROR\n";
    print $res->error_as_HTML();
    }
    }
     
    James Thornton, Jul 1, 2004
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. marius_13
    Replies:
    1
    Views:
    505
  2. Gaurav

    LWP:UserAgent not working ???

    Gaurav, Oct 3, 2003, in forum: Perl Misc
    Replies:
    9
    Views:
    542
    Gaurav
    Oct 7, 2003
  3. Dan
    Replies:
    3
    Views:
    133
    Brian Wakem
    Aug 19, 2005
  4. Larry Snider

    LWP::Simple Not Working

    Larry Snider, Dec 30, 2003, in forum: Perl Misc
    Replies:
    20
    Views:
    314
    Matt Garrish
    Jan 4, 2004
  5. Zapanaz
    Replies:
    4
    Views:
    134
    Tad McClellan
    Oct 12, 2005
Loading...

Share This Page