LWP POST Not Working

J

James Thornton

I'm trying to write an LWP script that POSTs to
https://lending.etrade.com/e/t/mortgage/expressratequote and returns
mortgage rates, but I cannot get it to work. Instead of returning
rates, it returns the same URL (as it should), but the page contains
the form autopopulated with the supplied values.

The page uses a session ID, and I think I'm managing it properly. To
rule out JavaScript interference, I turned off JavaScript in my
browser, submitted the form, and it returned the rates so JavaScript
isn't required. What am I missing? Thanks.

###################################

#!/usr/bin/perl -w

# Perl 5.8.0
# libwww-perl-5.79

use LWP::Debug qw(+);
use LWP::UserAgent;
use HTTP::Cookies;
use URI::URL;

my %tags = ();
#$tags{'frm'} = '/e/t/mortgage/expressratequote';
$tags{'loanpurpose'} = 'P';
$tags{'_formtarget'} = 'expressratequote';
$tags{'verifiedincome'} = 'y';
$tags{'cashout'} = 'none';
$tags{'genericloanprogram'} = '30-year fixed';
$tags{'amount'} = '200000';
$tags{'propertytype'} = 'SFR';
$tags{'estval'} = '300000';
$tags{'state'} = 'CA';
$tags{'purpose'} = '0';
$tags{'County'} = 'nothing';
$tags{'county1'} = 'nothing';
$tags{'getrates'} = 'submit';
my $curl = url("http:");
$curl->query_form(%tags);

my $ua = LWP::UserAgent->new(keep_alive => 50,
timeout => 30,
requests_redirectable => ['GET', 'HEAD', 'POST']);

$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1,
ignore_discard => 0));

my $cookie_jar = HTTP::Cookies->new;

#
# First request to init session ID
#
my $req = HTTP::Request->new(GET =>
'https://lending.etrade.com/e/t/mortgage/expressratequote');
$req->user_agent('Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)');

my $res = $ua->request($req);

# Grab the request object that generated the response
# (it could have redirected in which case the request object
# would be different than the one we created)
$req = $res->request();

$cookie_jar->extract_cookies($res);
&dump($req,$res);

#
# Actual Form Request
#
$req = HTTP::Request->new(POST =>
'https://lending.etrade.com/e/t/mortgage/expressratequote');
$req->content_type('application/x-www-form-urlencoded');
$req->content($curl->equery);
$req->referer('https://lending.etrade.com/e/t/mortgage/expressratequote');
$req->user_agent('Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)');
$cookie_jar->add_cookie_header($req);

$res = $ua->request($req);

# Grab the request object that generated the response
# (it could have redirected in which case the request object
# would be different than the one we created)
$req = $res->request();

$cookie_jar->extract_cookies($res);
&dump($req,$res);

print $res->content();
print "This is libwww-perl-$LWP::VERSION\n";


#
# dump sub for debugging
#
sub dump {
($req,$res) = @_;
print "\nREQUEST-HEADERS\n";
print $req->headers_as_string();
print "\nREQUEST-CONTENT\n";
print $req->content;
if ($res->is_success) {
print "\nRESPONSE-HEADERS\n";
print $res->headers_as_string();
print "\nRESPONSE-CONTENT\n";
#print $res->content;
} else {
print "\nRESPONSE-ERROR\n";
print $res->error_as_HTML();
}
}
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,905
Latest member
Kristy_Poole

Latest Threads

Top