Perl LWP content request behind firewall and proxy server

[demonhunter]

Hi,
I am trying to fetch a web content using LWP and HTTP modules behind
corp firewall. I set proxy server as shown in my internet explorer
connection setup.
When i run the script, i got successful response, but the return
content is not the webpage i requested at all. It shows something
like:
-------------------------------------------------
function FindProxyForURL(url,host) {
me=myIpAddress();
if (me == "127.0.0.1") {return "DIRECT";}
if (host == "127.0.0.1") {return "DIRECT";}
if (host == "localhost") {return "DIRECT";}
if (isPlainHostName(host)) {return "DIRECT";}
..
..
..
---------------------------------------------------

Below is my sample script:
require LWP::UserAgent;
require HTTP::Request;

my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->proxy(['http', 'ftp'], 'http://autoproxy:9090');

my $request = HTTP::Request->new(POST => 'http://www.google.com');
my $response = $ua->request($request);
if ($response->is_success) {
print $response->content; # or whatever
}
else {
die $response->status_line;
}

Anyone can help me on this? will appreciate.

by,
demon hunter

 

[Gunnar Hjalmarsson]

I am trying to fetch a web content using LWP and HTTP modules
behind corp firewall.

my $request = HTTP::Request->new(POST => 'http://www.google.com');

------------------------------------^^^^

A POST request to get a web page?

 

[demonhunter]

------------------------------------^^^^

A POST request to get a web page?

Oh, that suppose to be a GET, i did some changes here and there when
it doesnt work in GET. And it gave me the same
"FindProxyForURL(url,host)" content. Any idea why?

 

[nobull]

I am trying to fetch a web content using LWP and HTTP modules behind
corp firewall. I set proxy server as shown in my internet explorer
connection setup.

$ua->proxy(['http', 'ftp'], 'http://autoproxy:9090');

When i run the script, i got successful response, but the return
content is not the webpage i requested at all. It shows something
like:
-------------------------------------------------
function FindProxyForURL(url,host) {
me=myIpAddress();
if (me == "127.0.0.1") {return "DIRECT";}
if (host == "127.0.0.1") {return "DIRECT";}
if (host == "localhost") {return "DIRECT";}
if (isPlainHostName(host)) {return "DIRECT";}
.
.
.
---------------------------------------------------

Go back and look at the internet explorer connection setup again.

Where did you see 'http://autoproxy:9090'?

I suspect it was not the address of the proxy. I suspect was the URL
of the proxy autoconfiguration script. I further suppose that tje
HTTP server at autoproxy:9090 is very simplistic and just returns the
PAC in response to all GET requests.

AFAIK LWP doesn't understand PACs (because it would require a full
JavaScript subsystem). (Actually there are people working on
JavaScript for LWP so one day soon there may be PAC support).

To figure out the address of the proxy examine the PAC yourself and
look for a line that says:

return "PROXY hostort";

This newsgroup does not exist (see FAQ). Please do not start threads
here.