HTTP Get with Proxy authentication

[Adrian]

Hi Guys,

I am a total Perl newbie and I need some help with a simple script.

I need a script to download a file via HTTP. The tricky bit is that I
am behind a proxy server which requires authentication and the website
where the file resides also requires authentication.

Can you please post an example of a script that authenticates through a
proxy and then authenticates against a remote website and downloads a
file.

Your help will be greatly appreciated.

Thanks,

Adrian

 

[Matt Garrish]

Hi Guys,

What about the women?

I am a total Perl newbie and I need some help with a simple script.

If the script is so simple, why haven't you made any effort to write it
yourself?

I need a script to download a file via HTTP. The tricky bit is that I
am behind a proxy server which requires authentication and the website
where the file resides also requires authentication.

That is tricky. Even if someone were to write an example for you, and don't
hold your breath, you give no information about how your supposed to
authenticate.

Can you please post an example of a script that authenticates through a
proxy and then authenticates against a remote website and downloads a
file.

No, but if you run into a problem you can't solve through the docs or by
googling you're free to post here again for help.

Matt

 

[Adrian]

What about the women?


If the script is so simple, why haven't you made any effort to write it

That is tricky. Even if someone were to write an example for you, and don't
hold your breath, you give no information about how your supposed to
authenticate.


No, but if you run into a problem you can't solve through the docs or by
googling you're free to post here again for help.

Matt

Here is my solution.
Hopefully it helps the next person with a similar problem.


#! /usr/bin/perl -w
# client1.pl - a simple client
#----------------

use strict;
use Socket;
use FileHandle;

my $server = 'NAME OF PROXY GOES HERE';
my $portNumber = 8080;
my $proxyUserPassBase64 = 'BASE 64 ENNCODED ProxyUserroxyPassword
GOES HERE';

my $path = 'http://NAME OF EXTERNAL SITE GOES HERE:80/';
my $websiteUserPassBase64 = 'BASE 64 ENNCODED Userassword GOES
HERE';


# initialize host and port
my $host = shift || $server;
my $port = shift || $portNumber;

# get the port address
my $iaddr = inet_aton($host);
my $paddr = sockaddr_in($port, $iaddr);

# create the socket, connect to the port
socket(SOCKET, PF_INET, SOCK_STREAM, 0) or die "socket: $!";
connect(SOCKET, $paddr) or die "connect: $!";

autoflush SOCKET (1);


######### Send HTTP GET request

print SOCKET "GET $path HTTP/1.1\n";
print SOCKET "Host: $host\n";
print SOCKET "Connection: close\n";

print SOCKET "Authorization: Basic\n ";
print SOCKET "$websiteUserPassBase64\n";

print SOCKET "Proxy-Authorization: Basic\n ";
print SOCKET "$proxyUserPassBase64\n";

print SOCKET "Accept: text/html; */*\n";
print SOCKET "\n";


######### Receive HTTP response via SOCKET
my $data;

while ( <SOCKET> ) {
chomp;
$data .= "$_\n";
}

######### SOCKET (close); take down the session

close(SOCKET);


open OUTPUT, ">output.txt";
print OUTPUT $data;


close OUTPUT;


Cheers,

Adrian

 

[Adrian]

According to The American Heritage Dictionary of the English Language:
"guys: Informal. Persons of either sex."

 

[Matt Garrish]

Here is my solution.
Hopefully it helps the next person with a similar problem.

Glad you were able to find a solution on your own, but I doubt many people
will find it useful, especially since it's much easier to use the LWP module
(and that module provides a much greater range of options).

Matt

 

[Adrian]

Matt,

Thank you so much for your help. Its obvious that you are a perl
expert.

Not only that, but you were the only one kind enough to offer your
advice. If only other Perl developers were as helpful as you have
been, I may've even solved this problem sooner.

Also, would it be possible to post your easier and more flexible
solution here so that myself and others may also benefit from it.
Cheers,

Adrian

 

[Matt Garrish]

Also, would it be possible to post your easier and more flexible
solution here so that myself and others may also benefit from it.

There's no real point in my posting anything, as the documentation does a
much better job:

http://www.perldoc.com/perl5.8.0/lib/lwpcook.html

In your case, take a look at the section on proxies and authentication.

Matt