Newbie Problem/Question: LWP and SSLEAY

A

Andrew

Hello,

I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
my $res = $browser->request($req);
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";
}
 
K

ko

Andrew said:
Hello,

I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
Click to expand...
^^^^^^^^^^
get rid of the quotes
can shorten, see below
my $res = $browser->request($req);
Click to expand...

you can shorten the two lines above to:
my $res = $ua->get( $ARGV[0] );
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";
}
Click to expand...

Ran your script with the suggestions above and the line using proxy()
commented out, and it worked fine. So it could have something to do with
proxy(). You must pass a URL as the second argument to proxy(), from the
docs something like 'http://proxy.sn.no:8001/'. In your script its where
you use 'Edited_ContainsMyProxyInfo'.

HTH - keith
 
B

Bob Walton

Andrew wrote:

....
I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
my $res = $browser->request($req);
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";
}
Click to expand...

FWIW, your script runs fine for me (verbatim other than I commented the
proxy line as I'm not using a proxy) on https://www.helsinki.fi . I'm
using AS Perl build 806 (Perl 5.8.0) on Windoze 98SE. Output was:

D:\junk>perl junk412.pl https://www.helsinki.fi
HTTP/1.1 200 OK
Connection: close
Date: Sat, 22 Nov 2003 03:45:34 GMT
Accept-Ranges: bytes
ETag: "7f9df-23e-393f715f"
Server: Apache/1.3.17 (Unix)
Content-Length: 574
Content-Type: text/html
Last-Modified: Thu, 08 Jun 2000 10:11:43 GMT
Client-Date: Sat, 22 Nov 2003 03:42:17 GMT
Client-Peer: 128.214.205.16:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /CN=University of Helsinki CA/OU=Administration
Office,
IT Department/O=University of Helsinki/C=FI
Client-SSL-Cert-Subject: /C=FI/ST=-/L=Helsinki/O=University of
Helsinki/OU=IT De
partment/CN=www.helsinki.fi/[email protected]
Client-SSL-Cipher: EDH-RSA-DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Title: University of Helsinki

<html>
<head>
<title>University of Helsinki</title>
</head>

<body>
<h1><a href="http://www.helsinki.fi/">University of Helsinki</a></h1>

This is the secure www server
(<a href="http://www.apache.org/">Apache</a>/<a
href="http://www.modssl.org/">mod_ssl</a>)
of the University of Helsinki, Administration Office,
<a href="http://www.helsinki.fi/atk/">IT Department</a>.

<P>

Get the HY-CA certificate authority root certificate from
<a href="http://www.helsinki.fi/ca/">http://www.helsinki.fi/ca/</a>.
<P>

<hr>

<address>[email protected]</address>

</body>
</html>

D:\junk>
 
A

Andrew

Bob Walton said:
Andrew wrote:

...
I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
my $res = $browser->request($req);
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
Click to expand...
print "Failed: ", $res->status_line, "\n";
}
Click to expand...

FWIW, your script runs fine for me (verbatim other than I commented the
proxy line as I'm not using a proxy) on https://www.helsinki.fi . I'm
using AS Perl build 806 (Perl 5.8.0) on Windoze 98SE. Output was:

D:\junk>perl junk412.pl https://www.helsinki.fi
HTTP/1.1 200 OK
Connection: close
Date: Sat, 22 Nov 2003 03:45:34 GMT
Accept-Ranges: bytes
ETag: "7f9df-23e-393f715f"
Server: Apache/1.3.17 (Unix)
Content-Length: 574
Content-Type: text/html
Last-Modified: Thu, 08 Jun 2000 10:11:43 GMT
Client-Date: Sat, 22 Nov 2003 03:42:17 GMT
Client-Peer: 128.214.205.16:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /CN=University of Helsinki CA/OU=Administration
Office,
IT Department/O=University of Helsinki/C=FI
Client-SSL-Cert-Subject: /C=FI/ST=-/L=Helsinki/O=University of
Helsinki/OU=IT De
partment/CN=www.helsinki.fi/[email protected]
Client-SSL-Cipher: EDH-RSA-DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Title: University of Helsinki

<html>
<head>
<title>University of Helsinki</title>
</head>

<body>
<h1><a href="http://www.helsinki.fi/">University of Helsinki</a></h1>

This is the secure www server
(<a href="http://www.apache.org/">Apache</a>/<a
href="http://www.modssl.org/">mod_ssl</a>)
of the University of Helsinki, Administration Office,
<a href="http://www.helsinki.fi/atk/">IT Department</a>.

<P>

Get the HY-CA certificate authority root certificate from
<a href="http://www.helsinki.fi/ca/">http://www.helsinki.fi/ca/</a>.
<P>

<hr>

<address>[email protected]</address>

</body>
</html>

D:\junk>
Click to expand...


Interesting... I commented out the Proxy section as well and then the
script just fine (on a machine thats not behind a firewall). I will
have to check out the proxy information a little better to see what I
am doing wrong there. Edited_MyProxyInfo did have a URL to my Proxy
before posting. Thanks for the coding tips as well!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

LWP seems to hang 5
problems with LWP & HTTP 4
https request failing 2
can LWP handle this? 7
Help Needed - LWP Redirect problemss 8
LWP and firewall 1
HTTPS SSLeay problems - any experts out there? 0
LWP::UserAgent and SSL is it impossible? 4

Members online

Total: 346 (members: 3, guests: 343)
Robots: 186

Forum statistics

Threads
474,262
Messages
2,571,048
Members
48,769
Latest member
Clifft

Latest Threads

Top