Newbie Problem/Question: LWP and SSLEAY

A

Andrew

Hello,

I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
my $res = $browser->request($req);
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";
}
 
K

ko

Andrew said:
Hello,

I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
^^^^^^^^^^
get rid of the quotes
can shorten, see below
my $res = $browser->request($req);

you can shorten the two lines above to:
my $res = $ua->get( $ARGV[0] );
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";
}

Ran your script with the suggestions above and the line using proxy()
commented out, and it worked fine. So it could have something to do with
proxy(). You must pass a URL as the second argument to proxy(), from the
docs something like 'http://proxy.sn.no:8001/'. In your script its where
you use 'Edited_ContainsMyProxyInfo'.

HTH - keith
 
B

Bob Walton

Andrew wrote:

....
I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
my $res = $browser->request($req);
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";
}

FWIW, your script runs fine for me (verbatim other than I commented the
proxy line as I'm not using a proxy) on https://www.helsinki.fi . I'm
using AS Perl build 806 (Perl 5.8.0) on Windoze 98SE. Output was:

D:\junk>perl junk412.pl https://www.helsinki.fi
HTTP/1.1 200 OK
Connection: close
Date: Sat, 22 Nov 2003 03:45:34 GMT
Accept-Ranges: bytes
ETag: "7f9df-23e-393f715f"
Server: Apache/1.3.17 (Unix)
Content-Length: 574
Content-Type: text/html
Last-Modified: Thu, 08 Jun 2000 10:11:43 GMT
Client-Date: Sat, 22 Nov 2003 03:42:17 GMT
Client-Peer: 128.214.205.16:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /CN=University of Helsinki CA/OU=Administration
Office,
IT Department/O=University of Helsinki/C=FI
Client-SSL-Cert-Subject: /C=FI/ST=-/L=Helsinki/O=University of
Helsinki/OU=IT De
partment/CN=www.helsinki.fi/[email protected]
Client-SSL-Cipher: EDH-RSA-DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Title: University of Helsinki

<html>
<head>
<title>University of Helsinki</title>
</head>

<body>
<h1><a href="http://www.helsinki.fi/">University of Helsinki</a></h1>

This is the secure www server
(<a href="http://www.apache.org/">Apache</a>/<a
href="http://www.modssl.org/">mod_ssl</a>)
of the University of Helsinki, Administration Office,
<a href="http://www.helsinki.fi/atk/">IT Department</a>.

<P>

Get the HY-CA certificate authority root certificate from
<a href="http://www.helsinki.fi/ca/">http://www.helsinki.fi/ca/</a>.
<P>

<hr>

<address>[email protected]</address>

</body>
</html>

D:\junk>
 
A

Andrew

Bob Walton said:
Andrew wrote:

...
I am learning Perl and I have come across something I dont understand.
I have put together some code from examples and I see something
strange between data returned from HTTP and HTTPS sites. When I run
my script on a non secured site (http://www.yahoo.com for example) I
get connection info and page source. When I run the same script on a
HTTPS site (https://www.helsinki.fi/) I get the connection info, but
it returns an empty <HTML></HTML> bracket. Can anyone explain to me
why this is occuring in this script?

#!/usr/bin/perl
use warnings;
use strict;
use LWP;
use HTTP::Cookies;
use HTML::parser ();

my $browser = LWP::UserAgent->new();
my $cj = HTTP::Cookies->new(file
=>"/home/awilhite/perl/cookies.lwp",autosave => 1,);
$browser->proxy(['http','https'],'Edited_ContainsMyProxyInfo');
$browser->cookie_jar($cj);
my $req = HTTP::Request->new(GET => "$ARGV[0]");
my $res = $browser->request($req);
if ($res->is_success) {
print $res->as_string;
#print $res->content;
} else {
print "Failed: ", $res->status_line, "\n";

FWIW, your script runs fine for me (verbatim other than I commented the
proxy line as I'm not using a proxy) on https://www.helsinki.fi . I'm
using AS Perl build 806 (Perl 5.8.0) on Windoze 98SE. Output was:

D:\junk>perl junk412.pl https://www.helsinki.fi
HTTP/1.1 200 OK
Connection: close
Date: Sat, 22 Nov 2003 03:45:34 GMT
Accept-Ranges: bytes
ETag: "7f9df-23e-393f715f"
Server: Apache/1.3.17 (Unix)
Content-Length: 574
Content-Type: text/html
Last-Modified: Thu, 08 Jun 2000 10:11:43 GMT
Client-Date: Sat, 22 Nov 2003 03:42:17 GMT
Client-Peer: 128.214.205.16:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /CN=University of Helsinki CA/OU=Administration
Office,
IT Department/O=University of Helsinki/C=FI
Client-SSL-Cert-Subject: /C=FI/ST=-/L=Helsinki/O=University of
Helsinki/OU=IT De
partment/CN=www.helsinki.fi/[email protected]
Client-SSL-Cipher: EDH-RSA-DES-CBC3-SHA
Client-SSL-Warning: Peer certificate not verified
Title: University of Helsinki

<html>
<head>
<title>University of Helsinki</title>
</head>

<body>
<h1><a href="http://www.helsinki.fi/">University of Helsinki</a></h1>

This is the secure www server
(<a href="http://www.apache.org/">Apache</a>/<a
href="http://www.modssl.org/">mod_ssl</a>)
of the University of Helsinki, Administration Office,
<a href="http://www.helsinki.fi/atk/">IT Department</a>.

<P>

Get the HY-CA certificate authority root certificate from
<a href="http://www.helsinki.fi/ca/">http://www.helsinki.fi/ca/</a>.
<P>

<hr>

<address>[email protected]</address>

</body>
</html>

D:\junk>


Interesting... I commented out the Proxy section as well and then the
script just fine (on a machine thats not behind a firewall). I will
have to check out the proxy information a little better to see what I
am doing wrong there. Edited_MyProxyInfo did have a URL to my Proxy
before posting. Thanks for the coding tips as well!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,480
Members
44,900
Latest member
Nell636132

Latest Threads

Top