LWP::Request

[Kevin Su]

Can someone show me how to use LWP::request properly? Specifically,
for authorization. I want to get information from a protected website
by automating the log in process. I googled it online because someone
at my company recommended it to me, and I found a site containing:

"-C <username>:<password>
Provide credentials for documents that are protected by Basic
Authentication. If the document is protected and you did not specify
the username and password with this option, then you will be prompted
to provide these values."

http://toons.cs.ucsb.edu:4080/man?lwp-request

This looks useful to what I am trying to do, though I do not know how
to use it.

 

[Gregory Toomey]

Can someone show me how to use LWP::request properly? Specifically,
for authorization. I want to get information from a protected website
by automating the log in process. I googled it online because someone
at my company recommended it to me, and I found a site containing:

What you probably want (as dictinct from what you asked for) is how to use
basic authntication when retrieving a page:

#!/usr/bin/perl
use LWP::Simple;

my $screen_scraper=get("http://username:password\@site.com/restofurl");
# @ has been escaped


You can type URLS of the form http://username:[email protected]/restofurl
straight into your browser to avoid getting the popup for basic
authentication.

gtoomey

 

[J. Gleixner]

Can someone show me how to use LWP::request properly? Specifically,
for authorization. I want to get information from a protected website
by automating the log in process. I googled it online because someone
at my company recommended it to me, and I found a site containing:

"-C <username>:<password>
Provide credentials for documents that are protected by Basic
Authentication. If the document is protected and you did not specify
the username and password with this option, then you will be prompted
to provide these values."

http://toons.cs.ucsb.edu:4080/man?lwp-request

This looks useful to what I am trying to do, though I do not know how
to use it.

Type the following at your system's prompt

perldoc lwpcook

That'll give you a lot of useful examples for LWP. Specifically:

ACCESS TO PROTECTED DOCUMENTS
Documents protected by basic authorization can easily be
accessed like this:

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => 'http://www.linpro.no/secret/');
$req->authorization_basic('username', 'thepassword');
print $ua->request($req)->as_string;

The other alternative is to provide a subclass of
LWP::UserAgent that overrides the get_basic_credentials()
method. Study the lwp-request program for an example of
this.

You may need a few additional modules for SSL, see
http://search.cpan.org/ for those.

 

[Gregory Toomey]

....

That works OK for Internet Explorer, but not for Opera.

It's part of the HTTP definition. Blame Opera.

gtoomey

 

[Gisle Aas]

...

It's part of the HTTP definition. Blame Opera.

It's not. RFC 2616 does not allow userinfo in HTTP URLs.

 

[Alan J. Flavell]

On Thu, Jul 31, Gregory Toomey inscribed on the eternal scroll:

[re: userassword credentials embedded in a URL:]

news:bg8nm4>

^^^^^^^^^^^

(your usenet client appears to have problems with quoting message-ids,
by the way, although your References header looks OK)

...

It's part of the HTTP definition.

It's part of the generic URL/URI definition, but the http: scheme
excludes it. See
http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.2.2

I don't see this issue discussed explicitly in section 15 "Security",
but, reading the text there, it's obviously relevant to a number of
the issues covered there, such as the presence of sensitive
information in server logs.

See also RFC2396 section 3.2.2, which I would interpret as being a
general deprecation of its use in any "scheme".

Blame Opera.

For following security recommendations?

all the best

 

[Vlad Tepes]

That works OK for Internet Explorer, but not for Opera.

Such urls work in opera 7. If you attempt to go to such an address,
opera warns about a username in the url and asks you to confirm if you
want to go to the site.