case study: want to print redirects when LWP::UserAgent makes arequest

B

Bennett Haselton

I'm trying to figure out, if I have a LWP::UserAgent object and a
HTTP::Request object, how I can use the user-agent to request the
request object and print out a list of HTTP redirects that it
encounters along the way.

But, that's actually not the question. What I want to know is: How do
more experienced Perl programmers find the answers to these questions
when they need to get something done? I keep encountering problems
like this, reading the documentation, hitting a dead end, and having
to post in a forum or ask a friend. It's hard to believe that real
Perl programmers would have to do this for every simple question like
this, or nothing would ever get done.

Suppose you're an experienced Perl programmer but you don't know the
answer to this particular question. I look first in the documentation
page UserAgent.html. I see it says "The difference from request() is
that simple_request() will not try to handle redirects or
authentication responses. The request() method will in fact invoke
this method for each simple request it sends.", so I know about the
difference between those two.

The next thing that stands out is:
"$ua->redirect_ok( $request )
This method is called by request() before it tries to follow a
redirection to the request in $request. This should return a TRUE
value if this redirection is permissible."
It's not clear whether LWP::UserAgent calls this for *every* redirect,
or just the first one. (Taken literally, the documentation says it
will only do it for the first one -- "before it tries to follow a
redirection to the request in $request".) But still, that would mean
I'd have to create a subclass of LWP::UserAgent to override behavior
of this function. I'm hoping there's something easier.

So that's where I give up and ask someone. What would you do? Is
there something in the documentation that would make it obvious to you
what to try next? What do I need to train myself to look for?
 
J

John Bokma

Bennett Haselton said:
I'm trying to figure out, if I have a LWP::UserAgent object and a
HTTP::Request object, how I can use the user-agent to request the
request object and print out a list of HTTP redirects that it
encounters along the way.

my $response = $ua->get( $url );

returns a HTTP::Response

HTTP::Response documentation shows a method redirects, which "Returns a
list of redirect responses that lead up to ... The list order is oldest
first ..."

But, that's actually not the question. What I want to know is: How do
more experienced Perl programmers find the answers to these questions
when they need to get something done?

use YAML::Syck;
print Dump $response;

This shows a lot of info about $response
Knowing what the get method returns (to what it's blessed) and reading
the documentation.
I keep encountering problems
like this, reading the documentation, hitting a dead end, and having
to post in a forum or ask a friend. It's hard to believe that real
Perl programmers would have to do this for every simple question like
this, or nothing would ever get done.

Now and then I look at the source to understand what some things do. The
documentation is not always complete. Now and then I contact authors
with a request to update the documentation. On CPAN there is also the
possibility to add annotations.
 
P

Peter J. Holzer

Suppose you're an experienced Perl programmer but you don't know the
answer to this particular question. I look first in the documentation
page UserAgent.html. I see it says "The difference from request() is
that simple_request() will not try to handle redirects or
authentication responses. The request() method will in fact invoke
this method for each simple request it sends.", so I know about the
difference between those two.

The next thing that stands out is:
"$ua->redirect_ok( $request )
This method is called by request() before it tries to follow a
redirection to the request in $request. This should return a TRUE
value if this redirection is permissible."
It's not clear whether LWP::UserAgent calls this for *every* redirect,
or just the first one. (Taken literally, the documentation says it
will only do it for the first one -- "before it tries to follow a
redirection to the request in $request".)

I agree that this isn't crystal-clear, but I don't think your "literal"
interpretation is reasonable: Note that it says "redirection TO the
request in $request". So if you have a redirect cascade

http://foo.example.com
-> http://www.example.com/foo
-> http://www.example.com/show.pl?doc=foo

then it will first be called with $request "GET http://www.example.com/foo",
and if that was allowed and succeeded with the new $request
"GET http://www.example.com/show.pl?doc=foo"

For current versions of LWP::UserAgent, redirect_ok takes two arguments,
btw - the new request and the response which caused the redirect.

But still, that would mean
I'd have to create a subclass of LWP::UserAgent to override behavior
of this function.

Yes. If the default redirect_ok isn't sufficient (AFAIK you can turn on
and off redirects for each request type (GET, POST, ...), but that's it)
then subclassing and overriding this method is the cleanest (and
probably simplest) way.
I'm hoping there's something easier.

It sounds like just turning off redirects (see requests_redirectable) or
using simple_request is sufficent.

So that's where I give up and ask someone. What would you do? Is
there something in the documentation that would make it obvious to you
what to try next? What do I need to train myself to look for?

What I generally do when the documentation isn't sufficiently clear is
to write small test scripts which use the function I want to explore. I
may even use the debugger to step through this function and see what it
does.

I'm rather impatient though (as a Perl programmer I can admit that) and
if I can't figure out how something works within a short time I write it
myself. So supposing I couldn't figure out how redirect_ok was supposed
to work, I'd just fall back to using simple_request and write an extra
loop to handle the redirects.

hp
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top