Urllib2 upgrade is breaking urlopen()

[Jay Davis]

We have a lot of simple code using 'urllib', that basically
just goes out and opens and read()'s some http url. Very basic,
ten line scripts, and they work fine with urllib.

We want to use urllib2 for everything, though, but when we
do we get:

page=urllib2.urlopen("http://www.something.com/whatever.php")
....
File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
result = func(*args)
File "/usr/lib/python2.3/urllib2.py", line 463, in http_error_302
self.inf_msg + msg, headers, fp)
urllib2.HTTPError: HTTP Error 302: The HTTP server returned a redirect
error that would lead to an infinite loop.
The last 30x error message was:
Found


These redirect errors just don't happen with urrlib.urlopen(), and I
can
only find a few posts on deja that even mention this problem. Is
there a known workaround?

Thanks,

J.D.

 

[John J. Lee]

We have a lot of simple code using 'urllib', that basically
just goes out and opens and read()'s some http url. Very basic,
ten line scripts, and they work fine with urllib.

We want to use urllib2 for everything, though, but when we
do we get: [...]
urllib2.HTTPError: HTTP Error 302: The HTTP server returned a redirect
error that would lead to an infinite loop.
The last 30x error message was:
Found

Which version of Python are you using?

It might also help if you can give a publicly-accessible URL that
triggers the problem (mail it to me privately if you prefer).


John

 

[John J. Lee]

We have a lot of simple code using 'urllib', that basically
just goes out and opens and read()'s some http url. Very basic,
ten line scripts, and they work fine with urllib.

We want to use urllib2 for everything, though, but when we
do we get: [...]
urllib2.HTTPError: HTTP Error 302: The HTTP server returned a redirect
error that would lead to an infinite loop.
The last 30x error message was:
Found

Which version of Python are you using?

It might also help if you can give a publicly-accessible URL that
triggers the problem (mail it to me privately if you prefer).

Jay sent me a URL, and it turns out both urllib and urllib2 are
functioning as intended. There's a 302 loop caused by the server
expecting cookies to get returned. urllib goes into a loop until it
notices it's redirected 10 times, then just returns the 302 as a
normal response (which doesn't seem very nice, but appears to be what
was intended). urllib2 detects the loop and returns the 302 as an
HTTPError exception (which can be caught and .read() if necessary).


John