Do HTTPError objects always have a read method?

[Steven D'Aprano]

I understood that HTTPError objects always have a read method, so they
can be treated as if they were a webpage. E.g. something like this
simplified snippet:


address = 'http://www.yahoo.com/spamspamspamspamspam'
try:
conn = urllib2.urlopen(address)
except urllib2.HTTPError, e:
conn = e
print conn.read() # Print the requested page, or the error page.



But in the source code to urllib2 (Python 2.5):


class HTTPError(URLError, addinfourl):
"""Raised when HTTP error occurs, but also acts like non-error
return"""
__super_init = addinfourl.__init__

def __init__(self, url, code, msg, hdrs, fp):
self.code = code
self.msg = msg
self.hdrs = hdrs
self.fp = fp
self.filename = url
# The addinfourl classes depend on fp being a valid file
# object. In some cases, the HTTPError may not have a valid
# file object. If this happens, the simplest workaround is to
# not initialize the base classes.
if fp is not None:
self.__super_init(fp, hdrs, url)

def __str__(self):
return 'HTTP Error %s: %s' % (self.code, self.msg)



That tells me that HTTPError objects aren't guaranteed to include a file-
like interface. That makes me unhappy.

Under what circumstances do HTTPError objects not have a valid file
object? How common is this? Does anyone have an example of a URL that
fails in that fashion?



Thank you,

 

[Gabriel Genellina]

En Wed, 17 Sep 2008 22:48:27 -0300, Steven D'Aprano

I understood that HTTPError objects always have a read method, so they
can be treated as if they were a webpage. E.g. something like this
simplified snippet:


address = 'http://www.yahoo.com/spamspamspamspamspam'
try:
conn = urllib2.urlopen(address)
except urllib2.HTTPError, e:
conn = e
print conn.read() # Print the requested page, or the error page.



But in the source code to urllib2 (Python 2.5):


class HTTPError(URLError, addinfourl):
"""Raised when HTTP error occurs, but also acts like non-error
return"""
__super_init = addinfourl.__init__

def __init__(self, url, code, msg, hdrs, fp):
self.code = code
self.msg = msg
self.hdrs = hdrs
self.fp = fp
self.filename = url
# The addinfourl classes depend on fp being a valid file
# object. In some cases, the HTTPError may not have a valid
# file object. If this happens, the simplest workaround is to
# not initialize the base classes.
if fp is not None:
self.__super_init(fp, hdrs, url)

That tells me that HTTPError objects aren't guaranteed to include a file-
like interface. That makes me unhappy.

Under what circumstances do HTTPError objects not have a valid file
object? How common is this? Does anyone have an example of a URL that
fails in that fashion?

Well, there is at least one case (AbstractDigestAuthHandler at line 864 in
urllib2.py) where HTTPError is raised explicitely with fp=None. It's not a
common case, I think. If you rely on the exception having a read() method,
add it a fake one yourself:

try: ...
except urllib2.HTTPError, e:
if not hasattr(e, 'read'):
e.read = e.readline = lambda self: '' # or perhaps e.msg
conn = e

 

[Gabriel Genellina]

En Thu, 18 Sep 2008 23:12:22 -0300, Gabriel Genellina

En Wed, 17 Sep 2008 22:48:27 -0300, Steven D'Aprano


Well, there is at least one case (AbstractDigestAuthHandler at line 864
in urllib2.py) where HTTPError is raised explicitely with fp=None. It's
not a common case, I think. If you rely on the exception having a read()
method, add it a fake one yourself:

try: ...
except urllib2.HTTPError, e:
if not hasattr(e, 'read'):
e.read = e.readline = lambda self: '' # or perhaps e.msg
conn = e

Oops, that should have been e.read = lambda: ''