urllib2 http authorization question

Discussion in 'Python' started by Matthew Wilson, Dec 14, 2003.

  1. I am writing a script to check on my router's external IP address. My
    ISP refreshes my IP very often and I use dyndns for the hostname for my
    computer. My Netgear mr814 router has a webserver that uses HTTP basic
    authorization.

    My script uses urllib2 to connect to the router and read the html page
    with the current external IP address. This is the function I wrote to
    lookup the router's external IP:

    def mr814(router_user, router_password):
    "Looks up the IP from the router and returns it as a string."
    req = urllib2.Request('http://192.168.0.1/sysstatus.html')
    req.add_header("USER-AGENT",
    "Mozilla/4.76 [en] (X11; U; Linux 2.4.1-0.1.9 i586)")
    req.add_header("AUTHORIZATION", string.strip("Basic " +
    base64.encodestring("%s:%s" % (router_user, router_password))))
    IP = re.compile('(\d{1,3}\.){3}\d{1,3}')
    for line in urllib2.urlopen(req):
    if IP.search(line):
    ip = line.strip()
    return ip

    The first time I run this script, I always get a urllib2.HTTPError. But
    then If I ran the exact same script again afterward, I get access.

    >>> lookup.mr814('username', 'password')

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "lookup.py", line 13, in mr814
    for line in urllib2.urlopen(req):
    File "/usr/lib/python2.3/urllib2.py", line 136, in urlopen
    return _opener.open(url, data)
    File "/usr/lib/python2.3/urllib2.py", line 333, in open
    '_open', req)
    File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
    result = func(*args)
    File "/usr/lib/python2.3/urllib2.py", line 849, in http_open
    return self.do_open(httplib.HTTP, req)
    File "/usr/lib/python2.3/urllib2.py", line 843, in do_open
    return self.parent.error('http', req, fp, code, msg, hdrs)
    File "/usr/lib/python2.3/urllib2.py", line 359, in error
    return self._call_chain(*args)
    File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
    result = func(*args)
    File "/usr/lib/python2.3/urllib2.py", line 419, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    urllib2.HTTPError: HTTP Error 401: Unauthorized
    >>> lookup.mr814('admin', '54wins')

    '66.72.200.255'

    This is baffling me. Does the webserver ignore the AUTHORIZATION header
    on the first request? Should I do something like this:

    >>> try:

    .... lookup.mr814('admin', '54wins')
    .... except urllib2.HTTPError: #I only want to catch 401 here, but how?
    .... print "I got back the 401 error. Trying again..."
    .... lookup.mr814('admin', '54wins')
    .... except:
    .... print "something else went wrong."

    If that is recommended, how do I distinguish between 401 errors, vs
    other errors?

    Thanks for the help.
    Matthew Wilson, Dec 14, 2003
    #1
    1. Advertising

  2. In article <>, Matthew Wilson wrote:

    [snip]
    >>>> lookup.mr814('username', 'password')

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > File "lookup.py", line 13, in mr814
    > for line in urllib2.urlopen(req):
    > File "/usr/lib/python2.3/urllib2.py", line 136, in urlopen
    > return _opener.open(url, data)
    > File "/usr/lib/python2.3/urllib2.py", line 333, in open
    > '_open', req)
    > File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
    > result = func(*args)
    > File "/usr/lib/python2.3/urllib2.py", line 849, in http_open
    > return self.do_open(httplib.HTTP, req)
    > File "/usr/lib/python2.3/urllib2.py", line 843, in do_open
    > return self.parent.error('http', req, fp, code, msg, hdrs)
    > File "/usr/lib/python2.3/urllib2.py", line 359, in error
    > return self._call_chain(*args)
    > File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
    > result = func(*args)
    > File "/usr/lib/python2.3/urllib2.py", line 419, in http_error_default
    > raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    > urllib2.HTTPError: HTTP Error 401: Unauthorized
    >>>> lookup.mr814('admin', '54wins')

    > '66.72.200.255'


    To me, but maybe this is a copy-n-paste error, it looks like you aren't calling
    lookup.mr814() with exactly the same args:

    >>> lookup.mr814('username', 'password')


    versus

    >>> lookup.mr814('admin', '54wins')


    ?

    >
    > This is baffling me. Does the webserver ignore the AUTHORIZATION header
    > on the first request? Should I do something like this:
    >
    >>>> try:

    > ... lookup.mr814('admin', '54wins')
    > ... except urllib2.HTTPError: #I only want to catch 401 here, but how?
    > ... print "I got back the 401 error. Trying again..."
    > ... lookup.mr814('admin', '54wins')
    > ... except:
    > ... print "something else went wrong."
    >
    > If that is recommended, how do I distinguish between 401 errors, vs
    > other errors?


    I don't know if it's recommended, but take a look at the source for the
    urllib2.HTTPError class, at the definition of __str__ (which is used by the
    traceback system):

    def __str__(self):
    return 'HTTP Error %s: %s' % (self.code, self.msg)

    In other words, the actual HTTP error code is in self.code... you could check
    for that something like this:

    try:
    lookup.mr814('admin', '54wins')
    except urllib2.HTTPError, err:
    if err.code != 401:
    raise

    >
    > Thanks for the help.


    I hope the above helps,

    /Troels Therkelsen
    Troels Therkelsen, Dec 14, 2003
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. BADBOY

    http Authorization

    BADBOY, Jul 14, 2004, in forum: Java
    Replies:
    3
    Views:
    5,898
    Stephen Ostermiller
    Jul 15, 2004
  2. Christopher J. Bottaro

    HTTP GET request with basic authorization?

    Christopher J. Bottaro, Jan 2, 2005, in forum: Python
    Replies:
    1
    Views:
    555
    Fuzzyman
    Jan 4, 2005
  3. Josef Cihal
    Replies:
    0
    Views:
    726
    Josef Cihal
    Sep 5, 2005
  4. SeanRW
    Replies:
    1
    Views:
    346
    Dominick Baier [DevelopMentor]
    May 25, 2006
  5. Greg

    Soap message w/http authorization question...

    Greg, May 18, 2009, in forum: ASP .Net Web Services
    Replies:
    0
    Views:
    786
Loading...

Share This Page