using HTTP Digest auth with arbitrary HTTP methods?

Discussion in 'Python' started by John Reese, Jan 3, 2005.

  1. John Reese

    John Reese Guest

    Hello there. I've run into some missing functionality with HTTP Digest
    authentication in the 2.3 library and I was wondering if I'm just
    missing something.

    Missing functionality the first: urllib2

    1a. You can add "handlers" to your opener indicating that you want to
    use HTTP Digest auth. This is nice way to handle it, but I don't
    see any way to use a custom verb in your URLOpener -- it always
    uses either GET or POST depending on whether you provided data.
    Is there any way to specify an arbitrary method? This would allow
    urllib2 to be used to write WebDAV clients.

    1b. HTTPDigestAuthHandler is initialized with an HTTPPasswordMgr
    object, which unfortunately deals in cleartext passwords. Digest
    authentication can be computed using only a hash of username,
    password, and realm; it would be nice if there was an alternate
    version of HTTPPasswordMgr that let you deal in hashes instead of
    or in addition to plaintext passwords.

    Missing functionality the second: httplib.

    2a. httplib.HTTPConnection lets you execute arbitrary HTTP methods
    with arbitrary headers and data; this is the missing functionality
    in 1a above. However, you have to deal with following redirects
    and authentication and so forth yourself. Is there any way to use
    the flexibility of
    HTTPConnection.request(method, url[, body[, headers]])
    with the convenience of the chains of urllib2 handlers?


    The upshot is what I'm trying to do is write a WebDAV-using DAV client
    library, and I almost have everything I need to do it; the problem is I
    can't find an easy way to do digest authentication for arbitrary HTTP
    methods. WebDAV (RFC 2518), for those not familiar, is an extension to
    HTTP that defines some new methods and settles the semantics of some
    existing but rarely implemented HTTP methods (PUT and DELETE, for
    example) to define something similar to a file system. It's intended
    for things like letting a group of people author a web site.
    John Reese, Jan 3, 2005
    #1
    1. Advertising

  2. John Reese

    John Reese Guest

    In comp.lang.python, wrote:
    > Hello there. I've run into some missing functionality with HTTP Digest
    > authentication in the 2.3 library and I was wondering if I'm just
    > missing something.
    >
    > Missing functionality the first: urllib2
    >
    > 1a. You can add "handlers" to your opener indicating that you want to
    > use HTTP Digest auth. This is nice way to handle it, but I don't
    > see any way to use a custom verb in your URLOpener -- it always
    > uses either GET or POST depending on whether you provided data.
    > Is there any way to specify an arbitrary method? This would allow
    > urllib2 to be used to write WebDAV clients.
    >
    > 1b. HTTPDigestAuthHandler is initialized with an HTTPPasswordMgr
    > object, which unfortunately deals in cleartext passwords. Digest
    > authentication can be computed using only a hash of username,
    > password, and realm; it would be nice if there was an alternate
    > version of HTTPPasswordMgr that let you deal in hashes instead of
    > or in addition to plaintext passwords.


    Well, I figured out a workaround, but it requires changing urllib2.py
    since the bugs are in base classes.

    I instead copied it (to urllib3.py) and made the following changes:
    a. in AbstractDigestAuthHandler.get_authorization, call
    req.get_method() instead of req.has_data() and 'POST' or 'GET'
    (python has a ternary operator, who knew)
    b. in AbstractHTTPHandler.do_open, call req.get_method instead of the
    hard-coded if-logic which is the same as that in req.get_method

    Both of these seem like bugs in urllib2.

    Then I overrode urllib2.Request and made it possibly to set the method,
    and then passed an instance of my custom Request class (the one that
    shouldn't have to exist, since Request should allow method to be set
    explicitly) to OpenerDirector.open().

    I'd like to see these changes make it into the standard library -- after
    being vetted by whoever's in charge of urllib2. Anybody know who I
    should talk to?
    John Reese, Jan 3, 2005
    #2
    1. Advertising

  3. John Reese

    John J. Lee Guest

    John Reese <> writes:

    > In comp.lang.python, wrote:

    [...]
    > I instead copied it (to urllib3.py) and made the following changes:
    > a. in AbstractDigestAuthHandler.get_authorization, call
    > req.get_method() instead of req.has_data() and 'POST' or 'GET'
    > (python has a ternary operator, who knew)


    (Re ternary operator: Everybody who read this list at certain times in
    the past is painfully aware of that fact, and of precisely why it's
    not quite true, and of all the syntax alternatives for real ternary
    conditionals that will never be part of Python ;-)


    > b. in AbstractHTTPHandler.do_open, call req.get_method instead of the
    > hard-coded if-logic which is the same as that in req.get_method
    >
    > Both of these seem like bugs in urllib2.


    Yup, bugs both.


    > Then I overrode urllib2.Request and made it possibly to set the method,
    > and then passed an instance of my custom Request class (the one that
    > shouldn't have to exist, since Request should allow method to be set
    > explicitly) to OpenerDirector.open().
    >
    > I'd like to see these changes make it into the standard library -- after
    > being vetted by whoever's in charge of urllib2. Anybody know who I
    > should talk to?


    Nobody is really in charge: just go ahead and submit a patch. Drop me
    an email when you do, and I'll try to review it. The only reason
    urllib2 doesn't already do arbitrary HTTP methods is that nobody has
    spent the time to think carefully if a .set_method() really is the
    right way to do it, then followed through with the work needed to get
    a patch applied.

    As always, a precondition for change is that somebody thinks something
    through carefully, writes tests, documentation, patch and submits all
    three to the SF patch tracker with a brief explanation like the one
    you give above.

    BTW, Greg Stein started work on adding the stuff you need at the
    httplib level (as module httpx). He seems too busy to finish it, but
    see modules httpx and davlib (one or both are in the Python CVS
    sandbox). He thinks httplib is a better place for DAV than urllib2,
    and he should know. But go ahead and fix urllib2 anyway... :)


    John
    John J. Lee, Jan 3, 2005
    #3
  4. John Reese

    John Reese Guest

    On 03 Jan 2005 18:11:06 +0000, John J. Lee <> wrote:
    > (Re ternary operator: Everybody who read this list at certain times in
    > the past is painfully aware of that fact, and of precisely why it's
    > not quite true, and of all the syntax alternatives for real ternary
    > conditionals that will never be part of Python ;-)


    Yeah, I ran across the PEP after I posted this. Sorry to bring up a
    sore subject....

    > Nobody is really in charge: just go ahead and submit a patch. Drop me
    > an email when you do, and I'll try to review it. The only reason
    > urllib2 doesn't already do arbitrary HTTP methods is that nobody has
    > spent the time to think carefully if a .set_method() really is the
    > right way to do it, then followed through with the work needed to get
    > a patch applied.


    Patch id #1095362 on Sourceforge. It turns out one of the bugs had
    already been fixed in CVS -- the digest bug was the only one left, so
    this is a one-line fix.

    >
    > As always, a precondition for change is that somebody thinks something
    > through carefully, writes tests, documentation, patch and submits all
    > three to the SF patch tracker with a brief explanation like the one
    > you give above.
    >
    > BTW, Greg Stein started work on adding the stuff you need at the
    > httplib level (as module httpx). He seems too busy to finish it, but
    > see modules httpx and davlib (one or both are in the Python CVS
    > sandbox). He thinks httplib is a better place for DAV than urllib2,
    > and he should know. But go ahead and fix urllib2 anyway... :)


    These files are 2 and 3 years old. He's probably right that an
    interface more like httplib would be more appropriate for a DAV client;
    urllib2 is more about generalizing open() to use URLs than fully
    exposing the protocol. But urllib2 does have all those convenient
    handler-chains, and a good DAV client needs to handle a lot of those
    same situations, so it might be a good idea to at least find a way to
    share the handler classes.
    John Reese, Jan 3, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tammy Mc
    Replies:
    3
    Views:
    205
    Tammy Mc
    Oct 1, 2006
  2. Jonathan Rochkind

    http digest auth client?

    Jonathan Rochkind, Nov 22, 2010, in forum: Ruby
    Replies:
    3
    Views:
    246
    Ammar Ali
    Nov 23, 2010
  3. myalo
    Replies:
    4
    Views:
    1,260
    A. Sinan Unur
    Nov 28, 2007
  4. Kevin McMurtrie

    Digest auth from HTTP to LDAP

    Kevin McMurtrie, Jun 8, 2012, in forum: Java
    Replies:
    0
    Views:
    389
    Kevin McMurtrie
    Jun 8, 2012
  5. Replies:
    2
    Views:
    321
    Julian Cromarty
    Jun 26, 2013
Loading...

Share This Page