Handling cookies without urllib2 and cookielib

Discussion in 'Python' started by Joshua Kugler, Dec 15, 2007.

  1. Standard disclaimer: read, googled, read some more. If you have a link,
    please free free to point me there.

    I'm using HTTPlib to construct some functional tests for a web app we're
    writing. We're not using urllib2 because we need support for PUT and
    DELETE methods, which urllib2 does not do.

    We also need client-side cookie handling. So, I start reading about
    cookielib and run across a problem. It's cookie handling is tied quite
    tightly to urllib2's request object. httplib has somewhat different
    semantics in its request object. So, you can use cookielib with httplib.
    And cookie lib has no simple function (that I could find) for passing in a
    set-cookie header and getting back a CookieJar object (or even a list of
    Cookie objects).

    I'm sure I'm not the first to have to deal with httplib and cookies. Anyone
    have suggestions or pointers?

    j
     
    Joshua Kugler, Dec 15, 2007
    #1
    1. Advertising

  2. On 14 dic, 23:44, Joshua Kugler <> wrote:

    > I'm using HTTPlib to construct some functional tests for a web app we're
    > writing. We're not using urllib2 because we need support for PUT and
    > DELETE methods, which urllib2 does not do.
    >
    > We also need client-side cookie handling. So, I start reading about
    > cookielib and run across a problem. It's cookie handling is tied quite
    > tightly to urllib2's request object. httplib has somewhat different
    > semantics in its request object. So, you can use cookielib with httplib.
    > And cookie lib has no simple function (that I could find) for passing in a
    > set-cookie header and getting back a CookieJar object (or even a list of
    > Cookie objects).


    What about correcting the first thing, making urllib2 support HEAD/PUT/
    DELETE?

    import urllib2

    class Request(urllib2.Request):

    def __init__(self, url, data=None, headers={},
    origin_req_host=None, unverifiable=False,
    method=None):
    urllib2.Request.__init__(self, url, data, headers,
    origin_req_host, unverifiable)
    self.method = method

    def get_method(self):
    if self.method is None:
    if self.data is not None:
    return "POST"
    else:
    return "GET"
    return self.method

    py> f = urllib2.urlopen(Request("http://www.python.org/",
    method="HEAD"))
    py> print f.info()
    Date: Sun, 16 Dec 2007 00:03:43 GMT
    Server: Apache/2.2.3 (Debian) DAV/2 SVN/1.4.2 mod_ssl/2.2.3 OpenSSL/
    0.9.8c
    Last-Modified: Sat, 15 Dec 2007 16:25:58 GMT
    ETag: "60193-3e6a-a24fb180"
    Accept-Ranges: bytes
    Content-Length: 15978
    Connection: close
    Content-Type: text/html

    py> print len(f.read())
    0

    Notes:
    a) Instead of urlopen(url,...) you must use urlopen(Request(url,...))
    b) Redirection is not handled correctly in HTTPRedirectHandler (the
    request method should be copied over)
    c) I've not verified PUT / DELETE methods
    d) I'll try to make a proper patch later

    --
    Gabriel Genellina
     
    Gabriel Genellina, Dec 16, 2007
    #2
    1. Advertising

  3. Gabriel Genellina wrote:

    > On 14 dic, 23:44, Joshua Kugler <> wrote:
    >
    >> I'm using HTTPlib to construct some functional tests for a web app we're
    >> writing. We're not using urllib2 because we need support for PUT and
    >> DELETE methods, which urllib2 does not do.
    >>
    >> We also need client-side cookie handling. So, I start reading about
    >> cookielib and run across a problem. It's cookie handling is tied quite
    >> tightly to urllib2's request object. httplib has somewhat different
    >> semantics in its request object. So, you can use cookielib with httplib.
    >> And cookie lib has no simple function (that I could find) for passing in
    >> a set-cookie header and getting back a CookieJar object (or even a list
    >> of Cookie objects).

    >
    > What about correcting the first thing, making urllib2 support HEAD/PUT/
    > DELETE?

    <SNIP>

    We may have to do that, and then hack on the Redirect handler too so it will
    properly keep the request method. But that's not our preference, for
    obvious reasons. :)

    I just find it hard to believe that no one has ever needed to do cookie
    handling in a generic way (i.e. input: set-cookie header, output: cookie
    objects) before. May have to write my own. Or sublcass/extend cookielib.

    j
     
    Joshua Kugler, Dec 18, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. C. Titus Brown

    RFC 2965 cookies, cookielib, and mailman.

    C. Titus Brown, Dec 27, 2004, in forum: Python
    Replies:
    0
    Views:
    513
    C. Titus Brown
    Dec 27, 2004
  2. Alex Hunsley

    cookielib and urllib2: thread-safe?

    Alex Hunsley, Jan 26, 2005, in forum: Python
    Replies:
    1
    Views:
    484
    John J. Lee
    Jan 29, 2005
  3. John J Lee
    Replies:
    0
    Views:
    357
    John J Lee
    Jan 30, 2005
  4. Titus Brown
    Replies:
    0
    Views:
    482
    Titus Brown
    Feb 13, 2005
  5. Gilles Ganault

    [cookielib] How to add cookies myself?

    Gilles Ganault, Dec 16, 2008, in forum: Python
    Replies:
    0
    Views:
    572
    Gilles Ganault
    Dec 16, 2008
Loading...

Share This Page