Re: gzip HTTP results problem

Discussion in 'Python' started by Fredrik Lundh, Jul 29, 2003.

  1. Bill Loren wrote:

    > I've encountered a problem trying to decode gzip data
    > returned from an HTTP server I communicate with (I use urllib2).
    > I've tried to use both the gzip and zlib come-along python libraries but
    > alas.
    >
    > Have anyone of you ppl succeeded in talking gzip with an HTTP
    > server ?


    this might help:

    http://effbot.org/zone/consumer-gzip.htm

    (that piece of code is used in production code, so it should
    work...)

    </F>
     
    Fredrik Lundh, Jul 29, 2003
    #1
    1. Advertising

  2. Fredrik Lundh

    John J. Lee Guest

    "Fredrik Lundh" <> writes:

    > Bill Loren wrote:
    >
    > > I've encountered a problem trying to decode gzip data
    > > returned from an HTTP server I communicate with (I use urllib2).
    > > I've tried to use both the gzip and zlib come-along python libraries but
    > > alas.
    > >
    > > Have anyone of you ppl succeeded in talking gzip with an HTTP
    > > server ?

    >
    > this might help:
    >
    > http://effbot.org/zone/consumer-gzip.htm
    >
    > (that piece of code is used in production code, so it should
    > work...)


    That would go nicely with my unreleased latest version of
    ClientCookie, which is plug-compatible with urllib2. but makes it
    easier to add new functionality like this (I submitted a patch to
    Python library based on this a while back, and am hoping for
    comments).

    The idea is that you pass processor objects to build_opener just as if
    they were handler objects. Processors pre-process requests and
    post-process responses. This stops you having to subclass things like
    AbstractHTTPHandler. Ask if you want a copy.

    (code below is completely untested, unworking, purely illustrative!)

    import ClientCookie
    from GzipConsumer import GzipConsumer
    from cStringIO import StringIO

    class stupid_gzip_consumer:
    def __init__(self): self.data = []
    def feed(self, data): self.data.append(data)

    class stupid_gzip_wrapper:
    def __init__(self, response):
    self._response = response

    c = stupid_gzip_consumer()
    gzc = GzipConsumer(c)
    gzc.feed(response.read())
    self.__data = StringIO("".join(c.data))

    def __getattr__(self, key):
    # delegate unknown methods/attributes
    if key in ("read", "readline", "readlines"):
    return getattr(self.__data, key)
    else:
    return getattr(self._response, key)

    class HTTPGzipProcessor(ClientCookie.BaseProcessor):
    def http_response(self, request, response):
    # post-process response
    enc = response.hdrs.get["content-encoding"]
    if ("gzip" in enc) or ("compress" in enc):
    return stupid_gzip_wrapper(response)
    else:
    return response

    https_response = http_response


    opener = ClientCookie.build_opener(HTTPGzipProcessor)
    ClientCookie.install_opener(opener)

    response = urlopen("http://www.example.com/")
    print response.read()
    response.close()


    John
     
    John J. Lee, Jul 29, 2003
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bill Loren

    gzip HTTP results problem

    Bill Loren, Jul 29, 2003, in forum: Python
    Replies:
    0
    Views:
    319
    Bill Loren
    Jul 29, 2003
  2. Leif Wessman
    Replies:
    16
    Views:
    1,097
    Leif Wessman
    Dec 8, 2004
  3. Mark Smith

    How to uncompress HTTP GZIP data

    Mark Smith, Dec 19, 2005, in forum: Perl Misc
    Replies:
    6
    Views:
    296
    Paul Marquess
    Dec 20, 2005
  4. HTTP::Response and gzip

    , Jul 25, 2006, in forum: Perl Misc
    Replies:
    0
    Views:
    204
  5. James Marshall
    Replies:
    2
    Views:
    236
    James Marshall
    Aug 7, 2006
Loading...

Share This Page