Fetching a gzipped webpage

[Barry]

Hi,

Here's my attempt at petching a webpage which is gzip encoded -

import urllib.request
import gzip
import io

request = urllib.request.Request(url='http://en.wiktionary.org/wiki/
baby',headers={'Accept': 'text/html','User-Agent':'Mozilla/5.0
(iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/420.1
(KHTML, like Gecko) Version/3.0 Mobile/1A542a Safari/419.3'} )
response = urllib.request.urlopen(request)
data = response.read()

compressedstream = io.StringIO(data)

gzipper = gzip.GzipFile(fileobj=compressedstream)
data = gzipper.read()
print(data)


However it gives the error:

Traceback (most recent call last):
File "test.py", line 9, in <module>
TypeError: initial_value must be str or None, not bytes

How should I be doing this for Python 3?

Thanks

Barry

 

[Peter Otten]

Here's my attempt at petching a webpage which is gzip encoded -

import urllib.request
import gzip
import io

request = urllib.request.Request(url='http://en.wiktionary.org/wiki/
baby',headers={'Accept': 'text/html','User-Agent':'Mozilla/5.0
(iPhone; U; CPU iPhone OS 3_0 like Mac OS X; en-us) AppleWebKit/420.1
(KHTML, like Gecko) Version/3.0 Mobile/1A542a Safari/419.3'} )
response = urllib.request.urlopen(request)
data = response.read()

compressedstream = io.StringIO(data)

gzipper = gzip.GzipFile(fileobj=compressedstream)
data = gzipper.read()
print(data)


However it gives the error:

Traceback (most recent call last):
File "test.py", line 9, in <module>
TypeError: initial_value must be str or None, not bytes

How should I be doing this for Python 3?

Use io.BytesIO instead of io.StringIO, as Rob Williscroft showed you
yesterday.

Peter