J
Juanlu_001
Hello everyone. I'm writing a script in Python 2.7 which uses a
urllib2.OpenerDirector instance via urllib2.build_opener() to take
advantage of the urllib2.HTTPCookieProcessor class, because I need to
store and re-send the cookies I get:
opener =
urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar()))
However, after making several requests and moving the cookies around,
eventually I need to retrieve a list of URLs. I wanted to use
urllib.urlretrieve() because I read it downloads the file in chunks,
but I cannot because I need to carry my cookies on the request and
urllib.urlretrieve() uses a urllib.URLOpener, which doesn't have
support for cookie handlers like OpenerDirector has.
I want to know the reason of this strange way of splitting
functionality, and how can I achieve my goal.
Thank you in advance!
urllib2.OpenerDirector instance via urllib2.build_opener() to take
advantage of the urllib2.HTTPCookieProcessor class, because I need to
store and re-send the cookies I get:
opener =
urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar()))
However, after making several requests and moving the cookies around,
eventually I need to retrieve a list of URLs. I wanted to use
urllib.urlretrieve() because I read it downloads the file in chunks,
but I cannot because I need to carry my cookies on the request and
urllib.urlretrieve() uses a urllib.URLOpener, which doesn't have
support for cookie handlers like OpenerDirector has.
I want to know the reason of this strange way of splitting
functionality, and how can I achieve my goal.
Thank you in advance!