Need urllib.urlretrieve and urllib2.OpenerDirector together

Discussion in 'Python' started by Juanlu_001, Dec 26, 2010.

  1. Juanlu_001

    Juanlu_001 Guest

    Hello everyone. I'm writing a script in Python 2.7 which uses a
    urllib2.OpenerDirector instance via urllib2.build_opener() to take
    advantage of the urllib2.HTTPCookieProcessor class, because I need to
    store and re-send the cookies I get:

    opener =
    urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar()))

    However, after making several requests and moving the cookies around,
    eventually I need to retrieve a list of URLs. I wanted to use
    urllib.urlretrieve() because I read it downloads the file in chunks,
    but I cannot because I need to carry my cookies on the request and
    urllib.urlretrieve() uses a urllib.URLOpener, which doesn't have
    support for cookie handlers like OpenerDirector has.

    I want to know the reason of this strange way of splitting
    functionality, and how can I achieve my goal.

    Thank you in advance!
    Juanlu_001, Dec 26, 2010
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Sam Sungshik Kong

    urllib.urlretrieve error

    Sam Sungshik Kong, May 23, 2004, in forum: Python
    Replies:
    2
    Views:
    596
    Sam Sungshik Kong
    May 24, 2004
  2. ralobao

    Problem with urllib.urlretrieve

    ralobao, Jun 12, 2004, in forum: Python
    Replies:
    1
    Views:
    796
    fishboy
    Jun 12, 2004
  3. Ant
    Replies:
    1
    Views:
    350
    John J. Lee
    Jun 24, 2006
  4. Abandoned

    configure urllib.urlretrieve timeout

    Abandoned, Oct 7, 2007, in forum: Python
    Replies:
    1
    Views:
    592
    Steve Holden
    Oct 7, 2007
  5. Replies:
    9
    Views:
    154
Loading...

Share This Page