urllib's functionality with urllib2

Discussion in 'Python' started by Monty, Mar 13, 2005.

  1. Monty

    Monty Guest

    Hello,
    Sorry for this maybe stupid newbie question but I didn't find any
    answer in all my readings about python:

    With urllib, using urlretrieve, it's possible to get the number of
    blocks transferred and the total size of the file.

    Is it possible to get those informations with urllib2 ?

    ( I have to use urllib2 and a Request Object with headers, urlretieve
    doesn't work. Well, at least, all my attempts with urllib were
    denied...)

    Thanks for any help,

    Monty
    Monty, Mar 13, 2005
    #1
    1. Advertising

  2. Is there any reason why you can't import both?

    import urllib as u
    import urllib2 as uu
    Harlin Seritt, Mar 14, 2005
    #2
    1. Advertising

  3. Monty

    Fuzzyman Guest

    Monty wrote:
    > Hello,
    > Sorry for this maybe stupid newbie question but I didn't find any
    > answer in all my readings about python:
    >
    > With urllib, using urlretrieve, it's possible to get the number of
    > blocks transferred and the total size of the file.
    >
    > Is it possible to get those informations with urllib2 ?
    >
    > ( I have to use urllib2 and a Request Object with headers, urlretieve
    > doesn't work. Well, at least, all my attempts with urllib were
    > denied...)
    >


    urllib2.urlopen(req) returns a handle on the URL. This handle has
    various properties - if a content length header was sent it will
    include that.

    If you read from the url in a loop you can do this :

    .. h = urllib2.urlopen(req)
    .. print h.info()
    .. page = ''
    .. count = 0
    .. n = 100 # number of bytes to read at a time
    .. while True:
    .. a = h.read(n)
    .. if not a: break
    .. count += len(a) # len(a) may not be same as n for final read
    .. page += a # prob. better to append to a list and do
    ''.join(a) afterwards
    .. print 'Now transferred %s bytes.' % count

    Does this help ?

    Regards,

    Fuzzy
    http://www.voidspace.org.uk/python/index.shtml

    > Thanks for any help,
    >
    > Monty
    Fuzzyman, Mar 14, 2005
    #3
  4. Monty

    Monty Guest

    Thanks very much Fuzzy,
    It's help a lot. That's exactly what I was looking for.
    I didn't know much about those handle stuff and what to do with.
    The next hurdle for me will be to understand and manage multiple
    threads, and it's good to know that one can find such helpful advices
    here.( and maybe also on your website, I'm just opening the main page
    and it's look like if there was much to see and learn...)
    Bye,
    Monty
    Monty, Mar 14, 2005
    #4
  5. Monty

    Monty Guest

    No, there is no reason. I think I can import both, but I can't download
    the file I want using urllib.retrieve. And urllib2 lack the retrieve
    method (as much as I know at least) which very easily allow you to get
    a feedback of the download (transfert rate and so on). That was my
    problem.
    It's OK now, Fuzzy explain me how to do that. (I'm really a newbie in
    this marvellous world of programming, you know...)
    Thanks anyway to take the time to read my post and to answer it.
    Bye,
    Monty
    Monty, Mar 14, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris Lyon
    Replies:
    1
    Views:
    17,315
    Terry Carroll
    Jan 8, 2004
  2. Alex Stapleton
    Replies:
    2
    Views:
    328
    Fuzzyman
    Mar 14, 2005
  3. Alex Stapleton
    Replies:
    5
    Views:
    720
    Mike Meyer
    Mar 15, 2005
  4. Jonathan Gardner

    Asynchronous urllib (urllib+asyncore)?

    Jonathan Gardner, Feb 26, 2008, in forum: Python
    Replies:
    1
    Views:
    458
    Terry Jones
    Feb 27, 2008
  5. Chris McDonald
    Replies:
    0
    Views:
    267
    Chris McDonald
    Nov 1, 2010
Loading...

Share This Page