urllib's functionality with urllib2

[Monty]

Hello,
Sorry for this maybe stupid newbie question but I didn't find any
answer in all my readings about python:

With urllib, using urlretrieve, it's possible to get the number of
blocks transferred and the total size of the file.

Is it possible to get those informations with urllib2 ?

( I have to use urllib2 and a Request Object with headers, urlretieve
doesn't work. Well, at least, all my attempts with urllib were
denied...)

Thanks for any help,

Monty

 

[Harlin Seritt]

Is there any reason why you can't import both?

import urllib as u
import urllib2 as uu

 

[Fuzzyman]

Hello,
Sorry for this maybe stupid newbie question but I didn't find any
answer in all my readings about python:

With urllib, using urlretrieve, it's possible to get the number of
blocks transferred and the total size of the file.

Is it possible to get those informations with urllib2 ?

( I have to use urllib2 and a Request Object with headers, urlretieve
doesn't work. Well, at least, all my attempts with urllib were
denied...)

urllib2.urlopen(req) returns a handle on the URL. This handle has
various properties - if a content length header was sent it will
include that.

If you read from the url in a loop you can do this :

.. h = urllib2.urlopen(req)
.. print h.info()
.. page = ''
.. count = 0
.. n = 100 # number of bytes to read at a time
.. while True:
.. a = h.read(n)
.. if not a: break
.. count += len(a) # len(a) may not be same as n for final read
.. page += a # prob. better to append to a list and do
''.join(a) afterwards
.. print 'Now transferred %s bytes.' % count

Does this help ?

Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml

 

[Monty]

Thanks very much Fuzzy,
It's help a lot. That's exactly what I was looking for.
I didn't know much about those handle stuff and what to do with.
The next hurdle for me will be to understand and manage multiple
threads, and it's good to know that one can find such helpful advices
here.( and maybe also on your website, I'm just opening the main page
and it's look like if there was much to see and learn...)
Bye,
Monty

 

[Monty]

No, there is no reason. I think I can import both, but I can't download
the file I want using urllib.retrieve. And urllib2 lack the retrieve
method (as much as I know at least) which very easily allow you to get
a feedback of the download (transfert rate and so on). That was my
problem.
It's OK now, Fuzzy explain me how to do that. (I'm really a newbie in
this marvellous world of programming, you know...)
Thanks anyway to take the time to read my post and to answer it.
Bye,
Monty