urllib2.urlopen taking way too much time

  • Thread starter Phonethics Mobile Media
  • Start date
P

Phonethics Mobile Media

handler = urllib2.urlopen(req) is taking way too much time to retrieve
the URL. The same code using sockets in PHP doesn't delay this long.
I had 'Authorization':'Basic ' + base64.b64encode("username:password")
in my header though.
[ I didnt use HTTPPasswordMgr & HTTPPasswordMgrWithDefaultRealm
because I was unable to send headers with the example shown here :
http://docs.python.org/howto/urllib2.html#id6 ]

I have data = handler.readline() in a loop - this is an endless
network script (twitter streaming) so cant really wait for the entire
url contents to finish loading.

I had to socket.setdefaulttimeout to 30 to make it work.
Is it because python is getting chunks in high number of KBs ?

Python 2.6.2
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,020
Latest member
GenesisGai

Latest Threads

Top