urllib2 request blocks

[Derek Fountain]

I just tried this:

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?

 

[Christophe Cavalaria]

I just tried this:




It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?

You can try to write the URL like that :

"https://username:[email protected]/~dmarti/contrib-faq/"

 

[John J. Lee]

I just tried this:


It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

That script doesn't hang for me, using Python 2.3. I get a 200
response, with content the same as I see in Mozilla (Linux journal
author's guide and FAQ). The socket module does not verify the
authenticity of servers, so neither does urllib or urllib2.

How can I prepare my code for this eventuality and ensure it doesn't block?

Depends on why it's blocking.

sockets in 2.3 now support a timeout (that parameter is not fed
through to the httplib, urllib or urllib2 public interfaces, but you
can still set the default socket timeout in your application code).
If it's DNS that blocking, though, that won't help you. I think
proxies like squid do DNS lookups in a separate process (I guess you
can use threads too).

Alternatively, you can start messing with things like asyncore or
Twisted, or use threads (I don't recommend the latter :-/).


John