urllib2 request blocks

Discussion in 'Python' started by Derek Fountain, Apr 27, 2004.

  1. I just tried this:

    >>> import urllib2
    >>> urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )


    It sits forever. Loading that URL in a normal browser gets me a popup asking
    about server certificate authenticity, which presumably is what urllib2 is
    blocking at.

    How can I prepare my code for this eventuality and ensure it doesn't block?
     
    Derek Fountain, Apr 27, 2004
    #1
    1. Advertising

  2. Derek Fountain wrote:
    > I just tried this:
    >
    >
    >>>>import urllib2
    >>>>urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

    >
    >
    > It sits forever. Loading that URL in a normal browser gets me a popup asking
    > about server certificate authenticity, which presumably is what urllib2 is
    > blocking at.
    >
    > How can I prepare my code for this eventuality and ensure it doesn't block?


    You can try to write the URL like that :

    "https://username:p/~dmarti/contrib-faq/"
     
    Christophe Cavalaria, Apr 27, 2004
    #2
    1. Advertising

  3. Derek Fountain

    John J. Lee Guest

    Derek Fountain <> writes:

    > I just tried this:
    >
    > >>> import urllib2
    > >>> urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

    >
    > It sits forever. Loading that URL in a normal browser gets me a popup asking
    > about server certificate authenticity, which presumably is what urllib2 is
    > blocking at.


    That script doesn't hang for me, using Python 2.3. I get a 200
    response, with content the same as I see in Mozilla (Linux journal
    author's guide and FAQ). The socket module does not verify the
    authenticity of servers, so neither does urllib or urllib2.


    > How can I prepare my code for this eventuality and ensure it doesn't block?


    Depends on why it's blocking.

    sockets in 2.3 now support a timeout (that parameter is not fed
    through to the httplib, urllib or urllib2 public interfaces, but you
    can still set the default socket timeout in your application code).
    If it's DNS that blocking, though, that won't help you. I think
    proxies like squid do DNS lookups in a separate process (I guess you
    can use threads too).

    Alternatively, you can start messing with things like asyncore or
    Twisted, or use threads (I don't recommend the latter :-/).


    John
     
    John J. Lee, Apr 29, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Arjen
    Replies:
    3
    Views:
    454
    Scott Allen
    Feb 27, 2005
  2. Josef Cihal
    Replies:
    0
    Views:
    797
    Josef Cihal
    Sep 5, 2005
  3. nRk
    Replies:
    1
    Views:
    1,020
    Steven D'Aprano
    Feb 12, 2009
  4. matt
    Replies:
    1
    Views:
    285
    George Ogata
    Aug 6, 2004
  5. Steven Taylor
    Replies:
    9
    Views:
    271
    Brian Candler
    Apr 27, 2009
Loading...

Share This Page