Timeout in urllib2

Discussion in 'Python' started by Dan M, Nov 23, 2005.

  1. Dan M

    Dan M Guest

    I'm writing a system monitor script that needs to load web pages. I'm
    using urllib2.urlopen to get the pages, and I'm attempting to set the
    timeout value using socket.defaulttimeout.

    Specifically, I'm calling socket.defaultttimeout(10), then calling
    urllib2.urlopen to fetch a web page that never gets delivered. My code
    waits about 30 seconds before terminating.

    I am about to add threading to my app so that delays on a few servers
    won't be a critical issue, but I'd still like to understand why the call
    to socket.defaulttimeout doesn't affect the timeout on my urlopen calls.

    My code follows.

    #!/usr/local/bin/python
    import socket, time

    socket.setdefaulttimeout(10)
    import urllib2

    def doHttpTest():
    url = "http://url.that.never.returns"
    t_start = time.time()
    if httptest(url):
    print "Error on site ", url
    t_end = time.time()
    t_diff = t_end - t_start

    def httptest(url):
    timeout = 10
    socket.setdefaulttimeout(timeout)
    try:
    req = urllib2.Request(url)
    urllib2.urlopen(req)
    except urllib2.HTTPError, e:
    if e.code == 401:
    return 1
    elif e.code == 404:
    return 1
    elif e.code == 503:
    return 1
    else:
    return 1
    except urllib2.URLError, e:
    return 1
    else:
    return 0



    if __name__ == '__main__':
    try:
    doHttpTest()
    except KeyboardInterrupt:
    print "Exiting..."
    Dan M, Nov 23, 2005
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Doug Gray

    urllib2 timeout??

    Doug Gray, Apr 26, 2004, in forum: Python
    Replies:
    1
    Views:
    892
    John J. Lee
    Apr 29, 2004
  2. Gerard C Blais

    urllib2 and timeout question

    Gerard C Blais, May 2, 2005, in forum: Python
    Replies:
    2
    Views:
    5,005
    Gerard C Blais
    May 3, 2005
  3. Josef Cihal
    Replies:
    0
    Views:
    744
    Josef Cihal
    Sep 5, 2005
  4. warpy

    urllib2 urlopen timeout option

    warpy, Jan 3, 2009, in forum: Python
    Replies:
    0
    Views:
    1,873
    warpy
    Jan 3, 2009
  5. Mark Probert

    Timeout::timeout and Socket timeout

    Mark Probert, Oct 6, 2004, in forum: Ruby
    Replies:
    1
    Views:
    1,280
    Brian Candler
    Oct 6, 2004
Loading...

Share This Page