deliberate versus os socket timeout

[Robin Becker]

While messing about with some deliberate socket timeout code I got an unexpected
timeout after 20 seconds when my code was doing socket.setdefaulttimeout(120).

Closer inspection revealed that this error in fact seemed to come from the os
(in this case windows xp).

By inspection of test cases the error.reason from the deliberate socket timeout
looks like
'timed out'
whereas the windows caused timeout error.reason looks like
'(10060 operation timed out)'

it would be nice to know if that is in fact true and whether there is any way to
do the attribution of errors more sensibly.

Both of these seem to cause urllib2.URLError and presumably appear somewhere in
the socket code.

It might be nice if the deliberate timeout could be something like 'timed out
deliberately after xxx seconds'.

More importantly is there anything I can do to avoid these wrong os inspired
timeouts? I'm just using urllib2 to read from a remote site inside of a cgi
script. I'm not sure it's a big problem, but I have no idea what could cause it.

The code I'm using is based on one of jjlee's recipes

def httpGet(self):
import urllib2, socket
from xml.sax.saxutils import escape
oto = socket.getdefaulttimeout()
try:
socket.setdefaulttimeout(self.timeout)
self.url = url = self.makeUrl(self.params)
try:
self.response = r = urllib2.urlopen(url)
except urllib2.URLError, e:
if hasattr(e, 'reason'):
msg = 'HTTP error '+str(e.reason)
if 'time' in msg:
msg += '(our timeout=%s)' % socket.getdefaulttimeout()
return 1,escape(msg)
elif hasattr(e, 'code'):
return 1, escape('The server couldn\'t fulfill the
request.\nError code: '+str(e.code))
else:
# everything is fine
h = self.headers = {}
........
return 0,......
finally:
socket.setdefaulttimeout(oto)

 

[John J. Lee]

While messing about with some deliberate socket timeout code I got an
unexpected timeout after 20 seconds when my code was doing
socket.setdefaulttimeout(120).

Closer inspection revealed that this error in fact seemed to come from
the os (in this case windows xp).

By inspection of test cases the error.reason from the deliberate
socket timeout looks like
'timed out'
whereas the windows caused timeout error.reason looks like
'(10060 operation timed out)'

it would be nice to know if that is in fact true and whether there is
any way to do the attribution of errors more sensibly.

What do you mean "is in fact true"? Is what true? That the reason is
always formatted in those ways when those particular errors occur?
Not sure there's any reason to rely on that: I'd expect .reason to be
an exception object, so (reading the docs for module socket)
e.reason.args[0] should be the error code if one is available, in this
case errno.ETIMEDOUT (i.e. WSAETIMEDOUT, which means the timeout
occurred before a connection was established). Sometimes
e.reason.args will be length-1, and e.reason.args[0] will be a string,
again according to the docs for module socket.

Both of these seem to cause urllib2.URLError and presumably appear
somewhere in the socket code.

It might be nice if the deliberate timeout could be something like
timed out deliberately after xxx seconds'.

By "deliberate timeout", you mean the one you asked for by calling
socket.setdefaulttimeout().

In that case, I think e.reason will be a socket.timeout instance (as
opposed to socket.error in the WSAETIMEOUT case).

More importantly is there anything I can do to avoid these wrong os
inspired timeouts? I'm just using urllib2 to read from a remote site

[...]

Please define "wrong" ;-)

Googling suggests Windows doesn't let you configure its connect
timeout, except through a registry setting:

http://www.tech-archive.net/Archive/VC/microsoft.public.vc.language/2005-09/msg00918.html


John

 

[Robin Becker]

What do you mean "is in fact true"? Is what true? That the reason is
always formatted in those ways when those particular errors occur?
Not sure there's any reason to rely on that: I'd expect .reason to be
an exception object, so (reading the docs for module socket)
e.reason.args[0] should be the error code if one is available, in this
case errno.ETIMEDOUT (i.e. WSAETIMEDOUT, which means the timeout
occurred before a connection was established). Sometimes
e.reason.args will be length-1, and e.reason.args[0] will be a string,
again according to the docs for module socket.

well I suspect there's a whole twisty maze of stuff to do to reliably detect
whether a timeout was deliberate or not so I'll pass on that.

..........

More importantly is there anything I can do to avoid these wrong os
inspired timeouts? I'm just using urllib2 to read from a remote site

[...]

Please define "wrong" ;-)

well if I ask to be timed out after 120 seconds I don't expect a timeout error
after 20 so I would define the 20 second case to be somehow wrong or unexpected.

Googling suggests Windows doesn't let you configure its connect
timeout, except through a registry setting:

http://www.tech-archive.net/Archive/VC/microsoft.public.vc.language/2005-09/msg00918.html

the implication of windows providing its own timeout seems to make the socket
defaulttimeout irrelevant since I can have no per socket control over it. I
think this came up before and I just forgot about it mumble mumble