Catch script hangs

[Bakes]

Due to an ftp server issue, my python script sometimes hangs whilst
downloading, unable to receive any more data. Is there any way that I
could have python check, maybe through a thread or something, whether
it has hanged (or just, if it's still active after 10 seconds, stop
it?). I have looked at threading but there does not seem to be a stop
method on threading, which is a problem. Could the lower level thread
module be a solution?

I was thinking something like:
thread:
spawn threaded timer, if it gets to 20, close the thread
attempt download of file
cancel threaded timer.
exit thread.

would that work at all?

 

[exarkun]

Due to an ftp server issue, my python script sometimes hangs whilst
downloading, unable to receive any more data. Is there any way that I
could have python check, maybe through a thread or something, whether
it has hanged (or just, if it's still active after 10 seconds, stop
it?). I have looked at threading but there does not seem to be a stop
method on threading, which is a problem. Could the lower level thread
module be a solution?

No. There are a great many issues which arise when trying to forcibly
terminate a thread. Python doesn't expose this functionality because
most platforms don't provide it in a safe or reliable way.

You could give Twisted's FTP client a try. Since it isn't blocking, you
don't need to use threads, so it's easy to have a timeout.

You could also explore solutions based on signal.alarm(). A single-
threaded signal-based solution has some issues as well, but not nearly
as many as a thread-based solution.

Jean-Paul

 

[Sean DiZazzo]

Due to an ftp server issue, my python script sometimes hangs whilst
downloading, unable to receive any more data. Is there any way that I
could have python check, maybe through a thread or something, whether
it has hanged (or just, if it's still active after 10 seconds, stop
it?). I have looked at threading but there does not seem to be a stop
method on threading, which is a problem. Could the lower level thread
module be a solution?

I was thinking something like:
thread:
spawn threaded timer, if it gets to 20, close the thread
attempt download of file
cancel threaded timer.
exit thread.

would that work at all?

I messed around, and came up with this:


import ftplib, socket

class MyFTP(ftplib.FTP):
def storbinary(self, command, f, blocksize=8192, callback=None,
timeout=0):
"""
Override the storbinary method to make the socket.connection()
object available
outside the object, and to set the timeout of the socket
"""
self.voidcmd('TYPE I')
self.conn = self.transfercmd(command)
self.conn.settimeout(timeout)
while 1:
buf = f.read(blocksize)
if not buf:
break
self.conn.sendall(buf)
if callback: callback(buf)
self.conn.close()


ftp = MyFTP("ftp.host","user","password")


fname = "FireOnTheMountain.mov"
timeout = 1


try:
with open(fname, 'r') as fi:
#send the extra timeout arg
ftp.storbinary("STOR %s" % fname, fi, timeout=timeout)
except socket.timeout:
print "TIMED OUT!"
#if we shutdown the socket connection, it seems to properly stop
the transfer
ftp.conn.shutdown(socket.SHUT_RDWR)

ftp.quit()


I think that should at least let you set a timeout for sending each
packet that will end the connection gracefully if the timeout is
reached. Not sure if it raises any other problems.

~Sean