Stopping SocketServer on Python 2.5

[David George]

Hi guys,

I've been developing some code for a university project using Python.
We've been working on an existing codebase, cleaning it up and removing
dead wood.

We decided to make some changes to internal message handling by using a
SocketServer, which worked great when we were using 2.6, as we could
simply call its shutdown() method when we wanted to stop it from
'serving forever'.

Unfortunately, we've now needed to downgrade to python 2.5 to
accomodate the libtorrent python bindings we need to use as part of the
project.

So, my question is, is there any way to stop a SocketServer that's been
told to server forever in python 2.5?

Thanks in advance.

 

[Mark Tolonen]

Hi guys,

I've been developing some code for a university project using Python.
We've been working on an existing codebase, cleaning it up and removing
dead wood.

We decided to make some changes to internal message handling by using a
SocketServer, which worked great when we were using 2.6, as we could
simply call its shutdown() method when we wanted to stop it from 'serving
forever'.

Unfortunately, we've now needed to downgrade to python 2.5 to accomodate
the libtorrent python bindings we need to use as part of the project.

So, my question is, is there any way to stop a SocketServer that's been
told to server forever in python 2.5?

Sure, derive a class from one of the classes in SocketServer, and override
the methods that implement the shutdown behavior in 2.6.

-Mark

 

[Falcolas]

So, my question is, is there any way to stop a SocketServer that's been
told to server forever in python 2.5?

serve_forever, in python 2.5, is simply coded as:

while 1:
self.handle_request()

So, instead of calling serve_forever, call handle_request in your own
loop with a shutdown flag. Assuming, of course, that you aren't using
the Forking server, in which case things get more interesting.

~G

 

[David George]

Sure, derive a class from one of the classes in SocketServer, and
override the methods that implement the shutdown behavior in 2.6.

-Mark

Based on what you guys have said i've had a look at the code for
serve_forever() in both 2.5 and 2.6, and tried to create my own derived
class in this manner:

class MBThreadingTCPServer(SocketServer.ThreadingTCPServer):

def __init__(self, address_tuple, handler):
SocketServer.ThreadingTCPServer.__init__(self, address_tuple, handler)
self.__serving = True

def serve_forever(self):
while self.__serving:
SocketServer.ThreadingTCPServer.handle_request(self)

def shutdown(self):
self.__serving = False

Don't worry about the MB thing, it's just related to the name of our project.

I don't think i've done this right, but i've tried to implement the
serve_forever() functionality in my derived class, and also add the
shutdown() method so i can stop it.

Does this appear to be the right way to do things?

Cheers,

Dave

 

[Falcolas]

Based on what you guys have said i've had a look at the code for
serve_forever() in both 2.5 and 2.6, and tried to create my own derived
class in this manner:

class MBThreadingTCPServer(SocketServer.ThreadingTCPServer):

    def __init__(self, address_tuple, handler):
        SocketServer.ThreadingTCPServer.__init__(self, address_tuple, handler)
        self.__serving = True

    def serve_forever(self):
        while self.__serving:
            SocketServer.ThreadingTCPServer.handle_request(self)

    def shutdown(self):
        self.__serving = False

Don't worry about the MB thing, it's just related to the name of our project.

I don't think i've done this right, but i've tried to implement the
serve_forever() functionality in my derived class, and also add the
shutdown() method so i can stop it.

Does this appear to be the right way to do things?

Cheers,

Dave

More or less what I would do, though you should be able to call
self.handle_request. It's worth noting that handle_request generally
calls a blocking socket method, which means your self.__serving check
only happens the next time it handles a request.

~G

 

[David George]

More or less what I would do, though you should be able to call
self.handle_request. It's worth noting that handle_request generally
calls a blocking socket method, which means your self.__serving check
only happens the next time it handles a request.

~G

Yes, i've just noticed that this is the problem ... i've updated the
code to work like this:

class MBTCPServer(SocketServer.TCPServer):

def serve_until_stopped(self):
self.serving = True
self.has_shutdown = False
while self.serving:
self.handle_request()
self.has_shutdown = True

def shutdown(self):
self.serving = False

Simply calling the base class constructor when i build it (i believe
this is inferred).

Again, problem here is the issue of being unable to kill the server
while it's waiting on a request. In theory, i could force it to
continue by sending some sort of junk data with the method i use to
stop the server, but that seems a bit hacky, don't you think?

Cheers,

Dave

 

[Falcolas]

Again, problem here is the issue of being unable to kill the server
while it's waiting on a request. In theory, i could force it to
continue by sending some sort of junk data with the method i use to
stop the server, but that seems a bit hacky, don't you think?

Dave,

I agree, it does.

I'm in a bit over my head at this point, but does setting
self.socket.settimeout(0.5) cause the call to get_request (and thus
self.socket.accept()) to timeout? If so, that may be your ticket,
since socket.error exceptions are already caught by the TCPServer
class.

~G

 

[Mark Tolonen]

Dave,

I agree, it does.

I'm in a bit over my head at this point, but does setting
self.socket.settimeout(0.5) cause the call to get_request (and thus
self.socket.accept()) to timeout? If so, that may be your ticket,
since socket.error exceptions are already caught by the TCPServer
class.

Here's the relevant code from Python 2.6's SocketServer.py. It uses
select() to see if a request is waiting to be serviced so handle_request
won't block. If the poll interval expires the while loop will check the
__serving flag. Not ideal, but better than requiring a client to connect
before the flag is checked. The comment lists another idea:

def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.

Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__serving = True
self.__is_shut_down.clear()
while self.__serving:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if r:
self._handle_request_noblock()
self.__is_shut_down.set()

-Mark

 

[David George]

Dave,

I agree, it does.

I'm in a bit over my head at this point, but does setting
self.socket.settimeout(0.5) cause the call to get_request (and thus
self.socket.accept()) to timeout? If so, that may be your ticket,
since socket.error exceptions are already caught by the TCPServer
class.

Here's the relevant code from Python 2.6's SocketServer.py. It uses
select() to see if a request is waiting to be serviced so
handle_request won't block. If the poll interval expires the while
loop will check the __serving flag. Not ideal, but better than
requiring a client to connect before the flag is checked. The comment
lists another idea:

def serve_forever(self, poll_interval=0.5):
"""Handle one request at a time until shutdown.

Polls for shutdown every poll_interval seconds. Ignores
self.timeout. If you need to do periodic tasks, do them in
another thread.
"""
self.__serving = True
self.__is_shut_down.clear()
while self.__serving:
# XXX: Consider using another file descriptor or
# connecting to the socket to wake this up instead of
# polling. Polling reduces our responsiveness to a
# shutdown request and wastes cpu at all other times.
r, w, e = select.select([self], [], [], poll_interval)
if r:
self._handle_request_noblock()
self.__is_shut_down.set()

-Mark

Thanks to everybody for helping me with this matter, but eventually
i've had to settle for a simpler and probably far less elegant solution
due to time constraints.

It seems that SocketServer.py in 2.6 doesn't directly rely on anything
that's in Python 2.6, so i've simply copied the code across and i'm
using it in place of the version built into Python 2.5.

I will probably return to this at a later date and try for a more
elegant solution, but right now university deadlines are looming!

Thanks all,

Dave

 

[Aahz]

Thanks to everybody for helping me with this matter, but eventually
i've had to settle for a simpler and probably far less elegant solution
due to time constraints.

It seems that SocketServer.py in 2.6 doesn't directly rely on anything
that's in Python 2.6, so i've simply copied the code across and i'm
using it in place of the version built into Python 2.5.

That's actually a reasonably elegant solution!
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong." --GvR, python-ideas, 2009-3-1