SimpleXMLRPCServer and Threading

[Achim Domma]

Hi,

is SimpleXMLRPCServer multithreaded or how does it handle multiple
clients? I want to implement a simple server which will be queried by
multiple processes for work to be done. The server will simply hold a
queue with files to process. The clients will ask for the next file.

Do I have to sync access to the queue or is the server not threaded at all?

regards,
Achim

 

[Erik Johnson]

Hi,

is SimpleXMLRPCServer multithreaded or how does it handle multiple
clients? I want to implement a simple server which will be queried by
multiple processes for work to be done. The server will simply hold a
queue with files to process. The clients will ask for the next file.

Do I have to sync access to the queue or is the server not threaded at

all?


I don't claim to know much about the internals of the module, but it
imports BaseHTTPServer which I am sure is multithreaded. I did not test
this with functions, but when you register an object in the server,
different clients are accessing the same object:
(The following example is modified from that given in the module
documentation: http://docs.python.org/lib/module-SimpleXMLRPCServer.html)


$ cat server.py
#!/usr/bin/env python

from SimpleXMLRPCServer import SimpleXMLRPCServer

# Create server
server = SimpleXMLRPCServer(("localhost", 8000))
server.register_introspection_functions()

# Register pow() function; this will use the value of
# pow.__name__ as the name, which is just 'pow'.
server.register_function(pow)

# Register a function under a different name
def adder_function(x,y):
return x + y
server.register_function(adder_function, 'add')


# Register an instance; all the methods of the instance are
# published as XML-RPC methods (in this case, just 'div').
class MyObj:
SEQ = None

def __init__(self):
self.SEQ = range(10000)
self.index = 0

def next_seq(self):
n = self.SEQ[self.index]
self.index += 1
return n

server.register_instance(MyObj())

# Run the server's main loop
server.serve_forever()


# First client calls:2


# second client calls3


I did not programmatically drive these calls at the same time to try to
force simultaneous access from separate requests, but my intuition is that
it's a pretty good bet that SimpleXMLRPCServer has taken care of re-entrant
issues. It may be instructive for you to run that test yourself and
explicitly verify requests do not interact.

Alternatively, if you really want to know for sure... Python is an
open-source project. You can dig around in SimpleXMLRPCServer.py yourself
and figure out exactly what it is doing:
<module 'SimpleXMLRPCServer' from
'/usr/lib/python2.4/SimpleXMLRPCServer.pyc'>
ej@myBox ~
$ vi /usr/lib/python2.4/SimpleXMLRPCServer.py



see also:
http://docs.python.org/lib/module-BaseHTTPServer.html
http://docs.python.org/lib/module-SocketServer.html

 

[Jeff McNeil]

This is off of memory so I apologize if I don't get all of the details right.

The base SimpleXMLRPCServer uses TCPServer as it's server component
and SimpleXMLXMLRPCRequestHandler as it's handler. The handler is a
subclass of BaseHTTPRequestHandler, which itself doesn't handle any
multithreading. You need to use one of the *MixIn classes if you want
to handle concurrent requests.

-Jeff

Hi,

is SimpleXMLRPCServer multithreaded or how does it handle multiple
clients? I want to implement a simple server which will be queried by
multiple processes for work to be done. The server will simply hold a
queue with files to process. The clients will ask for the next file.

Do I have to sync access to the queue or is the server not threaded at

all?


I don't claim to know much about the internals of the module, but it
imports BaseHTTPServer which I am sure is multithreaded. I did not test
this with functions, but when you register an object in the server,
different clients are accessing the same object:
(The following example is modified from that given in the module
documentation: http://docs.python.org/lib/module-SimpleXMLRPCServer.html)


$ cat server.py
#!/usr/bin/env python

from SimpleXMLRPCServer import SimpleXMLRPCServer

# Create server
server = SimpleXMLRPCServer(("localhost", 8000))
server.register_introspection_functions()

# Register pow() function; this will use the value of
# pow.__name__ as the name, which is just 'pow'.
server.register_function(pow)

# Register a function under a different name
def adder_function(x,y):
return x + y
server.register_function(adder_function, 'add')


# Register an instance; all the methods of the instance are
# published as XML-RPC methods (in this case, just 'div').
class MyObj:
SEQ = None

def __init__(self):
self.SEQ = range(10000)
self.index = 0

def next_seq(self):
n = self.SEQ[self.index]
self.index += 1
return n

server.register_instance(MyObj())

# Run the server's main loop
server.serve_forever()


# First client calls:2


# second client calls3


I did not programmatically drive these calls at the same time to try to
force simultaneous access from separate requests, but my intuition is that
it's a pretty good bet that SimpleXMLRPCServer has taken care of re-entrant
issues. It may be instructive for you to run that test yourself and
explicitly verify requests do not interact.

Alternatively, if you really want to know for sure... Python is an
open-source project. You can dig around in SimpleXMLRPCServer.py yourself
and figure out exactly what it is doing:
<module 'SimpleXMLRPCServer' from
'/usr/lib/python2.4/SimpleXMLRPCServer.pyc'>
ej@myBox ~
$ vi /usr/lib/python2.4/SimpleXMLRPCServer.py



see also:
http://docs.python.org/lib/module-BaseHTTPServer.html
http://docs.python.org/lib/module-SocketServer.html

 

[Laszlo Nagy]

This is off of memory so I apologize if I don't get all of the details right.

The base SimpleXMLRPCServer uses TCPServer as it's server component
and SimpleXMLXMLRPCRequestHandler as it's handler. The handler is a
subclass of BaseHTTPRequestHandler, which itself doesn't handle any
multithreading. You need to use one of the *MixIn classes if you want
to handle concurrent requests.

Does it mean that, if I subclass from ThreadingMixIn then my exported
functions will be called from different threads? I created many multi
threaded servers, based on TCPServer, but I'm not sure about
XMLRPCServer. I used to create my own handler class and in that case, a
new handler instance was created for each request, then a new thread was
created and handle_request was called from that new thread. I guess that
in this case, a new SimpleXMLXMLRPCRequestHandler is created for each
request and a new thread is started.... but I'm not sure. Please help us
out.

By the way, we should have __server__ examples in the documentation of
xmlrpclib, right? What is the right way of contributing an example?

Best,

Laszlo