Question about xmlrpc and threading

[David Hirschfield]

An xmlrpc client/server app I'm writing used to be super-simple, but now
threading has gotten into the mix.

On the server side, threads are used to process requests from a queue as
they come in.
On the client side, threads are used to wait on the results of requests
to the server.

So the question is: how thread-safe is python xmlrpc? If the client
makes a request of the server by calling:

result = server.doSomething()

and while that is waiting in the background thread to complete, the
client calls another:

result = server.doSomethingElse()

will they interfere with each other? Right now I'm avoiding this problem
by queueing up calls to the server to be processed sequentially in the
background. But I'd prefer to allow requests to go in parallel. Should I
just make a new connection to the server for each request?

Any advice appreciated,
-David

 

[Martin P. Hellwig]

An xmlrpc client/server app I'm writing used to be super-simple, but now
threading has gotten into the mix.

On the server side, threads are used to process requests from a queue as
they come in.
On the client side, threads are used to wait on the results of requests
to the server.

So the question is: how thread-safe is python xmlrpc? If the client
makes a request of the server by calling:

result = server.doSomething()

and while that is waiting in the background thread to complete, the
client calls another:

result = server.doSomethingElse()

will they interfere with each other? Right now I'm avoiding this problem
by queueing up calls to the server to be processed sequentially in the
background. But I'd prefer to allow requests to go in parallel. Should I
just make a new connection to the server for each request?

Any advice appreciated,
-David

I'm not sure if I quite understand what you mean by "interfere with each
other" but namespaces also apply to xmlrpc servers.
But let's give a coding example say I have this async server created:

===
from SocketServer import ThreadingMixIn
from SimpleXMLRPCServer import SimpleXMLRPCServer
from time import sleep

# Overriding with ThreadingMixIn to create a async server
class txrServer(ThreadingMixIn,SimpleXMLRPCServer): pass

# the Test classs
class Test(object):
def __init__(self):
self.returnValue = "Something 1"

def doSomething(self):
sleep(5)
return self.returnValue

def doSomethingElse(self,value):
self.returnValue=value
return self.returnValue


# setup server and bind to the specified port
server = txrServer(('localhost', 8080))

# register test class
server.register_instance(Test())

# start the serving
server.serve_forever()
===

And I call the functiondoSomething() will return me:"Else What!" instead of "Something 1"
because it's a shared namespace of "self".


Now if I modify my example to this:

===
from SocketServer import ThreadingMixIn
from SimpleXMLRPCServer import SimpleXMLRPCServer
from time import sleep

# Overriding with ThreadingMixIn to create a async server
class txrServer(ThreadingMixIn,SimpleXMLRPCServer): pass

# the Test classs
class Test(object):
def __init__(self):
pass

def doSomething(self):
returnValue = "Something 1"
sleep(5)
return returnValue

def doSomethingElse(self,value):
returnValue=value
return returnValue


# setup server and bind to the specified port
server = txrServer(('localhost', 8080))

# register test class
server.register_instance(Test())

# start the serving
server.serve_forever()
===
Will have no effect on returnValue of doSomething() because they are not
shared.

But say that I add the sleep part to doSomethingElse() and callWhat do you think will happen? Will the 2nd call overwrite the firsts
calls variable?

I'm not going to spoil it any further ;-), please try the snippets out
for yourself (I bet you'll be pleasantly surprised).

hth

 

[David Hirschfield]

I definitely didn't make it clear enought what I was talking about. I
know all about the thread issues as far as namespaces go and that sort
of thing.
Let me try and be clearer:

Forget how my xmlrpc server is implemented, it doesn't matter for this
question. Just imagine it works and will process requests and return
results.
I have one connection to the server from my client, created via:

import xmlrpclib
server = xmlrpclib.ServerProxy(...)

So now my client calls:

result = server.doSomething()

and that will take some time to process on the server, so that part of
the client code blocks waiting for the result.
Now, some other thread in the client calls:

result = server.doSomethingElse()

while doSomething() is still processing on the server and the client is
still waiting for a result.
My question was whether this is allowed? Can two calls be made via the
same ServerProxy instance while a request is already underway?

Clearer?
-Dave

 

[Martin P. Hellwig]

David Hirschfield wrote:

My question was whether this is allowed? Can two calls be made via the
same ServerProxy instance while a request is already underway?

Clearer?
-Dave

<cut>

Much, and my preliminary answer is, I have no clue
But knowing that python will throw an exceptions when something is not
allowed and that we can analyze if the logic of the program breaks we do
can build a test case!

I've taken the opportunity, but I might be as wrong was I am right (and
still won't know which one it was):

So, I expect that when something un-allowed happens it will throw an
exception and otherwise that the workflow on the server should resemble
the work flow on the client:

So i created an async server with only one function (which is even worse
then your question):
def doSomething(self,threadID):
randomWait= random.randrange(0,10)
time.sleep(randomWait)
print("Thread %s has slept for %s seconds" % threadID,randomWait))
returnValue = "Thread %s has slept for %s seconds" % (threadID,randomWait)
return returnValue

and a threaded client:
class testThread(threading.Thread):
def run(self):
xmlrpclib.server = xmlrpclib.ServerProxy("http://localhost:8080")
x=xmlrpclib.server
print(x.doSomething(self.getName()))

for i in range(100):
testThread().start()

So when I run that I except that the output on the server is the same as
the one on the client with an error margin within the second range
(which would indicate that a certain thread has been scheduled a little
earlier then the other), guess what the output of the two was?

Well I say this much, most of them where in sync (within the error
margin) so I conclude that your question is answered with , yes you can
have a call when another one is processed without intervening with each
other( if you want that).