D
David Hirschfield
I've written a server-client system using XML-RPC. The server is using
the twisted.web.xmlrpc.XMLRPC class to handle connections and run
requests. Clients are just using xmlrpclib.ServerProxy to run remote
method calls.
I have a few questions about the performance of xmlrpc in general, and
specifically about increasing the speed with which remote methods return
their results to the client.
Right now the process of calling a remote method works as follows:
client:
generate some python objects
serialize those objects by cPickling them with cPickle.HIGHEST_PROTOCOL,
then wrap the pickles with xmlrpclib.Binary() so the data can be sent safely
call the remote method via a ServerProxy object using the Binary object
as the argument
server:
invoke the method and extract the pickled and Binary()'d arguments back
into the actual objects
do some work
take result objects and cPickle them and wrap them in a Binary object as
before
return the result to the client
client:
receive result and unpickle it into real data
All the above works fine...but I'm finding the following: while the
actual creation and pickling of the objects only takes a millisecond or
so, the actual time before the client call completes is a third of a
second or more.
So where's the slowdown? It doesn't appear to be in the
pickling/unpickling or object creation, so it has to be in xmlrpc
itself...but what can I do to improve that? It looks like xmlrpclib uses
xml.parsers.expat if it's available, but are there faster xml libs?
Looking at the xmlrpclib code itself, it seems to want to find either:
_xmlrpclib from the code in xmlrpclib.py:
try:
# optional xmlrpclib accelerator. for more information on this
# component, contact (e-mail address removed)
import _xmlrpclib
FastParser = _xmlrpclib.Parser
FastUnmarshaller = _xmlrpclib.Unmarshaller
except (AttributeError, ImportError):
FastParser = FastUnmarshaller = None
or it tries to find sgmlop:
#
# the SGMLOP parser is about 15x faster than Python's builtin
# XML parser. SGMLOP sources can be downloaded from:
#
# http://www.pythonware.com/products/xml/sgmlop.htm
#
Does anyone know what the performance gain from using either of those
above libraries would be?
On the other hand, maybe the slowdown is in twisted.web.xmlrpc? What
does that module use to do its work? Is it using xmlrpclib underneath?
Other xmlrpc libraries that are significantly faster that I should be
using instead?
Any help in improving my xmlrpc performance would be greatly appreciated,
-Dave
the twisted.web.xmlrpc.XMLRPC class to handle connections and run
requests. Clients are just using xmlrpclib.ServerProxy to run remote
method calls.
I have a few questions about the performance of xmlrpc in general, and
specifically about increasing the speed with which remote methods return
their results to the client.
Right now the process of calling a remote method works as follows:
client:
generate some python objects
serialize those objects by cPickling them with cPickle.HIGHEST_PROTOCOL,
then wrap the pickles with xmlrpclib.Binary() so the data can be sent safely
call the remote method via a ServerProxy object using the Binary object
as the argument
server:
invoke the method and extract the pickled and Binary()'d arguments back
into the actual objects
do some work
take result objects and cPickle them and wrap them in a Binary object as
before
return the result to the client
client:
receive result and unpickle it into real data
All the above works fine...but I'm finding the following: while the
actual creation and pickling of the objects only takes a millisecond or
so, the actual time before the client call completes is a third of a
second or more.
So where's the slowdown? It doesn't appear to be in the
pickling/unpickling or object creation, so it has to be in xmlrpc
itself...but what can I do to improve that? It looks like xmlrpclib uses
xml.parsers.expat if it's available, but are there faster xml libs?
Looking at the xmlrpclib code itself, it seems to want to find either:
_xmlrpclib from the code in xmlrpclib.py:
try:
# optional xmlrpclib accelerator. for more information on this
# component, contact (e-mail address removed)
import _xmlrpclib
FastParser = _xmlrpclib.Parser
FastUnmarshaller = _xmlrpclib.Unmarshaller
except (AttributeError, ImportError):
FastParser = FastUnmarshaller = None
or it tries to find sgmlop:
#
# the SGMLOP parser is about 15x faster than Python's builtin
# XML parser. SGMLOP sources can be downloaded from:
#
# http://www.pythonware.com/products/xml/sgmlop.htm
#
Does anyone know what the performance gain from using either of those
above libraries would be?
On the other hand, maybe the slowdown is in twisted.web.xmlrpc? What
does that module use to do its work? Is it using xmlrpclib underneath?
Other xmlrpc libraries that are significantly faster that I should be
using instead?
Any help in improving my xmlrpc performance would be greatly appreciated,
-Dave