xmlrpc slow in windows 7 if hostnames are used

[News123]

Hi,

I wrote a small xmlrpc client on Windows 7 with python 2.6

srv = xmlrpclib.Server('http://localhost:80')

I was able to perform about 1 rpc call per second


After changing to
srv = xmlrpclib.Server('http://127.0.0.1:80')

I was able to perform about 10 to 16 rpc calls per second.

So it seems, that under windows 7 the host name lookup occurs for every
RPC call (which is of course necessary for long running processes, as
the target host might change it's ip during the scripts execution)

For my use cases however I could live with one IP-address lookup at the
beginning of the script.


I wonder now about how I could deal with virtual web servers.

( a web server running with one ip-adress and one port, which behaves
differntly depending on the host name of the url request)

I'm just curious, as currently I don't have to access virtual servers
via xmlrpc,

Is there any way to 'pre-lookup the IP address', and to keep the host
name in the http POST request?


thanks for any ideas



N

 

[Gabriel Genellina]

I wrote a small xmlrpc client on Windows 7 with python 2.6

srv = xmlrpclib.Server('http://localhost:80')

I was able to perform about 1 rpc call per second


After changing to
srv = xmlrpclib.Server('http://127.0.0.1:80')

I was able to perform about 10 to 16 rpc calls per second.

So it seems, that under windows 7 the host name lookup occurs for every
RPC call

Not necesarily. There is another difference: 127.0.0.1 is an IPv4 address,
localhost maps to both IPv4 and IPv6 addresses :1)

I vaguely remember a problem with that - IPv6 is tried first, doesn't
work, only then IPv4, and that slows down the whole process.
Try disabling completely the IPv6 stack, if you don't need it.

 

[News123]

Yhanks a lot I'll check whether this is the root cause.

Currently my machine could live without IPV6


bye

N

 

[Jean-Michel Pichavant]

Yhanks a lot I'll check whether this is the root cause.

Currently my machine could live without IPV6


bye

N

Or you can simply use an explicit external address. Most of the time
xmlRPC server/clients are used between distant machines.
If your are using localhost for test purpose, then binding your server
on its external IP instead of the local one could solve your problem
(wihtout removing the IPV6 stack).

import socket

# server
server = SimpleXMLRPCServer((socket.gethostname(), 5000),
logRequests=False, allow_none=True)


# client
xmlrpclib.ServerProxy("http://%s.yourdomain.com:%s" %
(socket.gethostname(), 5000))

JM

PS : please don't top post )
PS : just wondering if using the port 80 is legal

 

[News123]

Hi Gabriel,

Not necesarily. There is another difference: 127.0.0.1 is an IPv4
address, localhost maps to both IPv4 and IPv6 addresses :1)

I vaguely remember a problem with that - IPv6 is tried first, doesn't
work, only then IPv4, and that slows down the whole process.
Try disabling completely the IPv6 stack, if you don't need it.

How can I completely disable IP6.

I went to my network device and disabled IPV6 under properties. The
XMRPC calls were stull slow.

bye


N

 

[News123]

Hi JM,

Or you can simply use an explicit external address. Most of the time
xmlRPC server/clients are used between distant machines.
If your are using localhost for test purpose, then binding your server
on its external IP instead of the local one could solve your problem
(wihtout removing the IPV6 stack).

import socket

# server
server = SimpleXMLRPCServer((socket.gethostname(), 5000),
logRequests=False, allow_none=True)


# client
xmlrpclib.ServerProxy("http://%s.yourdomain.com:%s" %
(socket.gethostname(), 5000))

Well This was exactly my question.
for virtual web servers I cannot just use the IP-address.
some XMLRPC servers do need the histname within the HTTP-POST request.

if I just replaced the hostname with the IP addres, then certain servers
would not be accessable.

I had to use the IP-address for connecteing, but to pass the hostname in
the HTTP-POST request.

I wondered how to convince puthon's SimpleXMLRPCServer (or any other
standard python xmlrpc server), such, that I can obtain above mentioned
goal.


bye

N

PS : just wondering if using the port 80 is legal

If nothing else on the host runs on port 80 the answer is yes.

 

[Steve Holden]

Hi Gabriel,





How can I completely disable IP6.

I went to my network device and disabled IPV6 under properties. The
XMRPC calls were stull slow.

There's not a lot of point taking actions like disabling IPv6 until you
actually know what the cause of the problem is (though I suppose a few
likely-sounding actions are worth taking if they don't require much
effort). Sounds to me like you need to get WireShark or some similar
protocol analyzer on the network and determine what traffic is passing
across the network.

regards
Steve

 

[Steve Holden]

Hi Gabriel,





How can I completely disable IP6.

I went to my network device and disabled IPV6 under properties. The
XMRPC calls were stull slow.

There's not a lot of point taking actions like disabling IPv6 until you
actually know what the cause of the problem is (though I suppose a few
likely-sounding actions are worth taking if they don't require much
effort). Sounds to me like you need to get WireShark or some similar
protocol analyzer on the network and determine what traffic is passing
across the network.

regards
Steve

 

[Jean-Michel Pichavant]

Hi JM,




Well This was exactly my question.
for virtual web servers I cannot just use the IP-address.
some XMLRPC servers do need the histname within the HTTP-POST request.

a valid IP address would make it

if I just replaced the hostname with the IP addres, then certain servers
would not be accessable.

I had to use the IP-address for connecteing why not using the host names?
, but to pass the hostname in
the HTTP-POST request.

I wondered how to convince puthon's SimpleXMLRPCServer (or any other
standard python xmlrpc server), such, that I can obtain above mentioned
goal.


bye

N

I'm puzzled.
Unless my english is failing me, everything would be solved using
hostnames if I follow you. Why don't you do that ?
I am no network/IP guru, but it sounds very weird to have requests
rejected when using IP addresses. Are you sure your host names are
resolved with the same IPM address you are using ?


JM

 

[Gabriel Genellina]

En Sat, 06 Feb 2010 22:15:48 -0300, Jean-Michel Pichavant

I'm puzzled.
Unless my english is failing me, everything would be solved using
hostnames if I follow you. Why don't you do that ?
I am no network/IP guru, but it sounds very weird to have requests
rejected when using IP addresses. Are you sure your host names are
resolved with the same IPM address you are using ?

HTTP 1.1 requires a Host: header field; this way, multiple web servers may
share the same IP address. So you can't identify a host by its IP alone;
the host name is required. This was devised in order to save IPv4
addresses; LACNIC (the Latin America addresses register) does not assign
addresses to ISP's based solely on web hosting anymore - they MUST share
existing IPs. And I think a similar policy is used on other regions.

 

[News123]

Hi JM,

a valid IP address would make it

What I meant is: Window 7 seems to be waaaayyyy slower if an xmlrpc
client uses a host name ('localhost') than an IP-address ('127.0.0.1).
the speed difference is between 1 request per second or 10-20 requests
per second.


many computers host multiple web servers under the same IP-address.
( see for example http://httpd.apache.org/docs/1.3/vhosts/name-based.html )

So I cannot just replace a host name with an IP-address and expect
to receive the correct data / correct xmlrpc server.

why not using the host names?

I did not want to use the hostname due to the slowdown of
"locahost vs. 127.0.0.1" issue on my host.

You are right, that I did not verify whether this issue exists also with
external servers and I should verify this first.

I'm puzzled.
Unless my english is failing me, everything would be solved using
hostnames if I follow you. Why don't you do that ?
I am no network/IP guru, but it sounds very weird to have requests
rejected when using IP addresses. Are you sure your host names are
resolved with the same IPM address you are using ?

The request would not be rejected, but apache can (if being configured
for name based virtual hosts) look at the hostname within the TCP/IP
payload (the entire url (including the host name) is normally also in
the GET / POST request ) and decide to deliver different data (or to
return HTTP errors) depending on the hostname in the payload.


I hope this clarifies.

Your answer gave me some thoughts though:

I still have to check whether the issue really exists with external
requests other than localhost.
Also I should probably try to attack the root cause
( probably with help of ethereal or a similiar tool) instead of trying
to work around it.

I could just change the windows /etc/hosts equivalent and tell localhost
to have only an IPV4 address (perhaps this increases the performance)


On the other hand:
Some people use name based virtual servers to provide special web
services by providing a 'fake-host name' in the http request. it might
be, that the fake host name doesn't even have a DNS entry.
( Though security by obscurity is a questionable practice )

so for some weird use cases it could be worth knowing how to connect to
one IP addres and to pass a different host name in the HTTP payload when
using an xmlrpcclient


bye


N

 

[Pete Forman]

> En Sat, 06 Feb 2010 22:15:48 -0300, Jean-Michel Pichavant


> HTTP 1.1 requires a Host: header field; this way, multiple web
> servers may share the same IP address. So you can't identify a host
> by its IP alone; the host name is required. This was devised in
> order to save IPv4 addresses; LACNIC (the Latin America addresses
> register) does not assign addresses to ISP's based solely on web
> hosting anymore - they MUST share existing IPs. And I think a
> similar policy is used on other regions.

If you really want to go for speed you should be able to set the Host:
header to the name but use the IP address to make the connection.

Something else that might be slowing you down is anti-spyware or
anti-virus. Several products put a long list of blacklist sites in
the hosts file. Windows can be rather slow to process that file.