Best method for inter process communications

J

JamesHoward

I am looking for a way of performing inter process communication over
XML between a python program and something else creating XML data.

What is the best way of performing this communication? I could bind a
socket to localhost and perform the data transfer that way, but that
seems inefficient due to the addition of TCP/IP or UDP/IP overhead.
Is there a way to stream data via a custom datastream (I.E. not STDIO,
STDERR, etc)?

Thanks in advance,
Jim Howard
 
M

marduk

I am looking for a way of performing inter process communication over
XML between a python program and something else creating XML data.

What is the best way of performing this communication? I could bind a
socket to localhost and perform the data transfer that way, but that
seems inefficient due to the addition of TCP/IP or UDP/IP overhead.
Is there a way to stream data via a custom datastream (I.E. not STDIO,
STDERR, etc)?

In *nix, you could also use a named pipe for cheap RPC (os.mkfifo()).

-a
 
S

Steve Holden

JamesHoward said:
I am looking for a way of performing inter process communication over
XML between a python program and something else creating XML data.

What is the best way of performing this communication? I could bind a
socket to localhost and perform the data transfer that way, but that
seems inefficient due to the addition of TCP/IP or UDP/IP overhead.
Is there a way to stream data via a custom datastream (I.E. not STDIO,
STDERR, etc)?
Nothing that portable across different platforms, I suspect.

However I see no reason why you can't use pipelines in Unix (in other
words what's wrong with using stdin/stdout?), and Windows offers
something called a named pipe. You could even use shared memory if you
wanted.

Given the radically inefficient representations that XML typically
requires, however, I think that worrying about localhost socket overhead
is unnecessary: if your application was *that* critical you wouldn't be
using XML in the first place.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
 
G

Grant Edwards

I am looking for a way of performing inter process communication over
XML between a python program and something else creating XML data.

What is the best way of performing this communication? I could bind a
socket to localhost and perform the data transfer that way, but that
seems inefficient due to the addition of TCP/IP or UDP/IP overhead.
Is there a way to stream data via a custom datastream (I.E. not STDIO,
STDERR, etc)?

You could use a named pipe or a Unix domain socket. The nice
thing about an IP socket is that you get network transparancy:
the two programs can be moved to two different machines.
 
J

John Nagle

Steve said:
JamesHoward wrote:
Given the radically inefficient representations that XML typically
requires, however, I think that worrying about localhost socket overhead
is unnecessary: if your application was *that* critical you wouldn't be
using XML in the first place.

Agreed. UNIX/Linux/Python interprocess communication is inefficient
compared to, say, QNX or Minix 3. But where Python and XML are involved,
you've already lost 2-3 orders of magnitude in performance over something
like MsgSend in QNX, or even OpenRPC. So don't worry about the socket
overhead.

John Nagle
 
J

JamesHoward

Thanks for the updates. I think I will try named processes first, but
may just end up using local sockets in the end and not worry about the
overhead.

Jim Howard
 
H

Hendrik van Rooyen

Steve Holden said:
Given the radically inefficient representations that XML typically
requires, however, I think that worrying about localhost socket overhead
is unnecessary: if your application was *that* critical you wouldn't be
using XML in the first place.

I heard a rumour once that a "hello world" string takes something like
10k bytes in XMLRPC - have never bothered to find out if its BS...

- Hendrik
 
M

Marc 'BlackJack' Rintsch

I heard a rumour once that a "hello world" string takes something like
10k bytes in XMLRPC - have never bothered to find out if its BS...

Just try it out:

In [8]: import xmlrpclib

In [9]: a = xmlrpclib.dumps(('hello world',))

In [10]: len(a)
Out[10]: 80

In [11]: print a
<params>
<param>
<value><string>hello world</string></value>
</param>
</params>

Even with some additional boilerplate like XML declaration etc. there is
still a little bit room until 10 kB are reached. :)

Ciao,
Marc 'BlackJack' Rintsch
 
I

Irmen de Jong

JamesHoward said:
I am looking for a way of performing inter process communication over
XML between a python program and something else creating XML data.

What is that "something else"?
--irmen
 
H

Hendrik van Rooyen

I heard a rumour once that a "hello world" string takes something like
10k bytes in XMLRPC - have never bothered to find out if its BS...

Just try it out:

In [8]: import xmlrpclib

In [9]: a = xmlrpclib.dumps(('hello world',))

In [10]: len(a)
Out[10]: 80

In [11]: print a
<params>
<param>
<value><string>hello world</string></value>
</param>
</params>

Even with some additional boilerplate like XML declaration etc. there is
still a little bit room until 10 kB are reached. :)

LOL - so one could say it was a bit of an exaggeration...

Thanks - Hendrik
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top