Bidrectional Subprocess Communication

[Emanuele D'Arrigo]

Hi everybody,

I'm trying to replicate the positive results of the Client/Server
scripts from the thread "Bidirectional Networking", but this time
using a Process/SubProcess architecture.

The SubProcess, acting as a server, is working just fine. But the
Process executing the SubProcess, the client, somehow doesn't hear any
of the standard output from the SubProcess. What am I missing?

Below you can find the two short pieces of code. Thanks for your help!

Manu

-------------------------------------------
# serverTest.py
import sys

print("Starting Server!")

while True:
data = sys.stdin.readline().strip()
if(data):
print("SERVER RECV: '" + data + "'")

if(data == "Stop!"):
break

print("Server Stopped!")

-------------------------------------------
#clientTest.py

import sys
import threading
from time import sleep
from subprocess import Popen, PIPE

## Server Thread
class ListenerThread(threading.Thread):

def __init__(self, inChannel):
threading.Thread.__init__(self)
self.inChannel = inChannel

def run(self):
while True:
data = self.inChannel.readline().strip()
print("serverTest.py says: " + data)

print("Starting Client!")

server = Popen("python serverTest.py", stdin=PIPE, stdout=PIPE)

listenerThread = ListenerThread(server.stdout)
listenerThread.setDaemon(True)
listenerThread.start()

server.stdin.write("Something very meaningful!\n")
server.stdin.write("Stop!")

print("Client Stopped!")

 

[greg]

(Pipes don't work the same as sockets, although unix-like systems try
hard to hide the differences...)

BSD-based unixes implement pipes using socketpair(), so
pipes actually *are* sockets (or at least they used to be,
not sure whether it's still true).

 

[Emanuele D'Arrigo]

- you have to close server.stdin when you don't have more data to send.
The server will see an end-of-file and knows it has to exit the loop.
Same thing on the client side.

Hi Gabriel, thanks for modifying the code to make it work. I've just
tried tinkering with it to see how far it would go. On your two
statements above: does this means that any data must be sent in one
batch and then the subprocess must shut down? What I was trying to
simulate is a client/server relationship through a subprocess, where
the server (the subprocess) keeps listening and the client sends data
when it wants to (and eventually viceversa). But when the
server.stdin.close() statement is issued, the pipe is closed for good
and can't be reopened (can it?).

- you have to wait until the server answers, else it will get a "broken
pipe" error or similar.

So, if I want to interrogate the subprocess multiple times I must end
and restart the ListenerThread multiple times then?

In the meantime, I better brush up on my streams... =)

Thanks for your help!

Manu

 

[Gabriel Genellina]

Hi Gabriel, thanks for modifying the code to make it work. I've just
tried tinkering with it to see how far it would go. On your two
statements above: does this means that any data must be sent in one
batch and then the subprocess must shut down? What I was trying to
simulate is a client/server relationship through a subprocess, where
the server (the subprocess) keeps listening and the client sends data
when it wants to (and eventually viceversa). But when the
server.stdin.close() statement is issued, the pipe is closed for good
and can't be reopened (can it?).

No, not at all. You can keep writing things to the pipe - as long as the
read side keeps reading, there is no limit on how much data you can send.
Just make sure you close the writing side of the pipe when you have no
more data to send, to signal the other side it's OK to exit the read loop.

So, if I want to interrogate the subprocess multiple times I must end
and restart the ListenerThread multiple times then?

No, I mean, since your example is bidirectional, the parent process must
still be alive and able to read when the subprocess replies. Given this
sequence:

parent writes "hi sub!"
child reads "hi sub!"
child writes "hi dad!"
parent reads "hi dad!"

if the parent just exits after sending "hi sub!", the child will get an
error when replying (I think it says "invalid handle" on Windows, "broken
pipe" on Linux, or something like this).

 

[Gabriel Genellina]

BSD-based unixes implement pipes using socketpair(), so
pipes actually *are* sockets (or at least they used to be,
not sure whether it's still true).

But not on Linux; a visible difference is that pipes are half-duplex on
Linux (it seems that's enough for POSIX). I don't know for sure how
they're implemented on Windows but they seem to be file system objects
(they use functions like CreateFile, ReadFile, WriteFile, etc.)