named pipe and Linux

[akineko]

Hello everyone,

I'm trying to use named pipes to fuse a Python program and a C
program.
One side creates pipes using os.mkfifo() and both sides use the same
named pipes (one side reads, another side writes). The read side uses
select.select() to wait for incoming messages and read the message
when select.select() says it is ready.

The length of the message is unknown to the read side.
I cannot use file.read() because it will block waiting for an EOF.
I cannot use file.readline() because how many lines have arrived is
unknown.
So, I needed to use os.read() with the exact number of characters to
read.

Under Solaris environment, os.fstat() provides the exact size of the
message that has arrived.
Thus, two processes can communicate each other through the named pipes
without blocking.

However, the above scheme didn't work under Linux.
Linux os.fstat() returns size=0 even the message is pending.
(I think Linux buffers the message in memory while Solaris buffers the
message in a file system)

My question is, how can I make the named pipe scheme work under Linux?
Is there any way to read the message without getting blocked?

I know this is more Linux question than Python question but I believe
many Python programmers are strong Linux programmers.

Any suggestions will be appreciated.

Best regards,
Aki Niimura

 

[Lawrence D'Oliveiro]

In message <5b5157dd-

The length of the message is unknown to the read side.

I think you want a message-based, not a stream-based, IPC mechanism. Look at
the docs on msgctl, msgget, msgop and so on.

 

[bobicanprogram]

Hello everyone,

I'm trying to use named pipes to fuse a Python program and a C
program.
One side creates pipes using os.mkfifo() and both sides use the same
named pipes (one side reads, another side writes). The read side uses
select.select() to wait for incoming messages and read the message
when select.select() says it is ready.

The length of the message is unknown to the read side.
I cannot use file.read() because it will block waiting for an EOF.
I cannot use file.readline() because how many lines have arrived is
unknown.
So, I needed to use os.read() with the exact number of characters to
read.

Under Solaris environment, os.fstat() provides the exact size of the
message that has arrived.
Thus, two processes can communicate each other through the named pipes
without blocking.

However, the above scheme didn't work under Linux.
Linux os.fstat() returns size=0 even the message is pending.
(I think Linux buffers the message in memory while Solaris buffers the
message in a file system)

My question is, how can I make the named pipe scheme work under Linux?
Is there any way to read the message without getting blocked?

I know this is more Linux question than Python question but I believe
many Python programmers are strong Linux programmers.

Any suggestions will be appreciated.

Best regards,
Aki Niimura

The SIMPL open source project (http://www.icanprogram.com/simpl)
provides an ultra lightweight toolkit useful for joining Python
programs to C programs using a Send/Receive/Reply mechanism first
pioneered by QNX. SIMPL uses a fifo synchronized shared memory
scheme for the local message pass. Through the use of generic
surrogate pairs SIMPL processes can be distributed across TCP/IP or
RS232 (think radio modem) networks often times without any changes or
recompiles. Through the use of another type of generic surrogate a
Python module running on a nonLinux OS can communicate transparently
with a module running on a Linux box.

A SIMPL application consists of two or more SIMPL modules
collaborating in this way. SIMPL modules can be written in any
number of languages including Python, C, C++, Tcl/Tk or JAVA. More
importantly SIMPL modules written in different languages can be mixed
in a given SIMPL application.

There is a Sudoku puzzle solver example here:

http://www.icanprogram.com/simplBook/simplBook.self.html

bob
SIMPL project facilitator