open/print or sys commands to write to a named pipe?

[Jim Mozley]

I need to send messages to an application whose interface for this
purpose is a named pipe (fifo).

I have a script to write to the pipe that is executed (via swatch)
whenever a certain syslog message is pattern matched. They problem is
that when I receive a group of messages nearly simultaneously I
sometimes see the resulting output in the application as one message.

E.g. the messages

"help me"
"help me too"
"and me"

should appear as three separate messages but on rare occations I get

"help me help me too and me"

shown in the application as one message.

I current do the following to write to the pipe:

open (PIPE, "> $pipe") or die "Cannot open pipe $!";
flock(PIPE,2);
print PIPE "$cmd";
close PIPE or die "Cannot close pipe $!";

I was wondering if I should use the following code below (taken from an
example I saw for writing to a pipe that I propose to put in a package):

my $fh = new IO::File;
sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
or croak "could not open $cmdfile for writing: $!";
flock( $fh, LOCK_EX );
seek( $fh, 0, SEEK_END );
my $numofbytes = syswrite( $fh, $cmd, length($cmd) );
flock( $fh, LOCK_UN );
carp "No data written!" unless $numofbytes;

Is one better than the other for this purpose?

Thanks,

Jim

 

[Ben Morrow]

I need to send messages to an application whose interface for this
purpose is a named pipe (fifo).

I have a script to write to the pipe that is executed (via swatch)
whenever a certain syslog message is pattern matched. They problem is
that when I receive a group of messages nearly simultaneously I
sometimes see the resulting output in the application as one message.

E.g. the messages

"help me"
"help me too"
"and me"

should appear as three separate messages but on rare occations I get

"help me help me too and me"

shown in the application as one message.

I current do the following to write to the pipe:

open (PIPE, "> $pipe") or die "Cannot open pipe $!";
flock(PIPE,2);

AAARGH! Don't do that! Use the constants defined in Fcntl.pm.

use Fcntl qw/:flock/;

flock PIPE, LOCK_EX;

print PIPE "$cmd";
close PIPE or die "Cannot close pipe $!";

I was wondering if I should use the following code below (taken from an
example I saw for writing to a pipe that I propose to put in a package):

my $fh = new IO::File;
sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
or croak "could not open $cmdfile for writing: $!";
flock( $fh, LOCK_EX );
seek( $fh, 0, SEEK_END );

What? Pipes aren't seekable...

my $numofbytes = syswrite( $fh, $cmd, length($cmd) );
flock( $fh, LOCK_UN );
carp "No data written!" unless $numofbytes;

Is one better than the other for this purpose?

No. The second appears to have been written by a C programmer.

What you need to do is sleep() for a couple of seconds after each
message, to give the reading program a chance to register that the
pipe has been closed. A better answer would have been to have used a
Unix-domain socket, but I doubt you have control over that...

Ben

 

[Jim Mozley]

Ben Morrow wrote:

AAARGH! Don't do that! Use the constants defined in Fcntl.pm.

use Fcntl qw/:flock/;

flock PIPE, LOCK_EX;

point taken

What? Pipes aren't seekable...

I'll take your word for that.

No. The second appears to have been written by a C programmer.

I don't know about that, but it is from a module on CPAN (Nagios::Cmd)
to interact with the application I'm trying to through messages at
(Nagios a network/system/service monitor)

What you need to do is sleep() for a couple of seconds after each
message, to give the reading program a chance to register that the
pipe has been closed.

I tried this by sleeping for 1 second and still saw the problem. This
puzzled me as I thought using flock's LOCK_EX would be honoured by any
other invocations of my script also using LOCK_EX. I guess I could
increase the time in the sleep, I'm not sure about the mechanism for
reading from the other end (its written in C) and how often it does this
(or even if it is time based).

I thought that the module on CPAN (published after I'd created my
script) could be the answer and wanted to check which approach was valid.

A better answer would have been to have used a
Unix-domain socket, but I doubt you have control over that...

I think the author of the app may be moving to this but cannot be sure.

Thanks for the help,

Jim

 

[Al Tobey]

Ben Morrow wrote:



point taken


I'll take your word for that.

The above code will also be sure to write to the end of a regular file.
Yes, you can open in APPEND mode, but that doesn't guarantee anything if
you keep the file open for multiple writes. The seek may not have any
effect on a pipe, but does not break on a pipe, either. So, the above
also works on regular files.

Actually, I'm not a C programmer - I was trying to avoid buffering. It's
probably not worth it once autoflush is on, so I've removed it.

I don't know about that, but it is from a module on CPAN (Nagios::Cmd)
to interact with the application I'm trying to through messages at
(Nagios a network/system/service monitor)


I tried this by sleeping for 1 second and still saw the problem. This
puzzled me as I thought using flock's LOCK_EX would be honoured by any
other invocations of my script also using LOCK_EX. I guess I could
increase the time in the sleep, I'm not sure about the mechanism for
reading from the other end (its written in C) and how often it does this
(or even if it is time based).

Nagios only reads the pipe at whatever interval you set in nagios.cfg. It
could be 1 second, but I think the default is 15 seconds.

Of course, flock/LOCK_EX only works for programs that use it for every
access to the file. If you're using Nagios::Cmd, for instance, it should
keep a storm of writes for clobbering each other.

I thought that

the module on CPAN (published after I'd created my

script) could be the answer and wanted to check which approach was
valid.


I think the author of the app may be moving to this but cannot be sure.

Nagios uses a fifo so that you can use standard unix tools to write to the
fifo (Yes, netcat can probably do it, too). It's about simplicity - I can
simply echo "MY_COMMAND ARGS" >>fifo.file instead of having to deal with
a bunch of network code.

 

[Brian McCauley]

IIRC the seek-before-each-print work-round for OSs that don't properly
implement O_APPEND is now included in recent Perl so you no longer
need that seek() except for backward compatability. Of course without
the flock there'd still be a race condition - another process can
extend the file between the seek() and the print() so that's still
needed on such an OS.

Yes, you can open in APPEND mode, but that doesn't guarantee anything if
you keep the file open for multiple writes.

Well asumming your OS actually supports O_APPEND mode it guarantees
that all writes go to the end of the file. Without locking it is
still possible that two simultaneous Perl output operations that
exceed the size that perl will do in a single write() syscall will end
up getting interleaved.

In the case of a PIPE (on a Unix-like OS) there is a size (known as
PIPE_BUF) below which single write()s to a pipe are guaranteed atomic.

You can find the value of PIPE_BUF on you OS thus:

require "limits.ph";
print PIPE_BUF();

I do not know what the lowest allowable value is.

--
\\ ( )
. _\\__[oo
.__/ \\ /\@
. l___\\
# ll l\\
###LL LL\\

 

[Ben Morrow]

You can find the value of PIPE_BUF on you OS thus:

require "limits.ph";
print PIPE_BUF();

I do not know what the lowest allowable value is.

SUSv3 says 512.

Ben

 

[Jim Mozley]

Al Tobey wrote:

Nagios only reads the pipe at whatever interval you set in nagios.cfg. It
could be 1 second, but I think the default is 15 seconds.

:$ I think this is the resolution to my problem. Without wanting to get
too OT I had the config set to 1 and a sleep 2 in my code. However the
application configuration file should have 1s to force interpreting the
1 as seconds rather than a minute. However, it does re-read the pipe
under certain circumstances before the interval and I think this is why
it was working most of the time. I'll stop here as I think this is
getting beyond a c.l.p.m discussion and my red face is melting the keyboard.

Jim