open/print or sys commands to write to a named pipe?

Discussion in 'Perl Misc' started by Jim Mozley, Nov 12, 2003.

  1. Jim Mozley

    Jim Mozley Guest

    I need to send messages to an application whose interface for this
    purpose is a named pipe (fifo).

    I have a script to write to the pipe that is executed (via swatch)
    whenever a certain syslog message is pattern matched. They problem is
    that when I receive a group of messages nearly simultaneously I
    sometimes see the resulting output in the application as one message.

    E.g. the messages

    "help me"
    "help me too"
    "and me"

    should appear as three separate messages but on rare occations I get

    "help me help me too and me"

    shown in the application as one message.

    I current do the following to write to the pipe:

    open (PIPE, "> $pipe") or die "Cannot open pipe $!";
    flock(PIPE,2);
    print PIPE "$cmd";
    close PIPE or die "Cannot close pipe $!";

    I was wondering if I should use the following code below (taken from an
    example I saw for writing to a pipe that I propose to put in a package):

    my $fh = new IO::File;
    sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
    or croak "could not open $cmdfile for writing: $!";
    flock( $fh, LOCK_EX );
    seek( $fh, 0, SEEK_END );
    my $numofbytes = syswrite( $fh, $cmd, length($cmd) );
    flock( $fh, LOCK_UN );
    carp "No data written!" unless $numofbytes;

    Is one better than the other for this purpose?

    Thanks,

    Jim
    Jim Mozley, Nov 12, 2003
    #1
    1. Advertising

  2. Jim Mozley

    Ben Morrow Guest

    Jim Mozley <> wrote:
    > I need to send messages to an application whose interface for this
    > purpose is a named pipe (fifo).
    >
    > I have a script to write to the pipe that is executed (via swatch)
    > whenever a certain syslog message is pattern matched. They problem is
    > that when I receive a group of messages nearly simultaneously I
    > sometimes see the resulting output in the application as one message.
    >
    > E.g. the messages
    >
    > "help me"
    > "help me too"
    > "and me"
    >
    > should appear as three separate messages but on rare occations I get
    >
    > "help me help me too and me"
    >
    > shown in the application as one message.
    >
    > I current do the following to write to the pipe:
    >
    > open (PIPE, "> $pipe") or die "Cannot open pipe $!";
    > flock(PIPE,2);


    AAARGH! Don't do that! Use the constants defined in Fcntl.pm.

    use Fcntl qw/:flock/;

    flock PIPE, LOCK_EX;

    > print PIPE "$cmd";
    > close PIPE or die "Cannot close pipe $!";
    >
    > I was wondering if I should use the following code below (taken from an
    > example I saw for writing to a pipe that I propose to put in a package):
    >
    > my $fh = new IO::File;
    > sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
    > or croak "could not open $cmdfile for writing: $!";
    > flock( $fh, LOCK_EX );
    > seek( $fh, 0, SEEK_END );


    What? Pipes aren't seekable...

    > my $numofbytes = syswrite( $fh, $cmd, length($cmd) );
    > flock( $fh, LOCK_UN );
    > carp "No data written!" unless $numofbytes;
    >
    > Is one better than the other for this purpose?


    No. The second appears to have been written by a C programmer.

    What you need to do is sleep() for a couple of seconds after each
    message, to give the reading program a chance to register that the
    pipe has been closed. A better answer would have been to have used a
    Unix-domain socket, but I doubt you have control over that... :)

    Ben

    --
    Musica Dei donum optimi, trahit homines, trahit deos. |
    Musica truces molit animos, tristesque mentes erigit. |
    Musica vel ipsas arbores et horridas movet feras. |
    Ben Morrow, Nov 12, 2003
    #2
    1. Advertising

  3. Jim Mozley

    Jim Mozley Guest

    Ben Morrow wrote:


    >>I current do the following to write to the pipe:
    >>
    >>open (PIPE, "> $pipe") or die "Cannot open pipe $!";
    >>flock(PIPE,2);

    >
    >
    > AAARGH! Don't do that! Use the constants defined in Fcntl.pm.
    >
    > use Fcntl qw/:flock/;
    >
    > flock PIPE, LOCK_EX;


    point taken

    >>print PIPE "$cmd";
    >>close PIPE or die "Cannot close pipe $!";
    >>
    >>I was wondering if I should use the following code below (taken from an
    >>example I saw for writing to a pipe that I propose to put in a package):
    >>
    >>my $fh = new IO::File;
    >>sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
    >> or croak "could not open $cmdfile for writing: $!";
    >>flock( $fh, LOCK_EX );
    >>seek( $fh, 0, SEEK_END );

    >
    >
    > What? Pipes aren't seekable...


    I'll take your word for that.

    >>my $numofbytes = syswrite( $fh, $cmd, length($cmd) );
    >>flock( $fh, LOCK_UN );
    >>carp "No data written!" unless $numofbytes;
    >>
    >>Is one better than the other for this purpose?

    >
    >
    > No. The second appears to have been written by a C programmer.


    I don't know about that, but it is from a module on CPAN (Nagios::Cmd)
    to interact with the application I'm trying to through messages at
    (Nagios a network/system/service monitor)

    > What you need to do is sleep() for a couple of seconds after each
    > message, to give the reading program a chance to register that the
    > pipe has been closed.


    I tried this by sleeping for 1 second and still saw the problem. This
    puzzled me as I thought using flock's LOCK_EX would be honoured by any
    other invocations of my script also using LOCK_EX. I guess I could
    increase the time in the sleep, I'm not sure about the mechanism for
    reading from the other end (its written in C) and how often it does this
    (or even if it is time based).

    I thought that the module on CPAN (published after I'd created my
    script) could be the answer and wanted to check which approach was valid.

    > A better answer would have been to have used a
    > Unix-domain socket, but I doubt you have control over that... :)


    I think the author of the app may be moving to this but cannot be sure.

    Thanks for the help,

    Jim
    Jim Mozley, Nov 12, 2003
    #3
  4. Jim Mozley

    Al Tobey Guest

    On Wed, 12 Nov 2003 17:22:34 +0000, Jim Mozley wrote:

    > Ben Morrow wrote:
    >
    >
    >>>I current do the following to write to the pipe:
    >>>
    >>>open (PIPE, "> $pipe") or die "Cannot open pipe $!";
    >>>flock(PIPE,2);

    >>
    >>
    >> AAARGH! Don't do that! Use the constants defined in Fcntl.pm.
    >>
    >> use Fcntl qw/:flock/;
    >>
    >> flock PIPE, LOCK_EX;

    >
    > point taken
    >
    >>>print PIPE "$cmd";
    >>>close PIPE or die "Cannot close pipe $!";
    >>>
    >>>I was wondering if I should use the following code below (taken from an
    >>>example I saw for writing to a pipe that I propose to put in a package):
    >>>
    >>>my $fh = new IO::File;
    >>>sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
    >>> or croak "could not open $cmdfile for writing: $!";
    >>>flock( $fh, LOCK_EX );
    >>>seek( $fh, 0, SEEK_END );

    >>
    >>
    >> What? Pipes aren't seekable...

    >
    > I'll take your word for that.

    The above code will also be sure to write to the end of a regular file.
    Yes, you can open in APPEND mode, but that doesn't guarantee anything if
    you keep the file open for multiple writes. The seek may not have any
    effect on a pipe, but does not break on a pipe, either. So, the above
    also works on regular files.

    >>>my $numofbytes = syswrite( $fh, $cmd,

    length($cmd) ); flock( $fh,
    >>>LOCK_UN );
    >>>carp "No data written!" unless $numofbytes;
    >>>
    >>>Is one better than the other for this purpose?

    >>
    >>
    >> No. The second appears to have been written by a C programmer.


    Actually, I'm not a C programmer - I was trying to avoid buffering. It's
    probably not worth it once autoflush is on, so I've removed it.

    > I don't know about that, but it is from a module on CPAN (Nagios::Cmd)
    > to interact with the application I'm trying to through messages at
    > (Nagios a network/system/service monitor)
    >
    >> What you need to do is sleep() for a couple of seconds after each
    >> message, to give the reading program a chance to register that the pipe
    >> has been closed.

    >
    > I tried this by sleeping for 1 second and still saw the problem. This
    > puzzled me as I thought using flock's LOCK_EX would be honoured by any
    > other invocations of my script also using LOCK_EX. I guess I could
    > increase the time in the sleep, I'm not sure about the mechanism for
    > reading from the other end (its written in C) and how often it does this
    > (or even if it is time based).


    Nagios only reads the pipe at whatever interval you set in nagios.cfg. It
    could be 1 second, but I think the default is 15 seconds.

    Of course, flock/LOCK_EX only works for programs that use it for every
    access to the file. If you're using Nagios::Cmd, for instance, it should
    keep a storm of writes for clobbering each other.

    > I thought that

    the module on CPAN (published after I'd created my
    > script) could be the answer and wanted to check which approach was
    > valid.
    >
    >> A better answer would have been to have used a Unix-domain socket, but
    >> I doubt you have control over that... :)

    >
    > I think the author of the app may be moving to this but cannot be sure.


    Nagios uses a fifo so that you can use standard unix tools to write to the
    fifo (Yes, netcat can probably do it, too). It's about simplicity - I can
    simply echo "MY_COMMAND ARGS" >>fifo.file instead of having to deal with
    a bunch of network code.

    > Thanks for the help,
    >
    > Jim
    Al Tobey, Nov 29, 2003
    #4
  5. Al Tobey <> writes:
    > Ben Morrow wrote:
    >>>my $fh = new IO::File;
    >>>sysopen( $fh, $pipe, O_SYNC|O_WRONLY|O_APPEND )
    >>> or croak "could not open $cmdfile for writing: $!";
    >>>flock( $fh, LOCK_EX );
    >>>seek( $fh, 0, SEEK_END );


    IIRC the seek-before-each-print work-round for OSs that don't properly
    implement O_APPEND is now included in recent Perl so you no longer
    need that seek() except for backward compatability. Of course without
    the flock there'd still be a race condition - another process can
    extend the file between the seek() and the print() so that's still
    needed on such an OS.

    > Yes, you can open in APPEND mode, but that doesn't guarantee anything if
    > you keep the file open for multiple writes.


    Well asumming your OS actually supports O_APPEND mode it guarantees
    that all writes go to the end of the file. Without locking it is
    still possible that two simultaneous Perl output operations that
    exceed the size that perl will do in a single write() syscall will end
    up getting interleaved.

    In the case of a PIPE (on a Unix-like OS) there is a size (known as
    PIPE_BUF) below which single write()s to a pipe are guaranteed atomic.

    You can find the value of PIPE_BUF on you OS thus:

    require "limits.ph";
    print PIPE_BUF();

    I do not know what the lowest allowable value is.

    --
    \\ ( )
    . _\\__[oo
    .__/ \\ /\@
    . l___\\
    # ll l\\
    ###LL LL\\
    Brian McCauley, Dec 1, 2003
    #5
  6. Jim Mozley

    Ben Morrow Guest

    Brian McCauley <> wrote:
    > You can find the value of PIPE_BUF on you OS thus:
    >
    > require "limits.ph";
    > print PIPE_BUF();
    >
    > I do not know what the lowest allowable value is.


    SUSv3 says 512.

    Ben

    --
    don't get my sympathy hanging out the 15th floor. you've changed the locks 3
    times, he still comes reeling though the door, and soon he'll get to you, teach
    you how to get to purest hell. you do it to yourself and that's what really
    hurts is you do it to yourself just you, you and noone else *
    Ben Morrow, Dec 1, 2003
    #6
  7. Jim Mozley

    Jim Mozley Guest

    Al Tobey wrote:


    > Nagios only reads the pipe at whatever interval you set in nagios.cfg. It
    > could be 1 second, but I think the default is 15 seconds.


    :$ I think this is the resolution to my problem. Without wanting to get
    too OT I had the config set to 1 and a sleep 2 in my code. However the
    application configuration file should have 1s to force interpreting the
    1 as seconds rather than a minute. However, it does re-read the pipe
    under certain circumstances before the interval and I think this is why
    it was working most of the time. I'll stop here as I think this is
    getting beyond a c.l.p.m discussion and my red face is melting the keyboard.

    Jim
    Jim Mozley, Dec 1, 2003
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. lee, wonsun
    Replies:
    1
    Views:
    478
    Jack Klein
    Nov 2, 2004
  2. Armin Gajda

    How to write to a named pipe

    Armin Gajda, Jul 28, 2006, in forum: Java
    Replies:
    1
    Views:
    1,888
  3. GinTon

    sys.stderr.write and sys.exit

    GinTon, Nov 23, 2006, in forum: Python
    Replies:
    5
    Views:
    568
    Dennis Lee Bieber
    Nov 24, 2006
  4. Replies:
    0
    Views:
    446
  5. Rocky
    Replies:
    22
    Views:
    211
    MichiganBob
    Jul 17, 2004
Loading...

Share This Page