Signal handling strangeness (Linux)

[Han Holl]

Hi,

The following program:

#!/usr/bin/ruby -w
IO.popen("tail -F --lines=0 --pid=#{Process.pid} /var/log/messages") do |h|
while line = h.gets
puts line
end
end
# end-of-program

can be terminated with 'kill -TERM pid'.

This one:

#!/usr/bin/ruby -w
trap('TERM') do
exit
end

IO.popen("tail -F --lines=0 --pid=#{Process.pid} /var/log/messages") do |h|
while line = h.gets
puts line
end
end
# end-of-program

cannot. It hangs on a waitpid (or wait4) for the tail process.
Why is this ? And is it reasonable ?

Han Holl

PS Forgot to say: this is Linux.

 

[Bertram Scharpf]

Hi,

Am Donnerstag, 04. Okt 2007, 18:47:25 +0900 schrieb Han Holl:

#!/usr/bin/ruby -w
trap('TERM') do
exit
end

IO.popen("tail -F --lines=0 --pid=#{Process.pid} /var/log/messages") do |h|
while line = h.gets
puts line
end
end
# end-of-program

cannot [...] be terminated with 'kill -TERM pid'.
It hangs on a waitpid (or wait4) for the tail process.
Why is this ? And is it reasonable ?

The wait is part of popen. So, `tail' runs until
Process.pid(==$$) terminates and $$ waits for the `tail' to
finish. It's a deadlock but it's the correct behaviour. You
should state a clear definition which task is to be stopped.
You can offer a pid or a kill command by another feature of
your Ruby application.

I did a lot process and thread scheduling and at least to me
it comes hard every time anew.

Bertram

 

[Han Holl]

The wait is part of popen. So, `tail' runs until
Process.pid(==$$) terminates and $$ waits for the `tail' to
finish. It's a deadlock but it's the correct behaviour. You
should state a clear definition which task is to be stopped.
You can offer a pid or a kill command by another feature of
your Ruby application.

Yes, but the deadlock isn't there without the trap('TERM').

Cheers,

Han Holl

 

[Bertram Scharpf]

Hi,

Am Donnerstag, 04. Okt 2007, 19:14:51 +0900 schrieb Han Holl:

Yes, but the deadlock isn't there without the trap('TERM').

I didn't examine it thoroughly but "exit" seems to raise an
exception "SystemExit". This exception will probably not be
raised before the wait statement finishes. The default
signal handler for 'TERM' will not raise anything but rather
try to stop the whole Ruby process.

You can catch the SystemExit exception by putting a
begin-rescue-end frame around your popen call.

Bertram

 

[Han Holl]

I didn't examine it thoroughly but "exit" seems to raise an
exception "SystemExit". This exception will probably not be
raised before the wait statement finishes. The default
signal handler for 'TERM' will not raise anything but rather
try to stop the whole Ruby process.

Now I notice that a ruby script that is terminated with an external
'kill -TERM' does _not_ run the at_exit() proc.
Because TERM is intended to enable processes to terminate gracefully
(cleaning up after themselves), this seems a bit harsh, but it seems
to be standard Unix behaviour.

Still, this deadlock is annoying.

Thanks,

Han Holl

 

[Robert Klemme]

Now I notice that a ruby script that is terminated with an external
'kill -TERM' does _not_ run the at_exit() proc.
Because TERM is intended to enable processes to terminate gracefully
(cleaning up after themselves), this seems a bit harsh, but it seems
to be standard Unix behaviour.

Still, this deadlock is annoying.

Why do you resort to "tail" at all? You could implement this pretty
easy in Ruby itself. Then you do not face this issue plus you are
more independent of your local system installation. IIRC I provided
an implementation in some posint a while ago and I believe there is
also a tail version in RAA.

Kind regards

robert

 

[Paul Brannan]

This one:

#!/usr/bin/ruby -w
trap('TERM') do
exit
end

IO.popen("tail -F --lines=0 --pid=#{Process.pid} /var/log/messages") do |h|
while line = h.gets
puts line
end
end
# end-of-program

cannot. It hangs on a waitpid (or wait4) for the tail process.
Why is this ? And is it reasonable ?

Same problem with this program:

$ ruby -v
ruby 1.8.2 (2004-12-22) [i686-linux]
$ ruby -e 'r = IO.popen("tail -f /dev/null"); r.close'
<hangs>

But this one does not:
$ ruby -e 'IO.popen("tail -f /dev/null")'

I see no way to close the fd without waiting on the child process, so if
you want this behavior, don't use popen (you can use Open3.popen3
instead).

Paul

 

[Paul Brannan]

Why do you resort to "tail" at all? You could implement this pretty
easy in Ruby itself. Then you do not face this issue plus you are
more independent of your local system installation. IIRC I provided
an implementation in some posint a while ago and I believe there is
also a tail version in RAA.

See also [ruby-talk:45639] and the following thread.

Paul

 

[Han Holl]

Why do you resort to "tail" at all? You could implement this pretty
easy in Ruby itself. Then you do not face this issue plus you are
more independent of your local system installation. IIRC I provided
an implementation in some posint a while ago and I believe there is
also a tail version in RAA.

Mainly because I want to be able to tail more than one file at the same time.
tail does a very nice job of serialization, giving easy to parse
headers to indicate which file the current output is from. It's also
capable of following the file name instead of the descriptor. And it
has a nice small memory footprint and low cpu usage.
I'm sure it's possible to redevelop tail in ruby but with all these
features it's not trivial.
And the --pid= feature is nice as well.
Why re-invent proven technology ?
And I'm not at all interested in running on anything that hasn't got GNU tail.

Cheers,

Han Holl

 

[Han Holl]

I see no way to close the fd without waiting on the child process, so if
you want this behavior, don't use popen (you can use Open3.popen3
instead).

The following works:
trap('TERM') do
exit!
end
so it would seem that 'waitpid() is closely related to the atexit() functions.
The following would be nice to have:
trap('TERM') do
kill child_process
end
This woulr let me do at_exit processing as well.
But I haven't found out how to get the pid of the child with IO.popen.

Cheers

Han Holl

 

[Paul Brannan]

so it would seem that 'waitpid() is closely related to the atexit() functions.

It doesn't have anything to do with atexit except that the atexit
handlers and finalizers are both run when an application terminates
normally.

The finalizer also gets run if the object gets finalized, so not closing
the handle before all references to the object go away can also cause
problems.

The following would be nice to have:
trap('TERM') do
kill child_process
end
This woulr let me do at_exit processing as well.
But I haven't found out how to get the pid of the child with IO.popen.

You can't.

(That's not 100% true -- I once implemented something like this in ruby
that passed a dummy flag to the child and used pgrep to find the child's
pid, but you probably don't want to go that route; it's easier to
fork/exec yourself or use popen4).

Paul