Timeout lengthy external executable

[Ema Fuma]

Hi all,
I'm new to Ruby, I'm trying to use it to create a script where I launch
some executables.
I'd like to timeout if one of them takes too long, I'm using IO.popen to
launch the exe.
The problem is that the timeout doesn't seem to work, so if my
executable hangs for some reason I have no way to stop it and go on.

Is there something I can do?
If I use "system" instead of IO.popen it works but I need popen to parse
stdout

I include a simplified code of what I'm doing (no parsing)

Thanks in advance for any tips on that
Bye

begin
timeout(5) {
IO.popen(command_media)
}
puts "Timeout didn't occur"
rescue Timeout::Error
puts "Timed out!"
end

 

[ara.t.howard]

Hi all,
I'm new to Ruby, I'm trying to use it to create a script where I
launch
some executables.
I'd like to timeout if one of them takes too long, I'm using
IO.popen to
launch the exe.
The problem is that the timeout doesn't seem to work, so if my
executable hangs for some reason I have no way to stop it and go on.

Is there something I can do?
If I use "system" instead of IO.popen it works but I need popen to
parse
stdout

I include a simplified code of what I'm doing (no parsing)

Thanks in advance for any tips on that
Bye

begin
timeout(5) {
IO.popen(command_media)
}
puts "Timeout didn't occur"
rescue Timeout::Error
puts "Timed out!"
end

if you're on windows try using 'systemu', which will allow you to
capture stdout/stderr and should also should not block your app.

gem install systemu

http://codeforpeople.com/lib/ruby/systemu/systemu-1.2.0/README

a @ http://codeforpeople.com/

 

[Ema Fuma]

gem install systemu

http://codeforpeople.com/lib/ruby/systemu/systemu-1.2.0/README

a @ http://codeforpeople.com/

Hi thanks for the quick answer,

I'm new to ruby and RubyGems, it seemed that the installation was
succesful
but I get an error when requiring the file. I'll try to figure it how,
what I would like to ask is that if with systemu I will be able to parse
the stdout and stderr line by line at real time (not waiting the process
to finish to parse stdout). From the doc it's not too clear to me

Thanks again


error when launching:
encode7.rb:92:in `require': no such file to load -- systemu (LoadError)
from encode7.rb:92

 

[ara.t.howard]

I'm new to ruby and RubyGems, it seemed that the installation was
succesful
but I get an error when requiring the file. I'll try to figure it how,

require 'rubygems'
require 'systemu'

what I would like to ask is that if with systemu I will be able to
parse
the stdout and stderr line by line at real time (not waiting the
process
to finish to parse stdout). From the doc it's not too clear to me

you cannot do this on windows safely. if you are on *nix you can
easily parse in realtime using the open4 lib, which is much more
powerful than systemu. however, only systemu is cross platform.

cheers.


a @ http://codeforpeople.com/

 

[Ema Fuma]

require 'rubygems'
require 'systemu'

Thanks, with that it worked perfectly!

you cannot do this on windows safely. if you are on *nix you can
easily parse in realtime using the open4 lib, which is much more
powerful than systemu. however, only systemu is cross platform.

Yes, I tried systemu and work exactly as you explained.
Yes, for me it would be better to have my script cross-platform
(providing for each OS the compiled executables to launch).
With IO.popen I was able to parse at run-time line by line the stdout
like this:

IO.popen(command) do |pipe|
pipe.each("\n") do |line|
puts line
end

And it's working (Windows and OSX), the only problem I had was that
timeout(10) wasn't ignored.
Now what I want to achieve by the run-time parsing are two things:
- display a progess of what is happening
- check that the process keeps writing on stdout, so I know it's still
alive. If it doesn't for 10 sec. exit the entire program.

Is that possible?

Thanks a lot again for your kindness!

 

[ara.t.howard]

Yes, I tried systemu and work exactly as you explained.
Yes, for me it would be better to have my script cross-platform
(providing for each OS the compiled executables to launch).
With IO.popen I was able to parse at run-time line by line the stdout
like this:

IO.popen(command) do |pipe|
pipe.each("\n") do |line|
puts line
end

And it's working (Windows and OSX), the only problem I had was that
timeout(10) wasn't ignored.
Now what I want to achieve by the run-time parsing are two things:
- display a progess of what is happening
- check that the process keeps writing on stdout, so I know it's still
alive. If it doesn't for 10 sec. exit the entire program.

Is that possible?

i'm 99% positive that trying to read from the pipe will conflict with
the timeout method - because of the way ruby uses green threads it's
easy to lock up threads where io is concerned. sometimes select can
be used but it's a case by case basis for mixing threads, io, and non-
blocking behaviour on windows. YMMV.

cheers.

a @ http://codeforpeople.com/

 

[Ema Fuma]

i'm 99% positive that trying to read from the pipe will conflict with

the timeout method - because of the way ruby uses green threads it's
easy to lock up threads where io is concerned. sometimes select can
be used but it's a case by case basis for mixing threads, io, and non-
blocking behaviour on windows. YMMV.

Ok, I thought I was doing something wrong, or not using the right
method.
Maybe I should use some customized timeout?
To me it looked strange that in a scripting language like Ruby it was
impossible to timeout certain operations and I thought someone else
found a solution.
Thanks

bye

 

[ara.t.howard]

Ok, I thought I was doing something wrong, or not using the right
method.
Maybe I should use some customized timeout?
To me it looked strange that in a scripting language like Ruby it was
impossible to timeout certain operations and I thought someone else
found a solution.
Thanks

bye

it's not ruby per-se, but the interaction of select on windows. you
haven't yet said - what platform are you on?

a @ http://codeforpeople.com/

 

[Ema Fuma]

it's not ruby per-se, but the interaction of select on windows. you

haven't yet said - what platform are you on?

I'm under Windows but I would like to use the same script on Linux.
Basically it's just
-launch an external executable that write it's progress on stdout
-parse the stdout line by line
-if there's no new line for more than 10 sec close the external exe

Thanks

 

[ara.t.howard]

I'm under Windows but I would like to use the same script on Linux.
Basically it's just
-launch an external executable that write it's progress on stdout
-parse the stdout line by line
-if there's no new line for more than 10 sec close the external exe

it's nearly impossible to do this on windows reliably. this are just
too many failure conditions, for instance say the child program does

gb = 2 ** 30

huge = '*' * gb

STDOUT.write huge

and the parent does

buf = child.gets

both processes will hang. the child because it fills the pipe, the
parent because it never sees a newline. you'd think this could be
solved by using timeout, but it cannot (on windows) because ruby's
threads schedule with select and therefore many/most io operations
will block the scheduler - causing a hang.

there is some hope for using socketpair and a tcp server in a cross
platoform manner, rough idea here:

http://pastie.org/173246

but this doesn't address issues like stderr or zombie processes.


anyhow, i'm not saying it *can't* be done on windows, but it's very
non-trival to combine pipes, timeout, and process management in a
robust fashion.


one idea might be to launch and external process to signal yourself
before reading, something like this:

signaler = Thread.new do
system "ruby -e' sleep 10; Process.killHUP.to_s,
#{ Process.pid })"
end

begin
child.gets
rescue SignalException
...
end


however signals are very limited on windows and i've no idea if even
the signal handlers would fire once a process is blocked - worth
trying though... it's definitely going to take something creative on
your end. one other thought is to spawn and external reaper before
reading, one which would kill the external exe for you

reaper = Thread.new
system "ruby -e' sleep 10; Process.kill -9, #{ child.pid }'"
end

and then, in the parent arrange to schedule this before each read, and
kill it afterwards...

i'd be interested to hear other's thoughts on the issue (dan berger in
the house?) as i'm no windows expert.

regards.

a @ http://codeforpeople.com/

 

[Ema Fuma]

Thanks for the very detailed explanation.
This is my first project using Ruby, I didn't expect that doing this
quite simple taks was so difficult, if I have to resort to signals and
threads maybe it would be better for me to do that in C++ where I'm more
experienced.
Thanks again