Interrupting the evaluation of a ruby script

[Emmanuel Emmanuel]

This is my problem :

Let's say I have a ruby script which first takes as an input a ruby
script (that could be anything, and that I don't control) and evals it
using the eval ruby command and proceeds to some computations.

Now the evaluation of the inputed script can take very long.... I would
like to be able to interrupt it (i.e., interrupt the eval command),
ideally from another ruby process, raising an exception that could be
trapped by my first process so the evaluation could be skipped and the
process could proceed to the other computations.

I could send a SIGINT (in C) to the first process ? And trap this signal
in ruby in the first process...
I guess that should work (can I send it in ruby ? what os the command ?)
?

However, If I understand correctly, that will not work on Windows
platform....

What is the solution ?

ANy idea ?

Thank you

Emmanuel

 

[Branden Tanga]

This is my problem :

Let's say I have a ruby script which first takes as an input a ruby
script (that could be anything, and that I don't control) and evals it
using the eval ruby command and proceeds to some computations.

Now the evaluation of the inputed script can take very long.... I would
like to be able to interrupt it (i.e., interrupt the eval command),
ideally from another ruby process, raising an exception that could be
trapped by my first process so the evaluation could be skipped and the
process could proceed to the other computations.

I could send a SIGINT (in C) to the first process ? And trap this signal
in ruby in the first process...
I guess that should work (can I send it in ruby ? what os the command ?)
?

However, If I understand correctly, that will not work on Windows
platform....

What is the solution ?

ANy idea ?

Thank you

Emmanuel

Ruby noob here. I think what technique you would have to use would be
determined by your requirements for stopping evaluation. If you need to
"freeze" the eval, check some value, then carry on (or not), you can
accomplish it within the same thread. If you need to have something like
say, an independent timer which could send an interrupt to the eval,
then you would need multi threading. If single threading is acceptable,
couldn't you do something like the following? (forgive me for the
horrible pseudocode)


my_string = "some stuff here to evaluate"
... figure out where to insert a yield statement into my_string
my_string.instance_eval do
some_condition == true ? (throw error) : (carry on)
end

 

[Emmanuel Emmanuel]

Thanks very much for your answer.

my_string = "some stuff here to evaluate"
... figure out where to insert a yield statement into my_string
my_string.instance_eval do
some_condition == true ? (throw error) : (carry on)
end

This would not work since the string I am evaluating could be anyything
(even 2000 line length), it is an input given by the user basically...
so no way I can parse it to understand where to insert the yield
statement !

Ruby noob here. I think what technique you would have to use would be
determined by your requirements for stopping evaluation. If you need to
"freeze" the eval, check some value, then carry on (or not), you can
accomplish it within the same thread. If you need to have something like
say, an independent timer which could send an interrupt to the eval,
then you would need multi threading. If single threading is acceptable,
couldn't you do something like the following? (forgive me for the
horrible pseudocode)

I guess you are right I should use multithreading... I am going to try
that .....
If I terminate the (sub)thread suddenly (from the main thread), is there
a way to catch this termination in the subthread so that I could do some
cleaning first ?

It is VERY important for me that the evaluation is very fast... I am
afraid that put it in a separate thread, and having the main thread just
being in charged of eventually stopping this subthread (so waking up
every second) is going to slow down the evaluation quite a lot.

I understood, ruby's threading system is not really optiomized ? Am I
wrong ?

What would be best way to have the main thread waking up every second
(to see if teh user asks for interruption) taking as little of CPU as
possible ?

Emmanuel

 

[Branden Tanga]

Disclosure: I have no experience with multi threading in ruby, but based
on these parameters:
* User can input a string of any length to be evaluated
* User determines when to interrupt the evaluation

You would need 2 threads, one to evaluate the string, and one to sit
there active, waiting to process an interrupt. So there would be no need
to "wake up" the 2nd thread, because it's always there, and always
running. The performance hit for having multiple threads like this would
be determined by how the ruby runtime takes advantage of multicore
processors in different operating systems.

Another concern that you must address, is how to determine which
evaluation to apply an interrupt to. If you have 50 concurrent users,
you would have 100 threads total. You would need to be sure to apply the
correct interrupt to the correct thread. Perhaps a global hash which
contains the id of the interrupt thread, and its corresponding
evaluation thread. Then when an interrupt thread receives the proper
input, it can look up its evaluation and send the interrupt command.

In any case, performance tuning an multi threaded application is not
easy. You might even have to look at alternative runtimes, like running
it through jruby/tomcat to take advantage of java multi threading
(assuming that jruby has the appropriate hooks into the jvm to
accomplish such a thing).

 

[Caleb Clausen]

Thanks very much for your answer.

This would not work since the string I am evaluating could be anyything
(even 2000 line length), it is an input given by the user basically...
so no way I can parse it to understand where to insert the yield
statement !


I guess you are right I should use multithreading... I am going to try
that .....
If I terminate the (sub)thread suddenly (from the main thread), is there
a way to catch this termination in the subthread so that I could do some
cleaning first ?

It is VERY important for me that the evaluation is very fast... I am
afraid that put it in a separate thread, and having the main thread just
being in charged of eventually stopping this subthread (so waking up
every second) is going to slow down the evaluation quite a lot.

I understood, ruby's threading system is not really optiomized ? Am I
wrong ?

What would be best way to have the main thread waking up every second
(to see if teh user asks for interruption) taking as little of CPU as
possible ?

If this is what you want to do, you should probably use the Timeout
module in timeout.rb, which does this already (spawn another thread to
limit time taken by some block of code).

OTOH, you must consider whether you can trust the code that your users
are giving you. It would be very easy for user code to simply kill the
timeout thread, (whether or not you use the timeout library) and thus
bypass your timeout. If you put user code in a separate process, it
would be somewhat harder for them to bypass the timeout, but I think
still possible.

The issue of executing code that you can't trust is a hairy one, and
not well solved for ruby.

 

[Emmanuel Emmanuel]

Thank you for both of you !!
It really helps me.
Now just one remark to Caleb :

bypass your timeout. If you put user code in a separate process, it
would be somewhat harder for them to bypass the timeout, but I think
still possible.

Sure ! I 100% agree !
Now the big question (that I don't know the answer) is how do I
interrupt the evaluation of
a ruby script from another process ?? (and I want somthing that works on
all platforms ....)

If you know the answer to that, please let me know (and I don't want to
just kill the process, I want to interrupt the evaluation)

Emmanuel

 

[Dimitri De Frenne]

You might want to take a look at Eventmachine
(http://eventmachine.rubyforge.org). You can run another thread (up to
20) within your main one with defer() while the main one keeps on
working.

Dimitri De Frenne

 

[Caleb Clausen]

Sure ! I 100% agree !
Now the big question (that I don't know the answer) is how do I
interrupt the evaluation of
a ruby script from another process ?? (and I want somthing that works on
all platforms ....)

If you know the answer to that, please let me know (and I don't want to
just kill the process, I want to interrupt the evaluation)

What is the difference between 'kill' and 'interrupt'? Do you mean by
'interrupt' that you want to stop the process and then potentially
start it again where it left off?

If you're going to put the user code in another process, the kill
syscall is almost certainly what you'll need to use to 'interrupt' it,
whatever that means. kill is sort of misnamed... it's the syscall to
send a signal to another process. Typically, this stops execution and
closes the process down, but you can intercept it and you can do
whatever you like. Try the following commands if you have linux to see
some relevant documentation:

man 1 kill
man 2 kill
man 2 signal

If stopping and restarting the process is what you want, you might get
some good mileage out of the STOP and CONT signals.

 

[Emmanuel Emmanuel]

What is the difference between 'kill' and 'interrupt'? Do you mean by
'interrupt' that you want to stop the process and then potentially
start it again where it left off?

Nope.
Kill is killing the process
Interrupt would be like raising an exception in the ruby code ....
(with threads, if I do the eval in a thread, if I terminate this thread,
the toplevel environment of ruby is still there)

If you're going to put the user code in another process, the kill
syscall is almost certainly what you'll need to use to 'interrupt' it,
whatever that means. kill is sort of misnamed... it's the syscall to
send a signal to another process. Typically, this stops execution and
closes the process down, but you can intercept it and you can do
whatever you like. Try the following commands if you have linux to see
some relevant documentation:

Ok, if you want I want to intercept the "kill" and not terminate the
process

man 1 kill
man 2 kill
man 2 signal

I understood that signals don't work on windows .... The only one it
seems to understand is kill, and I can't intercept it .. it seems to be
a well known problem

If stopping and restarting the process is what you want, you might get
some good mileage out of the STOP and CONT signals.

pretty sure Windows does not know about those....

EMmanuel

 

[Caleb Clausen]

I understood that signals don't work on windows .... The only one it
seems to understand is kill, and I can't intercept it .. it seems to be
a well known problem

MS documentation claims they have a posix-like signal(), which
supports a limited range of signals:

http://msdn.microsoft.com/en-us/library/xdkz3x12(VS.71).aspx

I haven't tried it. I don't have windows. Does this not actually work?

 

[Emmanuel Emmanuel]

MS documentation claims they have a posix-like signal(), which
supports a limited range of signals:

http://msdn.microsoft.com/en-us/library/xdkz3x12(VS.71).aspx

I haven't tried it. I don't have windows. Does this not actually work?

What I know (having tried it) is that on ruby, only the signal Kill can
be sent, and the "trapping" does not work....

Now reading what's written on the page you sent me 1) SIGINT isn ot
supported 2) it is not clear that I can raise one of the other signals
from a different process (the raise function is used within the same
process!)

Reading what's said on ruby on Windows .... it seems to be a well known
problem that one cannot send a signal to interrupt a process ....

Emmanuel

 

[Charles Oliver Nutter]

In any case, performance tuning an multi threaded application is not
easy. You might even have to look at alternative runtimes, like running
it through jruby/tomcat to take advantage of java multi threading
(assuming that jruby has the appropriate hooks into the jvm to
accomplish such a thing).

JRuby threads are Java threads, which on all the common JVM
implementations are just OS-level threads. So yes, threaded code in
Ruby running in JRuby actually uses native threads and actually runs
concurrently across cores. For this reason, you can run an entire
Rails site with a single JRuby instance and saturate all your cores in
the process.

- Charlie