basic threading question: can ruby use real threads?

[Joel VanderWerf]

Hi,

I'm not sure if this is what MenTaLGuY meant, but one way that
Thread#raise is unsafe, is that it can raise an exception in the
specified thread while that thread is executing an 'ensure' block.

This can cause a failure of critical resources to be cleaned up
correctly, such as locks on mutexes, etc., as some or all of the
code in the ensure block is skipped.

I first ran into this when I tried to use timeout{} to implement
a ConditionVariable#timed_wait, like:

require 'thread'
require 'timeout'
class ConditionVariable
def timed_wait(mutex, timeout_secs)
timeout(timeout_secs) { wait(mutex) } # THIS IS UNSAFE
end
end

Note that 'timeout' functions by creating a temporary new thread
which sleeps for the duration, then raises an exception in the
'current' thread that invoked timeout.

If the timeout raises its exception at an unlucky moment, the
various internals of ConditionVariable#wait and Mutex#synchronize
that depend on ensure blocks to restore their class invariants are
skipped, resulting in nasty things like a permanently locked mutex.

Not fun...

This is disturbing.

Is #timeout inherently unsafe, if it is implemented as a thread, even in
MRI ruby's green threads?

Ruby gives you a lot of freedom to do anything you want inside of ensure
clauses, and I guess this means that ensure clauses can't be given
special treatment--the ensure clause itself might be what needs to be
interrupted by the timeout. That seems to rule out treating ensure
clauses as a critical section, for example. And it seems to rule out a
method like Thread#raise_unless_in_ensure or
Thread#raise_after_ensure_finishes.

What if there were two kind of ensure clauses, one which is
uninterruptible (to be used only for cleanup that is deterministic) and
one which is interruptible (and not guaranteed to finish)?

What's the best practice in current MRI ruby? Use Timeout only in cases
where you know it is safe, and otherwise use #select timeouts or
whatever else is appropriate?

 

[Bill Kelly]

Ruby gives you a lot of freedom to do anything you want inside of ensure
clauses, and I guess this means that ensure clauses can't be given
special treatment--the ensure clause itself might be what needs to be
interrupted by the timeout. That seems to rule out treating ensure
clauses as a critical section, for example.

Yeah... I agree in general. However, . . .

What if there were two kind of ensure clauses, one which is
uninterruptible (to be used only for cleanup that is deterministic) and
one which is interruptible (and not guaranteed to finish)?

I'd probably settle for a Thread#raise_asap (or #raise_safe, or
whatever.) Because in theory, even though one *could* write
unfriendly code in an ensure block that took a long time to execute; in
practice all the ensure blocks I can recall seeing were doing what
appeared to be very simple, deterministic cleanup.

With that, it would be trivial to implement a Timeout#timeout_safe; and
I suspect it would be rare indeed when someone would need or want
to use the unsafe version...

What's the best practice in current MRI ruby? Use Timeout only in cases
where you know it is safe, and otherwise use #select timeouts or
whatever else is appropriate?

That's pretty much my current practice. I use Timeout sparingly, and am
extremely careful with it.


Regards,

Bill

 

[MenTaLguY]

This is disturbing.

Is #timeout inherently unsafe, if it is implemented as a thread, even in
MRI ruby's green threads?

Correct. #timeout as presently implemented is NEVER safe to use.

What if there were two kind of ensure clauses, one which is
uninterruptible (to be used only for cleanup that is deterministic) and
one which is interruptible (and not guaranteed to finish)?

Nope, that's still not sufficient. The fundamental problem is that other threads can _arbitrarily_ mess with control flow by injecting exceptions via Thread#raise. There's no way to guarantee invariants will be preserved, no matter how clever your ensure implementation is.

-mental

 

[MenTaLguY]

I'd probably settle for a Thread#raise_asap (or #raise_safe, or
whatever.) Because in theory, even though one *could* write
unfriendly code in an ensure block that took a long time to execute; in
practice all the ensure blocks I can recall seeing were doing what
appeared to be very simple, deterministic cleanup.

No, that's still not sufficient. If an exception can be injected at an arbitrary point by an external source, there's simply no way to write sane code.

The only safe model for inter-thread communication is one where both participants agree to communicate. You'd need something like a Thread#receive_exception that was explicitly called on the receiving side -- of course, that's not too helpful for the purposes to which Thread#raise is usually put.

The fundamental problem is really that most of Ruby's blocking operations don't have allowance for timeout built into their API, which is particularly sad since they're all built on top of select (in MRI, anyway) and that would be really easy to do. The whole #timeout thing is an unsafe workaround which can never be made safe by its very nature.

-mental

 

[Charles Oliver Nutter]

Have you got evidence for this? I do not believe it to be the case for a
non-green-threaded JVM.

The OP is incorrect. Java VMs always use all cores in the system, except
in a very few specialized VMs that are green threaded.

Even if we're talking about only one thread of execution, there's still
the GC thread which generally runs in parallel.

- Charlie

 

[Sylvain Joyeux]

- Kill and raise require the target thread to eventually reach a

checkpoint where they are willing to "listen" to the kill or raise
event. If they don't, the calling thread will wait forever.

I think this is fair. I also think that the core developer may need to
really think about what should be a checkpoint *in the language itself*
(for instance, end of a block, end of a method, whatever). For instance,
not allowing to have a checkpoint in an "ensure" context would fix the
ensure-related issues (but may not fix others that I don't see)

My main use of Thread#raise is "returning" from a ConditionVariable#wait.
Can I assume this is seen as a checkpoint by JRuby ?

Sylvain Joyeux