Inter-thread synchronization

[Caleb Clausen]

At his presentation for RubyConf, Koichi Sasada said that
Thread.critical= may be going away in the future. I stood up
afterwards and requested that it stay. Charles Nutter stood up and
requested that it go. After talking to Charles and thinking it over
for a while, I've decided to withdraw my request to keep
Thread.critical=. It's just too hard to implement Thread.critical=
efficiently and portably on multi-processor systems. (Hardware should
be able to do it, but OSes and JVMs don't support it.)

However, I would like to take this opportunity to open a discussion
about how synchronization should be done in Ruby. Most of what I'm
about to say has been said before on this list, but it's good to
re-iterate it, and try to get some ideas flowing:

All the synchronization mechanisms in thread.rb ought to be re-written
in C. The current ruby versions are too slow. It may be that much of
the current use of Thread.critical= is because Mutex and friends are
so painful currently.

I'm actually of two minds about whether a future ruby that supports
native threads should still have green threads too using the existing
Thread/Mutex/Queue/etc. Usually when making the transition from one to
the other, you want to keep the old interface around for
compatibility. Green threads are generally cooperative, whereas native
threads are preemptive; code written for the cooperative model seldom
works preemptively. However, in the case of ruby, it looks like the
green threads are actually preemptive from the programmers
perspective, so maybe code written for ruby's green threads will work
in a native-threaded implementation.

It would be nice to see some more high-level synchronization
mechanisms, such as read-write locks, priority queues, counting
semaphores, and maybe some kind of select-like facility to wait for
multiple events and/or locks at once.

Also, does anyone know if ruby's threads currently can inherit
priorities? (I'm guessing no...) I've just noticed that they have
priorities, but I don't see anything about priority inversion or
priority inheritance anywhere in the documentation.

Finally, a fairly trivial item. It's somewhat confusing that you have
to require 'thread' and not 'mutex' to get the Mutex class, whereas
Thread is built-in and one doesn't need to require 'thread' to get it
at all (if you can manage without high-level synchronization).

 

[James Edward Gray II]

Finally, a fairly trivial item. It's somewhat confusing that you have
to require 'thread' and not 'mutex' to get the Mutex class, whereas
Thread is built-in and one doesn't need to require 'thread' to get it
at all (if you can manage without high-level synchronization).

This shouldn't be a problem for YARV (merging this month!), which has
the built-in Mutex.

James Edward Gray II

 

[Brad Tilley]

I'm actually of two minds about whether a future ruby that supports
native threads should still have green threads too using the existing
Thread/Mutex/Queue/etc. Usually when making the transition from one to
the other, you want to keep the old interface around for
compatibility. Green threads are generally cooperative...

I thought this too. I've been using green threads in Ruby lately. They are not
that bad. Certainly, they do some things well... especially smaller things. If
it's not too difficult, having green threads available in YARV/Ruby 2.0 would
be nice. Perhaps users could choose between using the new native threads or the
old green threads. If this is too difficult, or not very feasible, that's OK. If
I could only have one, I would choose native threads.

Finally, a fairly trivial item. It's somewhat confusing that you have
to require 'thread' and not 'mutex' to get the Mutex class, whereas
Thread is built-in and one doesn't need to require 'thread' to get it
at all (if you can manage without high-level synchronization).

Isn't the same true about Queue? I found that confusing the first time I used
threads... until I discovered that I had to require 'thread' first:

irb(main):013:0> x = Queue.new
NameError: uninitialized constant Queue
from (irb):13
from :0
irb(main):014:0> require 'thread'
=> true
irb(main):015:0> x = Queue.new
=> #<Queue:0x606d4 @waiting=[], @que=[]>
irb(main):016:0> puts x.class
Queue
=> nil
irb(main):017:0>

 

[Hugh Sasse]

On Wed, 8 Nov 2006, Caleb Clausen wrote:

["and these three men made a solemn vow / Thread.critical must die" [1]]

However, I would like to take this opportunity to open a discussion
about how synchronization should be done in Ruby. Most of what I'm
about to say has been said before on this list, but it's good to
re-iterate it, and try to get some ideas flowing:

All the synchronization mechanisms in thread.rb ought to be re-written
in C. The current ruby versions are too slow. It may be that much of

I see the value of this, but also consider Much Earlier discussion about
Ruby in Ruby. If we keep the C parts small and tight, it may aid
a faster ruby on many platforms in some ways. But I take your point.

the current use of Thread.critical= is because Mutex and friends are
so painful currently.

I'm actually of two minds about whether a future ruby that supports
native threads should still have green threads too using the existing
Thread/Mutex/Queue/etc. Usually when making the transition from one to

Bear Queue in mind....

the other, you want to keep the old interface around for
Yes
compatibility. Green threads are generally cooperative, whereas native [...]

It would be nice to see some more high-level synchronization
mechanisms, such as read-write locks, priority queues, counting

Monitors, and there was something interesting about Ada Rendezvous
but it's 20 years since I read up on that, so I can't really remember...

semaphores, and maybe some kind of select-like facility to wait for
multiple events and/or locks at once.

Also, does anyone know if ruby's threads currently can inherit
priorities? (I'm guessing no...) I've just noticed that they have
priorities, but I don't see anything about priority inversion or
priority inheritance anywhere in the documentation.

...and back to Queues. I'd really like to see Heaps in the standard
library. I can offer a Ruby implementation, but don't have a C one
or much time to sort one out. But this would help in a number of
different problems, particularly Priority Queues. There may be a
good reason to leave them out, why they aren't there now, however.
Is there?

Finally, a fairly trivial item. It's somewhat confusing that you have
to require 'thread' and not 'mutex' to get the Mutex class, whereas
Thread is built-in and one doesn't need to require 'thread' to get it
at all (if you can manage without high-level synchronization).

Maybe there should be a general %{ require 'synchronize' } to cover
the various means to achieve that -- hiding implementation details
and all that?

Hugh
Hugh
[1] "John Barleycorn must die" (traditional)

 

[David Vallner]

--------------enig6432450589A19C56D63B8E80
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I see the value of this, but also consider Much Earlier discussion abou= t
Ruby in Ruby. If we keep the C parts small and tight, it may aid
a faster ruby on many platforms in some ways. But I take your point.
=20

Hmm. The Smalltalk / Java approach? I can see where Ruby would make that
a little difficult, but if together with a VM it meant pure-ruby
libraries are Sanely Fast, I'd be giddy with glee. (The way of C
extensions and shelling out are a deployment nightmare, as well as a
general nuisance.)

Monitors, and there was something interesting about Ada Rendezvous
but it's 20 years since I read up on that, so I can't really remember..= =2E
=20

IIRC, The Ada Rendezvous was sort of the swiss army knife of thread
synchronisation and mostly ended up as a means of implementing the more
common methods.

Though rereading whatever introductory material Google spat at me, it
seems more like a message-passing system. Which makes me wonder, how
complicated a Ruby DSL (well, I suppose it wouldn't be quite
domain-specific) that replicates most of the functionality of Erlang
would be?

=20
Maybe there should be a general %{ require 'synchronize' } to cover
the various means to achieve that -- hiding implementation details
and all that?

This might get confused with synchronising processes, which always
requires OS integration in case this would ever get added to the
standard library. (Disclaimer: I only -think- Unixen let you do
synchronisation between processes too - online material and my college
instructors were very averse of using the word "thread" - too Windowsy
and non-smug I presume, so I've no idea which concept they meant when.
Ignore this part if I'm not making sense because of that.)

Naming it "thread/synchronise" would be fine, but by now it would be way
too much code-breaking unless both variants are applicable.

David Vallner


--------------enig6432450589A19C56D63B8E80
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)

iD8DBQFFUbbMy6MhrS8astoRAm+SAJ9OXnW0BKpDppvuHBDL+Aj01l3VHwCZAaS8
jfcscz2BRORRW/BlQoSXiCA=
=Cmhs
-----END PGP SIGNATURE-----

--------------enig6432450589A19C56D63B8E80--