Threads + Forks in Ruby 1.9.1p129

[Lee Hinman]

Hey All,
I'm seeing some weirdness trying to get Ara's forkoff gem working on
ruby 1.9.1.

Here's some code that duplicates the problem:

#!/usr/bin/env ruby
require 'thread'
t1 = Thread.new do
pid = fork
unless pid
sleep(1)
exit
end
Process.waitpid(pid)
end
t2 = Thread.new do
pid = fork
unless pid
sleep(1)
exit
end
Process.waitpid(pid)
end
t1.join
t2.join

On Ruby 1.8.6, it runs for ~1.2 seconds, then exits. On Ruby 1.9.1, it
runs and hangs forever and I'm left with zombie processes every time I
run it.

Can someone help me understand why this code hangs indefinitely
please? Some kind of deadlock issue (similar to bug #1525) maybe?

Full ruby version: ruby 1.9.1p129 (2009-05-12 revision 23412) [i386-
darwin9.7.0] (on OSX 10.5)

Thanks,
Lee

 

[Roger Pack]

Hey All,
I'm seeing some weirdness trying to get Ara's forkoff gem working on
ruby 1.9.1.

Maybe it doesn't copy the threads over right or something?
http://ruby-doc.faithpromotingstories.org/doc/ruby-1.9.1-p129/classes/Kernel.html#M002828
seems to say that "threads are not copied" or something [?]

 

[Lee Hinman]

Hey All,
I'm seeing some weirdness trying to get Ara's forkoff gem working on
ruby 1.9.1.

Maybe it doesn't copy the threads over right or something?
http://ruby-doc.faithpromotingstories.org/doc/ruby-1.9.1-p129/classes/Kernel.html#M002828
seems to say that "threads are not copied" or something [?]

I'm wondering if this text might have something to do with it (from
the fork man page,
http://www.opengroup.org/onlinepubs/000095399/functions/fork.html ):

"A process shall be created with a single thread. If a multi-threaded
process calls fork(), the new process shall contain a replica of the
calling thread and its entire address space, possibly including the
states of mutexes and other resources. Consequently, to avoid errors,
the child process may only execute async-signal-safe operations until
such time as one of the exec functions is called. Fork handlers may be
established by means of the pthread_atfork() function in order to
maintain application invariants across fork() calls.
When the application calls fork() from a signal handler and any of the
fork handlers registered by pthread_atfork() calls a function that is
not asynch-signal-safe, the behavior is undefined."

The line "If a multi-threaded process calls fork(), the new process
shall contain a replica of the calling thread and its entire address
space, possibly including the states of mutexes and other resources"
seems to be important to this issue, however I'm not familiar enough
with Ruby's internals to understand how this would cause the
deadlock/hanging, does anyone have any ideas?

- Lee

 

[Eleanor McHugh]

Hey All,
I'm seeing some weirdness trying to get Ara's forkoff gem working on
ruby 1.9.1.

Maybe it doesn't copy the threads over right or something?
http://ruby-doc.faithpromotingstories.org/doc/ruby-1.9.1-p129/classes/Kernel.html#M002828
seems to say that "threads are not copied" or something [?]

I'm wondering if this text might have something to do with it (from
the fork man page,
http://www.opengroup.org/onlinepubs/000095399/functions/fork.html ):
[snip]

The line "If a multi-threaded process calls fork(), the new process
shall contain a replica of the calling thread and its entire address
space, possibly including the states of mutexes and other resources"
seems to be important to this issue, however I'm not familiar enough
with Ruby's internals to understand how this would cause the
deadlock/hanging, does anyone have any ideas?

POSIX threads are managed by the kernel rather than the process, hence
when fork copies a process to a child it only launches the currently
active thread. This is analogous to the situation with file
descriptors where the descriptors are copied into the new process but
they still point to the same underlying data structures rather than
duplicates.

On 1.8.6 with green threads a fork() could result in a child process
with all the threads present in the parent process, however I know
that with the Apple default install of 1.8.6 on Leopard this is not
the case. As I don't have the time to dig into the sources right now
I'll float the hypothesis that 1.8.6 is being built with pthreads.


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

 

[Roger Pack]

On 1.8.6 with green threads a fork() could result in a child process

with all the threads present in the parent process, however I know
that with the Apple default install of 1.8.6 on Leopard this is not
the case. As I don't have the time to dig into the sources right now
I'll float the hypothesis that 1.8.6 is being built with pthreads.

running this code:

Thread.new { puts 'hello'; loop { sleep 1; puts 'in 1' +
Process.pid.to_s} }
fork {
puts 'child' + Process.pid.to_s
sleep
}
puts 'parent' + Process.pid.to_s
sleep

on ruby 1.8.7 [linux, no pthreads] results in only the parent running
the thread, and also 1.9.2 does the same. For what it's worth it seems
that threads really don't run in child processes.

-=r

 

[Eleanor McHugh]

On 1.8.6 with green threads a fork() could result in a child process
with all the threads present in the parent process, however I know
that with the Apple default install of 1.8.6 on Leopard this is not
the case. As I don't have the time to dig into the sources right now
I'll float the hypothesis that 1.8.6 is being built with pthreads.

running this code:

Thread.new { puts 'hello'; loop { sleep 1; puts 'in 1' +
Process.pid.to_s} }
fork {
puts 'child' + Process.pid.to_s
sleep
}
puts 'parent' + Process.pid.to_s
sleep

on ruby 1.8.7 [linux, no pthreads] results in only the parent running
the thread, and also 1.9.2 does the same. For what it's worth it
seems
that threads really don't run in child processes.

Thank goodness for that. Whilst I much prefer the idea of a forked
process being a genuine duplicate of its parent (which I think is more
in line with the original conception of fork) consistency has much to
commend it.


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net