run test scripts concurrently

[Chris McMahon]

I've been futzing with this for some time and I am out of ideas.
I have a number of files named "test_something.rb" that each contain
one test like so:

class TC_foo < Test::Unit::TestCase
def test_foo
#do some stuff
#assert some things
end
end

I have a little harness that runs them one after another like so:

topdir = File.join(File.dirname(__FILE__))
Dir.chdir topdir do
tests = Dir["test*"]
tests.each{|x| require x}
end

But I would like to be able to run each individual script file in it's
own process simultaneously. I have been fooling around with
Thread.new, system(), exec(), and it seems that no matter what I try,
each script file has to finish before the next one will run.

Any suggestions for running all of my test files at the same time from
one controller/harness?

 

[ara.t.howard]

I've been futzing with this for some time and I am out of ideas.
I have a number of files named "test_something.rb" that each contain
one test like so:

class TC_foo < Test::Unit::TestCase
def test_foo
#do some stuff
#assert some things
end
end

I have a little harness that runs them one after another like so:

topdir = File.join(File.dirname(__FILE__))
Dir.chdir topdir do
tests = Dir["test*"]
tests.each{|x| require x}
end

But I would like to be able to run each individual script file in it's
own process simultaneously. I have been fooling around with
Thread.new, system(), exec(), and it seems that no matter what I try,
each script file has to finish before the next one will run.

Any suggestions for running all of my test files at the same time from
one controller/harness?

threads = []

Dir.chdir topdir do
tests = Dir["test*"]
tests.each{|x| threads << Thread.new{ system x or raise x } }
end

threads.each{|t| t.join}

-a

 

[Joel VanderWerf]

In my expereince, if you join all the spawned threads to the current thread,
this guarantees that they will run sequentially. They will certainly run,
which is a good thing, but they won't run concurrently.

BTW I am not saying this about threads in general, I am saying it about Ruby
threads.

This seems concurrent to me, or did you mean something different:

threads = (0..9).map {|i| Thread.new {sleep rand(5); puts i}}
threads.each {|t| t.join}

__END__

Output:

4
8
9
0
7
5
2
1
6
3

 

[Charles Oliver Nutter]

I tried to think of an explanation apart from the obvious one, but it seems
I am wrong. I think the fact that they are all sleeping their time away may
partly explain this outcome, but the threads are clearly running
concurrently.

Threads that are not sleeping may not live up to the promise of this
example. My threads tend not to run concurrently, but your example is an
excellent refutation.

Threads that are 100% Ruby code will yield and timeslice correctly.
However they still will never run concurrently, other than timeslicing.
Threads that make system calls will run sequentially, since system calls
can't be scheduled by Ruby's thread scheduler. If you have a thread make
a system call, that call must complete before the thread will yield.

There's some trickery with IO in some cases, but for the general case
this is how it works. You may try JRuby, which has fully concurrent
native thread support, but not everything in normal Ruby is 100%
supported yet...and Kernel#system isn't quite perfect yet.

 

[Joel VanderWerf]

Threads that are 100% Ruby code will yield and timeslice correctly.
However they still will never run concurrently, other than timeslicing.
Threads that make system calls will run sequentially, since system calls
can't be scheduled by Ruby's thread scheduler. If you have a thread make
a system call, that call must complete before the thread will yield.

That's true at the C API level, but not always true at the Ruby API
level, as you say...

There's some trickery with IO in some cases, but for the general case
this is how it works. You may try JRuby, which has fully concurrent
native thread support, but not everything in normal Ruby is 100%
supported yet...and Kernel#system isn't quite perfect yet.

Is select() really trickery? Anyway, in addition to the IO trickery,
there is also a concurrent #system, as this example shows:

$ cat x.rb
t = Thread.new do
system "sleep 1; echo SYSTEM; sleep 1; echo SYSTEM; sleep 1; echo SYSTEM"
end

3.times do
sleep 1
puts "RUBY"
end

t.join

$ ruby x.rb
RUBY
SYSTEM
RUBY
SYSTEM
RUBY
SYSTEM

For some purposes (networking) ruby's threads are pretty good, to a
point. For some other purposes, we can use #fork plus drb. There are
cases where one is SOL though.

We're all looking forward to native threads in future ruby VMs and JRuby.