Distributed testing with Test::Unit and Rinda

Discussion in 'Ruby' started by Joe Hosteny iv, Oct 9, 2006.

  1. Hi all,

    I have been working on some code to assist with distributed unit testing
    using Test::Unit and Rinda. I thought I'd post it here assuming that
    someone else might find it interesting or useful. It's a bit raw, and
    I'm still working out some bugs with unclean shutdowns of the test
    servers. Also, it's not documented (yet), but it's only about 240 lines
    of code.

    There are easier ways of doing this, of course, but I had a few
    requirements that caused me to write it this way:

    1) Distribute tests to the test servers on an individual test method
    basis
    2) Avoid (as much as possible) having to rewrite any of the Test::Unit
    code via method aliasing.

    You'll have to run a ring server - see ringserver.rb from Eric Hodel's
    site at http://segment7.net/projects/ruby/drb/rinda/ringserver.html.
    Also, I did not provide the 'attribute_accessors' file, since that is
    just like the one in the rails support package (except that is modified
    to be used in a Module instead of Class). The rest of the files are
    included inline below. Here is an explanation of what to do with each:

    service.rb -

    This file continas definitions for producer/consumer classes for the
    distributed test service, which is shared via a tuple space.

    distributed.rb -

    This file contains mixins for Test::Unit::TestCase and
    Test::Unit::TestSuite that enable them to use the distributed service.

    server.rb -

    Run this on every machine that will be given unit tests to run.

    tests.rb -

    This is a sample unit test file

    test.rb -

    This is a sample master script, which is run as 'ruby test.rb -d
    tests.rb.' If you run 'ruby test.rb tests.rb,' the tests are run
    locally.

    Regards,
    Joe Hosteny
    jhosteny at gmail dot com

    --service.rb--
    require 'rinda/ring'
    require 'rinda/tuplespace'
    require 'rinda/rinda'

    def log *args
    $stdout.write "(#{Thread.current}) "
    puts *args
    $stdout.flush
    end

    module Rinda
    class RingFinger
    # Change this to your local network broadcast netmask
    @@broadcast_list.push("192.168.1.255")
    end
    end

    module Service
    class Base
    def initialize(name)
    @name = name
    DRb.start_service
    log "Started DRb on URI #{DRb.uri}"
    Rinda::RingFinger.primary
    end

    def consumer?
    respond_to? :consume
    end

    def method_missing(meth, *args)
    ts = Thread.current[:tuplespace][2]
    ts = Rinda::TupleSpaceProxy.new(ts) if consumer?
    ts.send(meth, *args)
    end
    end

    class Producer < Base
    def initialize(name)
    super
    ts = Rinda::TupleSpace.new
    name = "#{@name}:#{DRb.uri}"
    tuple = Rinda::RingProvider.new(@name.to_sym, ts, name).provide
    Thread.current[:tuplespace] =
    Rinda::RingFinger.primary.read(tuple)
    trap("EXIT") do
    Rinda::RingFinger.primary.take(Thread.current[:tuplespace])
    end
    end
    end

    class Consumer < Base
    def consume
    tuple = [:name, @name.to_sym, nil, nil]
    Thread.current[:tuplespace] =
    Rinda::RingFinger.primary.take(tuple)
    log "Got tuplespace from URI:
    #{Thread.current[:tuplespace][2].__drburi}"
    begin
    yield self
    ensure
    Rinda::RingFinger.primary.write(Thread.current[:tuplespace])
    end
    end
    end
    end

    --distributed.rb--
    require 'test/unit'
    require 'test/unit/testresult'
    require 'attribute_accessors'
    require 'service'

    module DistributedTestCase
    module ClassMethods
    @@service = nil
    mattr_accessor :service

    @@file = nil
    mattr_accessor :file

    module Run
    end

    def start_client
    @@service = Service::Consumer.new('DistributedTest')
    end
    def start_server
    @@service = Service::producer.new('DistributedTest')
    loop do
    log "Waiting to take test"
    file, name, meth, oid = *(@@service.take([:test, nil]).last)
    log "Loading #{name}::#{meth} in file #{file}"
    load(file)
    klass = nil
    i = 0
    ObjectSpace.each_object do |obj|
    if (obj.class == Class and obj.to_s == name)
    klass = obj
    break
    end
    i += 1
    end
    log "Checked #{i} objects"
    begin
    test = klass.new(meth)
    log "Running #{name}::#{meth} in file #{file})"
    test.run(Test::Unit::TestResultProxy.new(@@service, oid))
    log "Finished running #{name}::#{meth} in file #{file}"
    rescue => e
    @@service.write([:result, oid, :exception, e])
    end
    end
    end

    def inherited(base)
    caller[0] =~ /(.+?):.*/
    @@file = File.expand_path($1)
    end
    end

    class << self
    def included(base)
    base.extend(ClassMethods)
    base.class_eval do
    alias_method :run_original, :run
    alias_method :run, :run_distributed
    end
    end
    end

    def run_distributed(result)
    if ClassMethods.service.consumer?
    th = Thread.new do
    log "New thread"
    ClassMethods.service.consume do |srv|
    oid = method(method_name).object_id
    log "Dispatching test #{self.class.to_s}::#{method_name}
    (#{oid})"
    srv.write [:test, [ClassMethods.file, self.class.to_s,
    method_name, oid]]
    log "Waiting for result from
    #{self.class.to_s}::#{method_name}"
    loop do
    tuple = [:result, oid, nil, nil]
    tuple = srv.take(tuple)
    args, method = tuple.pop, tuple.pop
    log "Test #{self.class.to_s}::#{method_name} called
    #{method}"
    if method == :exception
    raise args.class, "#{args.message}\n\t(remote)
    #{args.backtrace.join("\n\t(remote) ")}\n"
    end
    if %W(add_failure add_error).include? method.to_s
    klass = Test::Unit::Error
    klass = Test::Unit::Failure if method.to_s =~ /failure/
    result.send(method, klass.new(*args))
    else
    result.send(method)
    end
    break if method == :add_run
    end
    end
    log "Thread exiting"
    end
    callcc do |cc|
    throw :new_thread, [th, cc]
    end
    else
    run_original(result) do |s,n| end
    end
    end
    end

    module DistributedTestSuite
    class << self
    def included(base)
    base.class_eval do
    alias_method :run_original, :run
    alias_method :run, :run_distributed
    end
    end
    end

    def run_distributed(result, &block)
    threads = []
    th, cc = *catch:)new_thread) do
    run_original(result, &block)
    nil
    end
    if th
    threads << th
    cc.call
    end
    threads.each { |th| th.join }
    end
    end

    module Test
    module Unit
    class TestSuite
    include DistributedTestSuite
    end
    class TestCase
    include DistributedTestCase
    end
    class TestResultProxy
    def initialize(server, oid)
    @server = server
    @oid = oid
    end

    def method_missing(name, *args)
    name = name.id2name
    if name =~ /add_(.*)/
    if %W(failure error).include? $1
    args = args[0]
    if $1 =~ /failure/
    args = [args.test_name, args.location, args.message]
    else
    args = [args.test_name, args.exception]
    end
    end
    @server.write([:result, @oid, name.to_sym, args])
    end
    end
    end
    end
    end

    --server.rb--
    #!/bin/env ruby
    require 'optparse'
    require 'distributed'

    Test::Unit::TestCase.start_server

    --tests.rb--
    require 'test/unit'

    class TC_MyTest < Test::Unit::TestCase
    def setup
    puts "in setup"
    end

    def teardown
    puts "in teardown"
    end

    def test_it
    assert(false, 'Assertion was false.')
    end

    def test_pass
    assert(true, 'Assertion was true.')
    end
    end

    --test.rb--
    #!/bin/env ruby
    require 'optparse'
    require 'distributed'
    Test::Unit::TestCase.start_client
    require ARGV.shift


    --
    Posted via http://www.ruby-forum.com/.
    Joe Hosteny iv, Oct 9, 2006
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. gardnerpomper

    Distributed unit testing on condor?

    gardnerpomper, Nov 4, 2008, in forum: Python
    Replies:
    0
    Views:
    255
    gardnerpomper
    Nov 4, 2008
  2. Enrique Comba Riepenhausen

    Distributed Ruby/Threads/Rinda?

    Enrique Comba Riepenhausen, May 12, 2007, in forum: Ruby
    Replies:
    0
    Views:
    97
    Enrique Comba Riepenhausen
    May 12, 2007
  3. Hema Latha
    Replies:
    5
    Views:
    247
    Eleanor McHugh
    Oct 7, 2007
  4. Bill Mosteller
    Replies:
    0
    Views:
    223
    Bill Mosteller
    Oct 22, 2009
  5. timr
    Replies:
    2
    Views:
    164
Loading...

Share This Page