Using Test::Unit to automate reporting of failures

[BenWMills]

I'm using Test::Unit along with the Watir framework to run tests
against my website. I would like to be able to do 2 things:

1. Pass in a "server" parameter to allow the tests to be run against
the developement, staging or production sites.

I know how to get parameters using GetoptLong, but I can't figure out
how to pass these to my Test::Unit::TestCase class. I'm new to Ruby,
so forgive me if I'm way off, but it seems that an instance of the
TestCase class is created internally within Test::Unit. I can't see a
way of creating an instance of my TestCase, passing it the server it
should run the tests against and then running the tests.

2. Report any failures via email.

This time I know how to send the email, but I don't know how to access
the failures after the tests have run. I thought that the TestRunner
might give me access to the failures after the tests have run, but it
doesn't seem to be the case.

Any help on these 2 issues would be much appreciated.

Thanks,
Ben Mills

 

[Jan Svitok]

Hi,

I'm using Test::Unit along with the Watir framework to run tests
against my website. I would like to be able to do 2 things:

1. Pass in a "server" parameter to allow the tests to be run against
the developement, staging or production sites.

I know how to get parameters using GetoptLong, but I can't figure out
how to pass these to my Test::Unit::TestCase class. I'm new to Ruby,
so forgive me if I'm way off, but it seems that an instance of the
TestCase class is created internally within Test::Unit. I can't see a
way of creating an instance of my TestCase, passing it the server it
should run the tests against and then running the tests.

Without knowing your test, watir and without looking into Test::Unit
source, you can
set class variables and check those within your tests (if you don't
need to run more tests with different settings)

class Test < Test::Unit::TestCase
def self.server=(s)
@@server = s
end

def setup
@server = @@server
end
end

Test.server = "whatever"

Or, you can even use class singleton methods:

class Test < Test::Unit::TestCase
class << self
attr_accessor :server
end

def setup
@server = Test.server # or use Test.server wherever you'd use @server
end
end

Test.server = "whatever"

Now, by this I'm not saying that it's impossible to pass the param
directly. I'm just offering a workaround. (Well, after looking into
the source, I don't think it'll be easy to add more parameters to
constructor, but YMMV.)

2. Report any failures via email.

This time I know how to send the email, but I don't know how to access
the failures after the tests have run. I thought that the TestRunner
might give me access to the failures after the tests have run, but it
doesn't seem to be the case.

Look into C:\ruby\lib\ruby\1.8\test\unit\ui\console\testrunner.rb
source and see what you can do.

These are possibilities I can imagine:
- specify IO object that will receive the output in the TestRunner
constructor. Pass StringIO and mail its contents when the test is
done.
- add :attr_reader :failures to UI::Console::TestRunner and use it to
extract the info
- write your own Test::Unit::UI::Email::TestRunner

As you can see, it's useful to read the sources ;-) Don't be afraid of
them. Most of the sources in the library are pretty easy to read, and
they'll teach you a bunch of tricks, as they taught me.

If you'll end up modifying any of the core/stdlib sources, it's best
when you keep your changes separate - by subclassing, or by directly
rewritting some methods in the particular class. It'll be easier when
you upgrade ;-).

Jano