Parameterized tests with test/unit

[Alexey Verkhovsky]

Ruby is amazing... Try to do this in Java (it is possible, but not in so
concise and readable way).

Speaking of tests, is there any known way to get test/unit report the
results in XML, a la Ant Junit runner? Other markup easily convertible
to HTML would do just as fine.

<code>

require 'test/unit'

class ClassUnderTest
def square(i)
i * i
end
end

class ParaTest < Test::Unit::TestCase
# test methods are added dynamically to this class later in this file
end

expectedResults = {-5 => 25, -1 => 1, 0 => 0, 1 => 1, 10 => 100,
999999999 => 999999998000000001}

expectedResults.each_pair do |parameter, expectedResult|
ParaTest.module_eval <<-EOL
def testSquareOf#{parameter.to_s.sub('-', 'Minus')}
assert_equal #{expectedResult},
ClassUnderTest.new.square(#{parameter})
end
EOL
end

</code>

Best regards,
Alexey Verkhovsky

 

[Gavin Sinclair]

Ruby is amazing... Try to do this in Java (it is possible, but not in so
concise and readable way).

[...]

expectedResults = {-5 => 25, -1 => 1, 0 => 0, 1 => 1, 10 => 100,

999999999 =>> 999999998000000001}

expectedResults.each_pair do |parameter, expectedResult|
ParaTest.module_eval <<-EOL
def testSquareOf#{parameter.to_s.sub('-', 'Minus')}
assert_equal #{expectedResult},
ClassUnderTest.new.square(#{parameter})
end
EOL
end

Nifty, but I'd actually prefer to put all those tests in the one
method in cases like that.

Cheers,
Gavin

 

[Alexey Verkhovsky]

Nifty,

"Nifty" is putting it mildly

but I'd actually prefer to put all those tests in the one
method in cases like that.

For unit tests, such as my toy example, I would do the same.

In my situation (I try to use Ruby's test/unit for functional testing of
batch jobs) putting each parameter into its own test, thus ensuring that
if one assertion breaks other values are still tested, has its merit.

Best regards,
Alex

 

[Nathaniel Talbott]

Speaking of tests, is there any known way to get test/unit report the
results in XML, a la Ant Junit runner? Other markup easily convertible
to HTML would do just as fine.

Not presently... although you could certainly write a runner to do so.
It'd have a good chance of making its way in to the standard
distribution if you did.


Nathaniel
Terralien, Inc.

<(><

 

[Alexey Verkhovsky]

Not presently... although you could certainly write a runner to do so.
It'd have a good chance of making its way in to the standard
distribution if you did.

I think I will try to do that.

Can you help me get the "specification" right?

I see it as follows:

ui/batch/testrunner.rb, which is a subclass of
ui/console/testrunner.rb. It emits same output as console runner to
stdout/stderr, but also writes one XML file for each test suite it has
to run.

Naming convention and format of the XML is same as Ant JUnit runner (to
reuse Ant formatter / stylesheets).

XML files are placed in the directory specified in a mandatory argument
to the constructor.

Alternatives:

1. Output directory as an optional argument. I think, very few people
really want to write test reports to the root of the project during the
build, and therefore such default behavior would be counter-intuitive.
Better to fail cleanly with a good error message.

2. Emit output as YAML, and give YAML -> HTML conversion as part of the
package. That would be more interesting, but harder. Is there a good
YAML -> HTML converter?

3. Can probably package this feature as a mixin to anything that
subclasses TestRunnerUtils.

Brgds,
Alex

 

[Nathaniel Talbott]

Can you help me get the "specification" right?

I see it as follows:

ui/batch/testrunner.rb, which is a subclass of
ui/console/testrunner.rb. It emits same output as console runner to
stdout/stderr, but also writes one XML file for each test suite it has
to run.

Naming convention and format of the XML is same as Ant JUnit runner (to
reuse Ant formatter / stylesheets).

XML files are placed in the directory specified in a mandatory argument
to the constructor.

That sounds very reasonable; one possible alternative to subclassing
would be to just create a delegating wrapper around the runner (since
you want to run normally PLUS outputting XML). Just grab the mediator
instance variable out of the runner (you'll have to use instance_eval
or somesuch as there's not currently a public method to get it) and
attach your own listeners to it. The advantage to this is that you can
theoretically wrap any mediator-using runner.

HTH,


Nathaniel
Terralien, Inc.

<(><

 

[Alexey Verkhovsky]

one possible alternative to subclassing
would be to just create a delegating wrapper around the runner (since
you want to run normally PLUS outputting XML).

If I understand you right, you mean some sort of decorator like this:

Test::Unit::TestReporter.newXML, 'build/reports', AllTests.suite)

Correct?

Alex

 

[Nathaniel Talbott]

If I understand you right, you mean some sort of decorator like this:

Test::Unit::TestReporter.newXML, 'build/reports', AllTests.suite)

Correct?

I was thinking more like (untested):

class RunnerWrapper < SimpleDelegate
def initialize(runner, output_dir, format = :xml)
super(runner)

add_listeners(runner.instance_eval{@mediator})
end

def add_listeners(mediator)
# Collect the information you're interested in

# When you receive the TestRunnerMediator::FINISHED event,
# write out the report
end
end

Does that help?


Nathaniel
Terralien, Inc.

<(><

 

[Alexey Verkhovsky]

I was thinking more like (untested):

class RunnerWrapper < SimpleDelegate
def initialize(runner, output_dir, format = :xml)
super(runner)

add_listeners(runner.instance_eval{@mediator})
end

Alas, this doesn't work because runner.@mediator is instantiated in
runner.start, not in the constructor. So far I haven't found any way of
safely getting mediator instance from the runner.

To move forward, I have added the following line in the
TestRunner#start:

Index: testrunner.rb
===================================================================
RCS file: /src/ruby/lib/test/unit/ui/console/testrunner.rb,v
retrieving revision 1.7
diff -c -r1.7 testrunner.rb
*** testrunner.rb 21 Apr 2004 06:56:48 -0000 1.7
--- testrunner.rb 30 Jul 2004 22:48:30 -0000
***************
*** 38,43 ****
--- 38,44 ----
def start
setup_mediator
attach_to_mediator
+ yield @mediator if block_given?
return start_mediator
end

Not sure, if this is the right approach, but it is quite obvious to me
that test runners need to be somehow refactored in order to support the
kind of decoration we are talking about.

By the way, I fully agree with this comment:
# Perhaps there ought to be a TestRunner superclass? There
# seems to be a decent amount of shared code between test
# runners.

Please decide and tell me how it will be. Meantime, I go ahead with the
results formatter.

Best regards,
Alex