Why does a test fail when I predicted it's exception class?

[Richard]

Hi,

I have a test suite that invokes on test set that specifies

assert_raise(ZeroDivisionError, MyCalc.new.calc)

When I run the test suite, instead of "1 test, 0 failures, 0 errors",
I get the following. The code follows this output.

What's up?
Richard

K:\_Projects\Ruby\TestUnitTesting\BasicTest>ruby MyCalcTest.rb
..\BasicCalc.rb:5:in `/': divided by 0 (ZeroDivisionError)
from .\BasicCalc.rb:5:in `calc'
from .\BasicCalc.rb:9
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'
from ./tc_TestSet.rb:1
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'
from MyCalcTest.rb:4

BasicCalc.rb
=========
# BasicCalc.rb

class MyCalc
def calc
2/0
end
end

tc_TestSet.rb
==========
require '.\BasicCalc.rb'

class Test1 < Test::Unit::TestCase # The long-winded way
def test_excep
assert_raise(ZeroDivisionError, MyCalc.new.calc)
end
end

MyCalcTest.rb
==========
# Reverse-Polish Evaluation Test

require 'test/unit'
require 'tc_TestSet'

 

[Robert Feldt]

Hi,

I have a test suite that invokes on test set that specifies

assert_raise(ZeroDivisionError, MyCalc.new.calc)

When I run the test suite, instead of "1 test, 0 failures, 0 errors",
I get the following. The code follows this output.

What's up?

You are using assert_raise in the wrong way. The call you want to test
must be wrapped in a block so that it's exception can be catched and
compared to the expected one.

Try something like

assert_raise(RuntimeError) {raise ""}

/Robert

 

[Louis J Scoras]

tc_TestSet.rb
==========
require '.\BasicCalc.rb'

class Test1 < Test::Unit::TestCase # The long-winded way
def test_excep
assert_raise(ZeroDivisionError, MyCalc.new.calc)
end
end

You need to run the code that raises the exception in a block.

assert_raise(ZeroDivisionError) { MyCalc.new.calc }

should do the trick.

 

[Richard]

Hi Robert & Louis,

You guys were right on! Below is one of my actual test sets, the one
with the exception testing. It is more complicated than the toy I
posted, so it took me a few syntax-errors to get it to work. But then
I got my desired: 17 tests, 17 assertions, 0 failures, 0 errors

Thank you very much for your responses.

Best wishes,
Richard



require '.\RevPolishEvaluator.rb'

class ErrorTests < Test::Unit::TestCase # The succinct way
def test1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $ 7 --
*}).eval}; end
def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
+}).eval}; end
def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
end

 

[Eric Hodel]

You guys were right on! Below is one of my actual test sets, the one
with the exception testing. It is more complicated than the toy I
posted, so it took me a few syntax-errors to get it to work. But then
I got my desired: 17 tests, 17 assertions, 0 failures, 0 errors

require '.\RevPolishEvaluator.rb'

class ErrorTests < Test::Unit::TestCase # The succinct way
def test1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $
7 --
*}).eval}; end
def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
+}).eval}; end
def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
end

Better:

require 'test/unit'
require 'rev_polish_evaluator'

class TestPolish < Test::Unit::TestCase

def test_eval_bad_operator
# ...
end

def test_eval_divide_by_zero
# ...
end

def test_eval_no_input
e = assert_raise ArgumentError do Polish.new.eval end
assert_equal 'no arguments supplied', e.message
end

def test_eval_missing_data
# ...
end

end

Written this way your tests:

* follow the test/unit naming scheme (TestBlah where Blah is the
class you're testing)
* be in the same order as the output (alphabetical)
* will work with the -n flag:

$ ruby test_polish.rb -n /eval/

* will work with testrb:

$ testrb .

 

[Richard]

Hi Eric,

Thanks for weighing in on my question.

def test_eval_no_input

e = assert_raise ArgumentError do Polish.new.eval end
assert_equal 'no arguments supplied', e.message
end

I like the additions here a lot:
1. Capturing the exception object
2. Checking not only the correct exception type is raised but also
checking the programmed message passed to 'raise'

def test_eval_bad_operator

1. I take it that to conform to unit-test's expectations, the test
names should begin with test_ and not test1_, test2_ etc.

2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
Wouldn't that conform equally well and cause any error msgs produced by
Test to be presented in lexicographical order (with fewer than 10
tests; double digits if I wanted up to 99 tests, etc.)?

3. I like to produce tests in one-line format if they'll fit on my
screen reasonably so that I can write and scan them more quickly. Do
see any substantive problem in my continuing to do that?

$ ruby test_polish.rb -n /eval/

I'm running Ruby_1.8.2-15 over WinXP-Pro/SP2. I tried this command and
got nothing:

=== Command Window ====
K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>ruby
RevPolishEvaluator.rb -n /eval/

K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
====== end ============

I thought the problem might be that I had no errors, so that there was
nothing for Test to report. So I introduced an erroneous assertion
(expressed in three lines rather than my preferred one-line format)
but still got nothing.

Then I ran this expanded test through SciTE and got (both in one-line
format as well as three-line format):

==== SciTE output ====

ruby RevPolishEvaluatorTest.rb

Loaded suite RevPolishEvaluatorTest
Started
..............F....
Finished in 0.031 seconds.

1) Failure:
test_5_dummy(ErrorTests) [./tc_TestSet2.rb:12]:
<1> expected but was
<2>.

18 tests, 18 assertions, 1 failures, 0 errors

Exit code: 1

=== end ===

Do you have any idea why the command with the "-n" switch failed on my
system? My code follows.

Again, thanks for your suggestions. I look forward to your comments.

Best wishes,
Richard

=== RevPolishEvaluator.rb ===
### Code same as posted earlier
=== end ===

=== RevPolishEvaluatorTest.rb ===
# Reverse-Polish Evaluation Test

require 'test/unit'
require 'tc_TestSet1' # <= 13 tests that produced no errors;
file not shown here
require 'tc_TestSet2'
=== end ===

=== tc_TestSet2.rb ===
require '.\RevPolishEvaluator.rb'

class ErrorTests < Test::Unit::TestCase # The succinct way
def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test_2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $ 7
-- *}).eval}; end
def test_3_missingdata
e = assert_raise(ArgumentError) {Polish.new(%w{2 +}).eval}
assert_equal('Stack underflow', e.message) #
<= Had to change the msg to confirm to my coding
end
def test_4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
def test_5_dummy
assert_equal(1, 2)
end
end
=== end ===

 

[Eric Hodel]

1. I take it that to conform to unit-test's expectations, the test
names should begin with test_ and not test1_, test2_ etc.

Numbering tests is not typical behavior (I've seen it very rarely).
Using #setup and #teardown its easy to avoid the need to number tests.

The only restrictions test/unit has are that the must start with
'test' and accept 0 arguments.

2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
Wouldn't that conform equally well and cause any error msgs
produced by
Test to be presented in lexicographical order (with fewer than 10
tests; double digits if I wanted up to 99 tests, etc.)?

I match test methods to implementation methods and test classes to
implementation classes. This allows easy auditing (for example with
ZenTest) or even visually of your test coverage.

def test_foo() end

matches

def foo() end

and if I need to test a couple edge-cases of foo:

def test_foo_empty_foopy() end
def test_foo_no_blah() end

And everything stays in run order.

3. I like to produce tests in one-line format if they'll fit on my
screen reasonably so that I can write and scan them more quickly. Do
see any substantive problem in my continuing to do that?

I don't care, they're your tests (but I hate ; so I had to get rid of
them.)

I sometimes write:

def some_method() do_the_stuff end

because the ; is ugly, but almost always for test stub objects.

Most people add newlines.

I'm running Ruby_1.8.2-15 over WinXP-Pro/SP2. I tried this command
and
got nothing:

=== Command Window ====
K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>ruby
RevPolishEvaluator.rb -n /eval/

K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
====== end ============

I thought the problem might be that I had no errors, so that there was
nothing for Test to report. So I introduced an erroneous assertion
(expressed in three lines rather than my preferred one-line format)
but still got nothing.

You ran your implementation, not your tests.

Then I ran this expanded test through SciTE and got (both in one-line
format as well as three-line format):

==== SciTE output ====
Loaded suite RevPolishEvaluatorTest
Started
.............F....
Finished in 0.031 seconds.

This one ran with -n

Do you have any idea why the command with the "-n" switch failed on my
system? My code follows.

Try again running your test file

Again, thanks for your suggestions. I look forward to your comments.

Oh, also you can make your test shorter with setup:

=== tc_TestSet2.rb ===
require '.\RevPolishEvaluator.rb'

class ErrorTests < Test::Unit::TestCase # The succinct way

def setup() @p = Polish.new end

def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end

def test_1_noarg() assert_raise ArgumentError do @p.eval end end

(As you can see, I hate punctuation.)

 

[Richard]

Hi Erik,

Hi Eric,

Using #setup and #teardown its easy to avoid the need to number tests.

Thanks for mentioning them.

The only restrictions test/unit has are that the must start with
'test' and accept 0 arguments. Excellent

I match test methods to implementation methods and test classes to
implementation classes. This allows easy auditing [snip]

I like that, too.

And everything stays in run order.

Ah, that's what I didn't know. I thought my numbering might be
required to achieve that.

I don't care, they're your tests [snip] Great.

Do you have any idea why the command with the "-n" switch failed on my
system? My code follows.

Try again running your test file

Woops!

Your answers here solved a problem I posted yesterday about another
messed up test. I have now deleted that post because "I've seen the
light."

Below is a toy testing setup. I stuck with my numbered approach for
the moment because it fit these tests. This setup worked quite nicely.
If you're in the mood to comment on them, I'd be a grateful recipient.
But I think I'm OK on this topic now. Incidentally, I thought about
trying to use lamda expressions to reduce the duplication, but enough
is enough.

Again, thank you for providing such thoughtful comments.

Best wishes,
Richard

SetupTeardown.rb
=============
class Foo
MyConst = "Foo's Constant"
def initialize(n)
@n = n
end
def bar
"I'm Foo#bar with #{MyConst}#{@n}"
end
end

puts Foo.new(1).bar if __FILE__ == $0

TestSetupTeardown.rb
================
# "TestSetupTeardown"

require 'test/unit'
require 'tc_Test1'

tc_Test.rb
========
require './SetupTeardown'
require 'test/unit'
class TestST < Test::Unit::TestCase
def setup
@instance1 = Foo.new(1)
@instance2 = Foo.new(2)
end
def teardown
@instance1 = nil
@instance2 = nil
end
def test1
assert_equal("I'm Foo#bar with Foo's Constant1",
@instance1.bar)
assert_equal("I'm Foo#bar with Foo's Constant2",
@instance2.bar)
end
end

Result (using SciTE)
=====

ruby TestSetupTeardown.rb

Loaded suite TestSetupTeardown
Started
..
Finished in 0.0 seconds.

1 tests, 2 assertions, 0 failures, 0 errors

 

[Rob Sanheim]

Hi,

Your answers here solved a problem I posted yesterday about another
messed up test. I have now deleted that post because "I've seen the
light."

Below is a toy testing setup. I stuck with my numbered approach for
the moment because it fit these tests. This setup worked quite nicely.
If you're in the mood to comment on them, I'd be a grateful recipient.
But I think I'm OK on this topic now. Incidentally, I thought about
trying to use lamda expressions to reduce the duplication, but enough
is enough.

Why do you want to number your tests? Your tests should be written in
such a way that they don't depend upon the order they run, so
numbering them to try and get a specific order is a bad idea. If its
just to get a nice output when you run the suite, you should look at
tweaking the test runner or autotest or whatever you use to run your
tests.

Again, thank you for providing such thoughtful comments.

Best wishes,
Richard

SetupTeardown.rb
=============
class Foo
MyConst = "Foo's Constant"
def initialize(n)
@n = n
end
def bar
"I'm Foo#bar with #{MyConst}#{@n}"
end
end

puts Foo.new(1).bar if __FILE__ == $0

TestSetupTeardown.rb
================
# "TestSetupTeardown"

require 'test/unit'
require 'tc_Test1'

tc_Test.rb
========
require './SetupTeardown'
require 'test/unit'
class TestST < Test::Unit::TestCase
def setup
@instance1 = Foo.new(1)
@instance2 = Foo.new(2)
end
def teardown
@instance1 = nil
@instance2 = nil

You don't need to set these to nil, as a new instance of the test
class will be create for each test case. You normally only see
teardown used for things like closing a database connection, making
sure a stubbed method gets removed, etc.

end
def test1
assert_equal("I'm Foo#bar with Foo's Constant1",
@instance1.bar)
assert_equal("I'm Foo#bar with Foo's Constant2",
@instance2.bar)
end
end

- Rob

 

[Richard]

Hi Robm,

Thanks for adding your views to this discussion.

Why do you want to number your tests? Your tests should be written in
such a way that they don't depend upon the order they run, so
numbering them to try and get a specific order is a bad idea. If its
just to get a nice output when you run the suite, you should look at
tweaking the test runner or autotest or whatever you use to run your
tests.

I like the numbering so that when I start to develop a slew of tests
for a real app and get a failure(s), I can quickly locate the offending
test in cases where the app is right and the test needs to be
corrected. Maybe when I get some real experience in Test Driven
Development I'll find that's not as helpful as I anticipate it to be.

I don't do it to get the tests to run in any particular order.

You don't need to set these to nil, as a new instance of the test
class will be create for each test case. You normally only see
teardown used for things like closing a database connection, making
sure a stubbed method gets removed, etc.

OK.

Again, thanks for responding

Best wishes,
Richard