Taming Test::Unit

[Ben Giddings]

Recently I've been trying to use Ruby's Test::Unit framework to test a
network server. When I do things a certain way, it works great. It
runs all my tests and gives the expecte results. However there are a
few things I don't understand how to do.

Right now, I have setup and teardown methods that make the TCP
connection to the server. If these are simple like:

def setup
@sock = TCPSocket.new($host, $port)
end

def teardown
@sock.close
end

Then everything works. However, I'd also like to be able to keep the
connection open while each test runs. Each method I try to do this
fails, and I'm not quite sure why.

I first tried:

def setup
if @sock.nil?
@sock = TCPSocket.new($host, $port)
end
end

And commenting out the teardown method, figuring that the socket would
be created only once, and that when the program exited it would take
care of closing the socket, but instead what happens is that the first
test works, but the for subsequent tests, @sock is nil, and because the
socket is never closed, TCPSocket.new fails with a connection refused
method. Does anybody know why @sock would become nil?

I also tried defining an 'initialize' method to set up the socket at the
start of the test case, but evidently 'initialize' is used internally in
some sneaky way I don't understand.

So, could someone who understands the unit test framework better than me
explain how I can:

* Pass a hostname and port as commandline parameters to a script
* Create a socket connected to that host/port
* Run a series of tests using that socket
* Disconnect the socket

Ideally, without using ugly things like global variables, etc.?

Also, in an oviously related question: how can I prevent unit tests from
running just because my file happens to define a class deriving from
Test::Unit::TestCase?

Thanks,

Ben

 

[Joel VanderWerf]

Recently I've been trying to use Ruby's Test::Unit framework to test a
network server. When I do things a certain way, it works great. It
runs all my tests and gives the expecte results. However there are a
few things I don't understand how to do.

Right now, I have setup and teardown methods that make the TCP
connection to the server. If these are simple like:

def setup
@sock = TCPSocket.new($host, $port)
end

def teardown
@sock.close
end

Then everything works. However, I'd also like to be able to keep the
connection open while each test runs. Each method I try to do this
fails, and I'm not quite sure why.

I first tried:

def setup
if @sock.nil?
@sock = TCPSocket.new($host, $port)
end
end

And commenting out the teardown method, figuring that the socket would
be created only once, and that when the program exited it would take
care of closing the socket, but instead what happens is that the first
test works, but the for subsequent tests, @sock is nil, and because the
socket is never closed, TCPSocket.new fails with a connection refused
method. Does anybody know why @sock would become nil?

IIRC, a new instance is created for each test_XXX method, and setup and
teardown are called on that instance. So @sock is nil because setup was
called on a different instance.

I also tried defining an 'initialize' method to set up the socket at the
start of the test case, but evidently 'initialize' is used internally in
some sneaky way I don't understand.

So, could someone who understands the unit test framework better than me
explain how I can:

* Pass a hostname and port as commandline parameters to a script
* Create a socket connected to that host/port
* Run a series of tests using that socket
* Disconnect the socket

Ideally, without using ugly things like global variables, etc.?

Class variables? I wish I had a better answer. Sometimes Test::Unit
doesn't fit the testing patterns I want to use...

 

[Nathaniel Talbott]

Recently I've been trying to use Ruby's Test::Unit framework to test a
network server. When I do things a certain way, it works great. It
runs all my tests and gives the expecte results. However there are a
few things I don't understand how to do.

Hopefully I can help out a bit...

And commenting out the teardown method, figuring that the socket would
be created only once, and that when the program exited it would take
care of closing the socket, but instead what happens is that the first
test works, but the for subsequent tests, @sock is nil, and because
the socket is never closed, TCPSocket.new fails with a connection
refused method. Does anybody know why @sock would become nil?

I also tried defining an 'initialize' method to set up the socket at
the start of the test case, but evidently 'initialize' is used
internally in some sneaky way I don't understand.

As Joel said, each test method runs within the context of a _new_
instance of a test class. Thus you're starting out with a clean slate
every time. This is by design, since it is usually best to test each
piece of functionality in isolation to prevent side-effects from
creating false positives. The one place where this is not practical is
when testing resources are expensive; then it makes sense to only set
things up once (a fact you're obviously already acquainted with).

So, could someone who understands the unit test framework better than
me explain how I can:

* Pass a hostname and port as commandline parameters to a script
* Create a socket connected to that host/port
* Run a series of tests using that socket
* Disconnect the socket

Ideally, without using ugly things like global variables, etc.?

There are several ways, none of them satisfactory. First of all, as
Joel already mentioned, lazy initialize the resource and stuff it in a
class variable (he problematic thing becomes disconnecting it when
you're done - I don't have a good solution for this). Another way is to
build your suites manually, and override TestSuite#run to call
suite-level setup and teardown methods.

While I'm quite busy right now, I'll be finishing up the project that
I'm on at the end of the month, and I plan to give Test::Unit some
much-needed TLC. So please be patient with me... I haven't forgotten it


[ P.S. Test::Unit won't touch any command-line arguments after a '--'
in the list, so you should be able to just read the hostname and port
out of the ARGV array. ]

Also, in an oviously related question: how can I prevent unit tests
from running just because my file happens to define a class deriving
from Test::Unit::TestCase?

Well, I can't really see how it's related, but the best way I know of
is to not subclass TestCase except when you want the subclass to be
run. Instead, mix-in common functionality from a Module.

HTH,


Nathaniel
Terralien, Inc.

<(><

 

[Bret Pettichord]

There are several ways, none of them satisfactory. First of all, as Joel
already mentioned, lazy initialize the resource and stuff it in a class
variable (he problematic thing becomes disconnecting it when you're done -
I don't have a good solution for this).

What about putting the teardown in an END block?

Another way is to build your suites manually, and override TestSuite#run
to call suite-level setup and teardown methods.

I like this suggestion. It might be nice to do this automatically in future
versions of test-unit.

[ P.S. Test::Unit won't touch any command-line arguments after a '--' in
the list, so you should be able to just read the hostname and port out of
the ARGV array. ]

Another approach is to do this:

# ARGV need to be deleted to enable the Test::Unit functionatily that grabs
# the remaining ARGV as a filter on what tests to run
$HIDE_IE = ARGV.include?('-b'); ARGV.delete('-b')


______________________________________
Bret Pettichord, Software Tester
Consultant - www.pettichord.com
Author - www.testinglessons.com
Blogger - www.io.com/~wazmo/blog

Scripting Web Tests Tutorial
XP/Agile Universe, Aug 15, Calgary
www.pettichord.com/training.html