Test::Unit speed

[Francis Hwang]

Okay, this is sort of embarrassing, but for some time now I've been
using the (deprecated) RubyUnit instead of Test::Unit. Since RubyConf
I've been using Test::Unit for newer projects, and I'm definitely happy
with all the various improvements.

But now I'm trying to switch over a pre-existing private project to
Test::Unit, and it seems to run a lot slower than RubyUnit. I've got
about 350 unit tests, which used to run in about a minute in RubyUnit.
But now when I try to run them in Test::Unit, they won't even complete.
Individual tests run fine, but everything seems like it's a lot slower.

Has anybody else encountered this problem? Any particular quirks or
workarounds I should be aware of? Thanks in advance.

F.

 

[James Edward Gray II]

Has anybody else encountered this problem? Any particular quirks or
workarounds I should be aware of? Thanks in advance.

In my testing of quiz submissions for Regex.build(), I ran into some
nasty slowdowns with Test::Unit. I was running a LOT of tests, but it
was still quite a bit slower than it needed to be.

James Edward Gray II

 

[Francis Hwang]

Let me ask more specifically: Is anybody out there using Test::Unit to
run hundreds of tests? If you are, and have any tips as to how I can
manage the speed issue, your tips would be much much appreciated.

F.

 

[Nathaniel Talbott]

 

[Nathaniel Talbott]

Let me ask more specifically: Is anybody out there using Test::Unit to
run hundreds of tests? If you are, and have any tips as to how I can
manage the speed issue, your tips would be much much appreciated.

Francis,

I've certainly had test suites with hundreds of tests, and never seemed
to have problems. Have you been able to run a profiler on it? I'd be
more than willing to apply patches, especially if they don't affect the
API.


Nathaniel
Terralien, Inc.

<(><

 

[Francis Hwang]

Hi Nathaniel,

I'll start playing around with a few of the tests, and profiling them.
I just wanted to check first if others were trying to test on this
scale before I bothered. It's quite possible I'm just doing something
braindead. I'll poke around and let you know what I find.

F.

 

[Tim Sutherland]

Okay, this is sort of embarrassing, but for some time now I've been
using the (deprecated) RubyUnit instead of Test::Unit. Since RubyConf
I've been using Test::Unit for newer projects, and I'm definitely happy
with all the various improvements.

But now I'm trying to switch over a pre-existing private project to
Test::Unit, and it seems to run a lot slower than RubyUnit. I've got
about 350 unit tests, which used to run in about a minute in RubyUnit.
But now when I try to run them in Test::Unit, they won't even complete.
Individual tests run fine, but everything seems like it's a lot slower.

Has anybody else encountered this problem? Any particular quirks or
workarounds I should be aware of? Thanks in advance.

I never used RubyUnit, but the setup and teardown methods are called once
per test-case. Perhaps you're expecting that they're only called once per
TestCase class.

 

[Eric Hodel]

Let me ask more specifically: Is anybody out there using Test::Unit to
run hundreds of tests? If you are, and have any tips as to how I can
manage the speed issue, your tips would be much much appreciated.

On a lowly Pentium 233, I run 303 tests from Ruby2C in 12 seconds. The
time to run the test suite starting from 10s of tests seems to have
grown linearly. On a decent box (PowerBook) the tests can all run in
1-2 seconds.

$ wc -l test_* | grep total
3793 total
$ time ./test_all.rb
Loaded suite ./test_all
Started
........................................................................
........................................................................
........................................................................
........................................................................
...............
Finished in 12.027582 seconds.

303 tests, 481 assertions, 0 failures, 0 errors

real 0m13.810s
user 0m12.445s
sys 0m0.168s

 

[Francis Hwang]

Just wanted to follow up on this, a month later: The speed issues I
thought were coming from Test::Unit was actually coming from a lot of
different issues relating to switching to Ruby 1.8.2 and using certain
libs inefficiently. Test::Unit doesn't seem to be causing any problems,
and it's definitely got a lot of nice features that make organizing my
test cases easier. Thanks, Nathaniel!

One question I haven't been able to resolve: Is there a way to use
multiple file args or fileglobs to have more than one test-case file
loaded in at the command line? When I do something like

ruby test/article/ArticleIndexElement.rb test/article/ArticleSummary.rb

or just

ruby test/article/*.rb

then Test::Unit seems to just load in the first TestCase and drop
everything else.

Francis,

I've certainly had test suites with hundreds of tests, and never
seemed to have problems. Have you been able to run a profiler on it?
I'd be more than willing to apply patches, especially if they don't
affect the API.


Nathaniel
Terralien, Inc.

<(><

Francis Hwang
http://fhwang.net/