Gems undermine unit testing

[Bret Pettichord]

I put the following line at the top of each of my unit tests for Watir:

$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..') if $0 == __FILE__

This ensures that the version of Watir that is being tested is the one that
is in the same hierarchy as the unit tests (by putting its directory in the
front of the load-path). You can accomplish the same thing by setting the
load path at the command line.

... Unless you are using a gem. I recently started packaging Watir as a gem
and i now see that, somehow, the gem mechanism prepends the gem library
directories to the very front, so the gem-installed version is tested
instead of the one under development.

Now i need to make sure that i do a "gem uninstall watir" before running
unit tests or else they are going to be testing the wrong version -- the
one that has already been tested and released. This is a type II error --
the tests say they've passed but in fact they haven't actually tested what
they were supposed to and could easily miss errors.

Since i have many contributors to the project, i would like a more reliable
mechanism to help ensure that the people running unit tests are actually
testing the version under development.

Any suggestions?

Bret


_____________________
Bret Pettichord
www.pettichord.com

 

[Jim Weirich]

I put the following line at the top of each of my unit tests for Watir:

$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..') if $0 ==
__FILE__

This ensures that the version of Watir that is being tested is the one that
is in the same hierarchy as the unit tests (by putting its directory in the
front of the load-path). You can accomplish the same thing by setting the
load path at the command line.

... Unless you are using a gem. I recently started packaging Watir as a gem
and i now see that, somehow, the gem mechanism prepends the gem library
directories to the very front, so the gem-installed version is tested
instead of the one under development.

Bret,

I release all my software as gems They are installed on my system as gems and
yet am able to test all my software without contortions. I use the Rake test
task to run my tests (and it sets up the library paths on the command line).

If your software is available in the load path, gems will not attempt load the
gem in front of that. An explicit require_gem will try to load a particular
version of your library, so you don't want to use require_gem in your test
setup (and probably don't need it in the real code either).

 

[Bret Pettichord]

I release all my software as gems They are installed on my system as gems
and
yet am able to test all my software without contortions. I use the Rake test
task to run my tests (and it sets up the library paths on the command line).

I'll take a look at this. I've just started to use Rake.

If your software is available in the load path, gems will not attempt load
the
gem in front of that. An explicit require_gem will try to load a particular
version of your library, so you don't want to use require_gem in your test
setup (and probably don't need it in the real code either).

I checked the load path in my testsuite, after executing "require 'watir'".
And i saw that the gem path to watir was on the load path before the
directory that had the development version. I don't use require_gem. But
you are telling me that that wasn't supposed to happen.

After i uninstalled the watir gem, i was able to run my tests correctly.

Bret


_____________________
Bret Pettichord
www.pettichord.com

 

[Devin Mullins]

I checked the load path in my testsuite, after executing "require
'watir'". And i saw that the gem path to watir was on the load path
before the directory that had the development version. I don't use
require_gem. But you are telling me that that wasn't supposed to happen.

Hmmm... I don't know a lot about gems, but it's certainly counter to my
setup:

C:\prog\rubycode>notepad binding_of_caller.rb

C:\prog\rubycode>type binding_of_caller.rb
puts <<'YAY!!!'
I'm a little teapot
Short and stout.
This is my handle.
This is my spout.

When I get all steamed up,
Hear me shout:
"Tip me over and
Pour me out!"
YAY!!!

C:\prog\rubycode>notepad test.rb

C:\prog\rubycode>type test.rb
require 'pp'
require 'binding_of_caller'
puts
pp $:

C:\prog\rubycode>ruby test.rb
I'm a little teapot
Short and stout.
This is my handle.
This is my spout.

When I get all steamed up,
Hear me shout:
"Tip me over and
Pour me out!"

["c:/prog/ruby/lib/ruby/site_ruby/1.8",
"c:/prog/ruby/lib/ruby/site_ruby/1.8/i386-msvcrt",
"c:/prog/ruby/lib/ruby/site_ruby",
"c:/prog/ruby/lib/ruby/1.8",
"c:/prog/ruby/lib/ruby/1.8/i386-mswin32",
"."]

C:\prog\rubycode>del binding_of_caller.rb

C:\prog\rubycode>ruby test.rb

["c:/prog/ruby/lib/ruby/gems/1.8/gems/rails-0.13.1/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/rails-0.13.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionwebservice-0.8.1/lib/action_web_service/vendor/",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionwebservice-0.8.1/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionwebservice-0.8.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionmailer-1.0.1/lib/action_mailer/vendor/",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionmailer-1.0.1/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionmailer-1.0.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionpack-1.9.1/lib/action_view/helpers/../../action_controller/vendor/html-scanner",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionpack-1.9.1/lib/action_view/vendor",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionpack-1.9.1/lib/../../activesupport/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionpack-1.9.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionpack-1.9.1/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/actionpack-1.9.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activesupport-1.1.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activerecord-1.11.1/lib/../../activesupport/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activerecord-1.11.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activerecord-1.11.1/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activerecord-1.11.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activesupport-1.1.1/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/activesupport-1.1.1/lib",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/rake-0.5.4/bin",
"c:/prog/ruby/lib/ruby/gems/1.8/gems/rake-0.5.4/lib",
"c:/prog/ruby/lib/ruby/site_ruby/1.8",
"c:/prog/ruby/lib/ruby/site_ruby/1.8/i386-msvcrt",
"c:/prog/ruby/lib/ruby/site_ruby",
"c:/prog/ruby/lib/ruby/1.8",
"c:/prog/ruby/lib/ruby/1.8/i386-mswin32",
"."]

After i uninstalled the watir gem, i was able to run my tests correctly.

BTW, using gems doesn't preclude being version-specific, unless you are
using Edge Watir a lot, and dealing with cases that aren't packaged as gems:

http://docs.rubygems.org/read/chapter/4#page71

Devin

 

[Jim Weirich]

I'll take a look at this. I've just started to use Rake.

Let me know if you need any help with it.

I checked the load path in my testsuite, after executing "require 'watir'".
And i saw that the gem path to watir was on the load path before the
directory that had the development version. I don't use require_gem. But
you are telling me that that wasn't supposed to happen.

After i uninstalled the watir gem, i was able to run my tests correctly.

Then I am mystified. *Something* is causing the gem to be loaded. The fact
that the tests run correctly without the gem installed indicate the it is not
activated by need (since obviously you don't need it), nor by require_gem.

Does the watir zip file contain all the tests to reproduce this behavior?
Perhaps I'll take a look today.

 

[Austin Ziegler]

I release all my software as gems They are installed on my system as gem= s and
yet am able to test all my software without contortions. I use the Rake = test
task to run my tests (and it sets up the library paths on the command lin=

e).

I, too, release all my software as gems. I don't, however, use the
Rake test task, finding that it works poorly on Windows. However, all
of my Rake files *do* run my tests inline. Look at what I do with
Text::Format for an example. The PDF::Writer Rake task is up to date
for this, I just don't yet have any unit tests for it

You *will* need to make sure that, for local installations, you put
your libraries into the $LOAD_PATH, otherwise RubyGems will find them.

-austin

 

[Alexey Verkhovsky]

The current version of RubyGems library (0.8.10) doesn't alter the
LOADPATH. Instead it overrides Kernel#require with something like this:

module Kernel
alias require__ require

def require(path)
require__ path
rescue LoadError => load_error
... tries to require a gem ...
end
end

So, it will always try to get stuff from LOADPATH first, only then go to
RubyGems.

Earlier versions used to change LOADPATH, indeed. So, the answer to your
question _may_ be as simple as "upgrade RubyGems"

Alex

 

[Bret Pettichord]

Then I am mystified. *Something* is causing the gem to be loaded. The fact
that the tests run correctly without the gem installed indicate the it is not
activated by need (since obviously you don't need it), nor by require_gem.

I was using breakpoint to try and debug this situation, but i now think
that it may have had side effects itself that confused me more.

I did
require 'breakpoint'; breakpoint

In my script, and then from irb, checked $LOAD_PATH. That was when i saw
that the directory to the watir gem library preceeded the 'current
directory' that i had prepended to it.

I'm now wondering whether i in fact was getting the right watir library
through require because the load path didn't get changed until later. But
if this is true, then if i had required a gem library before the watir
require, i would not have loaded the one i wanted.

It seems that the gem mechanism changes the load path to speed up further
requires. But i am wondering if it modifies it more drastically than it
needs to. If i am require'ing 'breakpoint', the paths to other gem
libraries should not be showing up on the load path.

This is all somewhat speculative, because while i was trying to track this
down, i started getting a different version of breakpoint than i intended.
There apparently is a 'breakpoint' library in one of the rails gems and it
was being loaded instead and it was redefining the module class in a way
that made the modules in my code break. I ended up uninstalling a dozen
gems that i had installed last week because i knew i was running into
interaction problems and i needed to simplify my situation.

Bret

_____________________
Bret Pettichord
www.pettichord.com