Ruby Test Frameworks

Discussion in 'Ruby' started by Michael Furmaniuk, Dec 9, 2008.

  1. I'm looking for a Ruby Framework that I can use to aid in testing a
    filter driver, we currently use a Perl based set of scripts that has
    some wrapping and does system events for files, directories and
    registry changes (it's multi-platform for Windows and *nixes).

    Ideally, I want to start capturing data in a database and give a nice
    simple framework around it, which is what led me to Rails and from that
    to Ruby - and I thought being able to make our Harness more OO to allow
    for more reuse and remove duplication (which there is a lot of in our
    scripts). Plus one language to handle test results and the test harness
    seemed more efficient. Most of the Ruby Test Frameworks that I have
    looked through are all Web or GUI test frameworks, Test::Unit is nice
    but I would like to know if there is a Framework for application and
    system testing that I have missed. Has anyone used a framework for
    application and system testing they could recommend, or even an existing
    one that they have used successfully?

    Some of my initial requirements are:
    - Runs on Windows and Linux
    - Supports calling external code/scripts, we have Java and C code that
    is called for memory utilization and socket testing
    - Can generate system events for create/modify/delete of files and
    directories
    - Has some hooks into Windows Registry, IIS and Linux RPM (Nice to
    haves but not required)
    - Can integrate with an IDE (Visual Studio would be nice)

    Any help is appreciated.

    Thanks!
    --
    Posted via http://www.ruby-forum.com/.
     
    Michael Furmaniuk, Dec 9, 2008
    #1
    1. Advertising

  2. Michael Furmaniuk wrote:
    > Most of the Ruby Test Frameworks that I have
    > looked through are all Web or GUI test frameworks, Test::Unit is nice
    > but I would like to know if there is a Framework for application and
    > system testing that I have missed. Has anyone used a framework for
    > application and system testing they could recommend, or even an existing
    > one that they have used successfully?


    I don't know of anything specific to 'application and system testing'.
    If I were you, I'd start with Test::Unit or its bigger brothers Shoulda
    or Rspec, and see if it does what you actually need.

    > Some of my initial requirements are:
    > - Runs on Windows and Linux
    > - Supports calling external code/scripts, we have Java and C code that
    > is called for memory utilization and socket testing


    You can call external programs using backticks, system() or IO.popen:
    e.g.

    def test_foo
    result = `foo`
    assert_equal 0, $? # exit code
    assert_match /success/, result # output on stdout
    end

    You can call actual C using RubyInline or FFI.

    > - Can generate system events for create/modify/delete of files and
    > directories


    You can do those actions using FileUtils. Or do you mean you want to be
    *notified* of those sorts of events happening in the filesystem? I'm not
    sure about that.

    > - Has some hooks into Windows Registry, IIS and Linux RPM (Nice to
    > haves but not required)


    Not sure, I believe there is Win32 API (but I'm a Linux person).

    > - Can integrate with an IDE (Visual Studio would be nice)


    I think Test::Unit has some graphical runners (but I'm a CLI person :)
    --
    Posted via http://www.ruby-forum.com/.
     
    Brian Candler, Dec 9, 2008
    #2
    1. Advertising

  3. On Tue, Dec 9, 2008 at 3:42 PM, Brian Candler <> wrote:
    > Michael Furmaniuk wrote:
    >> Most of the Ruby Test Frameworks that I have
    >> looked through are all Web or GUI test frameworks, Test::Unit is nice
    >> but I would like to know if there is a Framework for application and
    >> system testing that I have missed. Has anyone used a framework for
    >> application and system testing they could recommend, or even an existing
    >> one that they have used successfully?

    >
    > I don't know of anything specific to 'application and system testing'.
    > If I were you, I'd start with Test::Unit or its bigger brothers Shoulda
    > or Rspec, and see if it does what you actually need.


    For the simple stuff, dust is my tool of choice. Its a very simple
    wrapper around
    Test::Unit, so there is a shallow learning curve, but you get
    meaningful test names,
    and it is very easy to generate large blocks of tests from arrays of
    inputs. Handy
    for black box testing, testing regexes etc.

    Shoulda and RSpec I think are tackling different needs. The learning
    curve is greater
    and there is more magic involved, but I suspect that test
    organisation, and consolidation
    of reporting is probably a greater priority than natural language test
    description.

    >> Some of my initial requirements are:
    >> - Runs on Windows and Linux
    >> - Supports calling external code/scripts, we have Java and C code that
    >> is called for memory utilization and socket testing

    >
    > You can call external programs using backticks, system() or IO.popen:
    > e.g.
    >
    > def test_foo
    > result = `foo`
    > assert_equal 0, $? # exit code
    > assert_match /success/, result # output on stdout
    > end
    >
    > You can call actual C using RubyInline or FFI.


    In addition, check out JtestR for plain old Java testing (it works great).

    >
    >> - Can generate system events for create/modify/delete of files and
    >> directories

    >
    > You can do those actions using FileUtils. Or do you mean you want to be
    > *notified* of those sorts of events happening in the filesystem? I'm not
    > sure about that.
    >
    >> - Has some hooks into Windows Registry, IIS and Linux RPM (Nice to
    >> haves but not required)

    >
    > Not sure, I believe there is Win32 API (but I'm a Linux person).
    >
    >> - Can integrate with an IDE (Visual Studio would be nice)


    Theres a team integrating Ruby into visual studio, I can't recall offhand.

    >
    > I think Test::Unit has some graphical runners (but I'm a CLI person :)


    I haven't found a good solution for out of the box test reporting in Ruby.
    RSpec has some great reporting, but I need something a bit more turnkey.

    I currently use dust to test java code through JtestR on top of an existing
    Java build system (so our test reporting is done through JUnit reports and
    cobertura works for our code coverage of the java code).
     
    Richard Conroy, Dec 9, 2008
    #3
  4. Richard Conroy wrote:
    > Shoulda and RSpec I think are tackling different needs. The learning
    > curve is greater
    > and there is more magic involved, but I suspect that test
    > organisation, and consolidation
    > of reporting is probably a greater priority than natural language test
    > description.


    For Rspec I agree - I found it too contrived - but shoulda fitted my
    needs much better. It gives you the meaningful test names, and also
    setup/teardown contexts which avoids having to make a new class for
    every set of fixtures. But you keep good old-fashioned assertions at the
    bottom.

    class Foo < Test::Unit::TestCase
    context "Piping in FOO" do
    setup do
    File.delete("output.txt")
    @res = `echo "FOO" | ./wibble`
    end

    should "generate output BAR" do
    assert_match /BAR/, @res
    end

    should "create file output.txt" do
    assert File.exist?("output.txt")
    end

    # etc
    end
    end
    --
    Posted via http://www.ruby-forum.com/.
     
    Brian Candler, Dec 9, 2008
    #4
  5. Richard Conroy wrote:
    > On Tue, Dec 9, 2008 at 3:42 PM, Brian Candler <>
    > wrote:
    > Shoulda and RSpec I think are tackling different needs. The learning
    > curve is greater
    > and there is more magic involved, but I suspect that test
    > organisation, and consolidation
    > of reporting is probably a greater priority than natural language test
    > description.


    Yes, natural language test description is nice, but if it comes at the
    expense of reporting and organization I can deal with the odd
    description. I had looked a little at RSpec but not in depth, and
    Shoulda I saw referenced in a page somewhere but it did not seem a fit
    for what I was looking for, I will check them more in depth.

    >> You can call actual C using RubyInline or FFI.

    >
    > In addition, check out JtestR for plain old Java testing (it works
    > great).


    Thanks for these, and the other ideas, I will check them out!
    --
    Posted via http://www.ruby-forum.com/.
     
    Michael Furmaniuk, Dec 9, 2008
    #5
  6. Michael Furmaniuk

    Yanik Magnan Guest

    SSdtIGd1ZXNzaW5nIHByb2JhYmx5IHRoZSBJcm9uUnVieSBndXlzIGFyZSB0aGUgb25lcyBnZXR0
    aW5nIFZpc3VhbApTdHVkaW8gd29ya2luZyB3aXRoIFJ1YnkuIDpQCgotLSBZYW5payBNYWduYW4g
    4oCiIGh0dHA6Ly9yLWNoLm5ldAoKCgpPbiBUdWUsIERlYyA5LCAyMDA4IGF0IDM6NTEgUE0sIFJp
    Y2hhcmQgQ29ucm95IDxyaWNoYXJkLmNvbnJveUBnbWFpbC5jb20+IHdyb3RlOgo+IFRoZXJlcyBh
    IHRlYW0gaW50ZWdyYXRpbmcgUnVieSBpbnRvIHZpc3VhbCBzdHVkaW8sIEkgY2FuJ3QgcmVjYWxs
    IG9mZmhhbmQuCg==
     
    Yanik Magnan, Dec 10, 2008
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. kanchy kang
    Replies:
    2
    Views:
    388
    Magnus Lycka
    Feb 24, 2006
  2. kanchy kang
    Replies:
    1
    Views:
    359
    Scott David Daniels
    Feb 24, 2006
  3. Skybuck Flying

    Call oddities: &Test() vs &Test vs Test

    Skybuck Flying, Oct 4, 2009, in forum: C Programming
    Replies:
    1
    Views:
    726
    Skybuck Flying
    Oct 4, 2009
  4. Rustom Mody

    python test frameworks

    Rustom Mody, Nov 7, 2010, in forum: Python
    Replies:
    0
    Views:
    196
    Rustom Mody
    Nov 7, 2010
  5. Mark Lawrence
    Replies:
    0
    Views:
    158
    Mark Lawrence
    Aug 27, 2012
Loading...

Share This Page