Embedded vs. Non-embedded Tests

Discussion in 'Ruby' started by Trans, Sep 4, 2007.

  1. Trans

    Trans Guest

    As far as I know, Facets is the only large project that uses embedded
    unit tests. The tests are housed in block comments, eg.

    =begin test
    ...
    =end

    I have a command line tool that can run the embedded test directly.
    For deployment, I have another tool that extracts all the embedded
    tests and copies them to stand-alone test/ files so others can run
    them. It works fairly well. And the advantage of course is that the
    unit tests are right there with the libs they test.

    However, considering that just about everyone else is putting units
    test in their on test files, I'm wondering if I actually missing out
    on superior advantages to that approach. If not why haven't others
    pursued better tools for supporting embedded tests?

    Thanks,
    T.
    Trans, Sep 4, 2007
    #1
    1. Advertising

  2. On 9/4/07, Trans <> wrote:
    > As far as I know, Facets is the only large project that uses embedded
    > unit tests. The tests are housed in block comments, eg.
    >
    > =begin test
    > ...
    > =end
    >
    > I have a command line tool that can run the embedded test directly.
    > For deployment, I have another tool that extracts all the embedded
    > tests and copies them to stand-alone test/ files so others can run
    > them. It works fairly well. And the advantage of course is that the
    > unit tests are right there with the libs they test.
    >
    > However, considering that just about everyone else is putting units
    > test in their on test files, I'm wondering if I actually missing out
    > on superior advantages to that approach. If not why haven't others
    > pursued better tools for supporting embedded tests?


    I think that you're swimming against the tide.

    Personally, it just feels right to me to separate the test code from
    the code under test, most test equipment belongs in the garage instead
    of the trunk of the car.

    And when it gets beyond just unit testing individual classes, it's not
    apparent to me which file an embedded test should call home.

    From a practical tool perspective, there are lots of things which
    expect a relationship between code files and test files. Two which
    spring immediately to mind are autotest and rcov.

    --
    Rick DeNatale

    My blog on Ruby
    http://talklikeaduck.denhaven2.com/
    Rick DeNatale, Sep 4, 2007
    #2
    1. Advertising

  3. On 9/4/07, Peter Cooper <> wrote:
    > On 9/4/07, Rick DeNatale <> wrote:
    > >
    > > Personally, it just feels right to me to separate the test code from
    > > the code under test, most test equipment belongs in the garage instead
    > > of the trunk of the car.

    >
    >
    > A lot of diagnostics now takes place in modern cars, however. Tire pressure
    > checks, engine wear, oil, etc. Sure, you'd only work and make changes to the
    > car in the garage, but a lot of the "testing" is a hidden ongoing day to day
    > process.


    For the most part, that's instrumentation, not testing.

    In the case where sensor information is used to affect the operation
    of the vehicle, such as
    engine managment, abs, or traction control, it's no more testing
    equipment than carburettors, brakes, and master cylinders. Yes, data
    gets logged, but that's not testing, any more than a server log file
    is.

    > And if you get onto, say, Formula 1 cars.. some "repairs" can even take
    > place on the road and the telemetry provides almost as much info to the
    > engineers while the car is running out on the track as they can get in the
    > garage. I've always wondered if/how to make software as easy to diagnose
    > with real-time telemetry and the ability to make changes to a running
    > program's parameters..


    Yes, but most the testing equipment is not in the car, the engineers
    in the pits have laptops, and the heavy stuff is in places like Woking
    or Maranello, connected through the 'internets.'

    --
    Rick DeNatale

    My blog on Ruby
    http://talklikeaduck.denhaven2.com/
    Rick DeNatale, Sep 4, 2007
    #3
  4. Trans

    Trans Guest

    On Sep 4, 12:07 pm, "Rick DeNatale" <> wrote:
    > On 9/4/07, Trans <> wrote:
    >
    >
    >
    > > As far as I know, Facets is the only large project that uses embedded
    > > unit tests. The tests are housed in block comments, eg.

    >
    > > =begin test
    > > ...
    > > =end

    >
    > > I have a command line tool that can run the embedded test directly.
    > > For deployment, I have another tool that extracts all the embedded
    > > tests and copies them to stand-alone test/ files so others can run
    > > them. It works fairly well. And the advantage of course is that the
    > > unit tests are right there with the libs they test.

    >
    > > However, considering that just about everyone else is putting units
    > > test in their on test files, I'm wondering if I actually missing out
    > > on superior advantages to that approach. If not why haven't others
    > > pursued better tools for supporting embedded tests?

    >
    > I think that you're swimming against the tide.
    >
    > Personally, it just feels right to me to separate the test code from
    > the code under test, most test equipment belongs in the garage instead
    > of the trunk of the car.


    Interesting analogy. We do put some gages right on the dash. But
    that's more like embedding the tests directly into code --eg. rescue
    and try clauses. So maybe you're analogy is a good one. I can see some
    reasons for doing so, it certainly make it easier to send some one a
    test, that they can try against their own implementation, for
    instance. But then I start to wonder about API documentation too.
    Should that also be separated out into dedicated files? Why tests but
    not docs?

    > And when it gets beyond just unit testing individual classes, it's not
    > apparent to me which file an embedded test should call home.


    That's true. Embedded tests only apply for unit test. Integration
    tests still need to be is separate files anyway.

    > From a practical tool perspective, there are lots of things which
    > expect a relationship between code files and test files. Two which
    > spring immediately to mind are autotest and rcov.


    That's part of what I was wondering. I've never tried these tools. I
    guess I've always suspected they'd be too magical to introduce into an
    already established project, and thus lead to more trouble than
    they're worth --though I'm sure they're quite useful if one starts
    using them from the get-go. Am I wrong about that? But even so,
    couldn't such tools be adapted to handle embedded tests too?

    T.
    Trans, Sep 4, 2007
    #4
  5. On 9/4/07, Trans <> wrote:
    >
    >
    > On Sep 4, 12:07 pm, "Rick DeNatale" <> wrote:


    > > Personally, it just feels right to me to separate the test code from
    > > the code under test, most test equipment belongs in the garage instead
    > > of the trunk of the car.

    >
    > Interesting analogy. We do put some gages right on the dash. But
    > that's more like embedding the tests directly into code --eg. rescue
    > and try clauses. So maybe you're analogy is a good one. I can see some
    > reasons for doing so, it certainly make it easier to send some one a
    > test, that they can try against their own implementation, for
    > instance. But then I start to wonder about API documentation too.
    > Should that also be separated out into dedicated files? Why tests but
    > not docs?


    Well , it probably seems more obvious to most to put the doc with the
    code (assuming that there is any doc <G>). Note that we've just now
    seemed to get tools like dcov.

    It's long been pretty standard process to have tools which extract doc
    from code. On the other hand there are some arguments for keeping
    even these two separate. In "The Psychology of Computer Programming"
    Gerry Weinberg talked about the trap of "debugging" the comments
    instead of the code, and advocated tools which would not just extract
    code commentary but REMOVE it.

    On the other hand without having the doc with the code and tools like
    rdoc we'd probably have even less documentation than we do now, just a
    guess.

    --
    Rick DeNatale

    My blog on Ruby
    http://talklikeaduck.denhaven2.com/
    Rick DeNatale, Sep 4, 2007
    #5
  6. Peter Cooper wrote:
    > And if you get onto, say, Formula 1 cars.. some "repairs" can even take
    > place on the road and the telemetry provides almost as much info to the
    > engineers while the car is running out on the track as they can get in
    > the
    > garage. I've always wondered if/how to make software as easy to diagnose
    > with real-time telemetry and the ability to make changes to a running
    > program's parameters..


    Take a look at Eiffels Design By Contract. I'm a huge fan of it. Allows
    you to move most testing code into the contract, the unit tests are then
    merely running your code with contracts enabled.
    For an idea on how that works, take a look at the movies with "Design by
    Contract" in the title here: http://eiffel.com/developers/presentations/
    Would be a very nice feature if ruby had that, even though I'd introduce
    additional syntax and I've no idea how to integrate it nicely.

    Regards
    Stefan
    --
    Posted via http://www.ruby-forum.com/.
    Stefan Rusterholz, Sep 5, 2007
    #6
  7. Trans

    Alex Young Guest

    Stefan Rusterholz wrote:
    > Peter Cooper wrote:
    >> And if you get onto, say, Formula 1 cars.. some "repairs" can even take
    >> place on the road and the telemetry provides almost as much info to the
    >> engineers while the car is running out on the track as they can get in
    >> the
    >> garage. I've always wondered if/how to make software as easy to diagnose
    >> with real-time telemetry and the ability to make changes to a running
    >> program's parameters..

    >
    > Take a look at Eiffels Design By Contract. I'm a huge fan of it. Allows
    > you to move most testing code into the contract, the unit tests are then
    > merely running your code with contracts enabled.
    > For an idea on how that works, take a look at the movies with "Design by
    > Contract" in the title here: http://eiffel.com/developers/presentations/
    > Would be a very nice feature if ruby had that, even though I'd introduce
    > additional syntax and I've no idea how to integrate it nicely.

    Allow me to shamelessly copy and paste from an old post of mine:

    $ cat testable.rb
    module Test; module Unit; class TestCase; end; end; end

    module Testable
    def test_case(&block)
    Class.new(Test::Unit::TestCase).module_eval &block
    end
    end

    $ cat builtin_tests.rb
    require 'testable'

    class MyWorkingClass
    extend Testable

    def foo(a,b)
    return a+b
    end

    test_case {
    def test_foo_null
    assert_equal 2, MyWorkingClass.new.foo(1,1)
    end
    }
    end

    if __FILE__ == $0
    puts MyWorkingClass.new.foo(1,1)
    end

    $ ruby builtin_tests.rb
    2
    $ testrb builtin_tests.rb
    Loaded suite builtin_tests.rb
    Started
    Alex Young, Sep 5, 2007
    #7
  8. Alex Young wrote:
    > Allow me to shamelessly copy and paste from an old post of mine:
    >
    > $ cat testable.rb
    > module Test; module Unit; class TestCase; end; end; end
    >
    > module Testable
    > def test_case(&block)
    > Class.new(Test::Unit::TestCase).module_eval &block
    > end
    > end
    >
    > $ cat builtin_tests.rb
    > require 'testable'
    >
    > class MyWorkingClass
    > extend Testable
    >
    > def foo(a,b)
    > return a+b
    > end
    >
    > test_case {
    > def test_foo_null
    > assert_equal 2, MyWorkingClass.new.foo(1,1)
    > end
    > }
    > end
    >
    > if __FILE__ == $0
    > puts MyWorkingClass.new.foo(1,1)
    > end
    >
    > $ ruby builtin_tests.rb
    > 2
    > $ testrb builtin_tests.rb
    > Loaded suite builtin_tests.rb
    > Started
    > .
    > Finished in 0.00095 seconds.
    >
    > 1 tests, 1 assertions, 0 failures, 0 errors
    > $
    >
    >
    > Something like that?


    No, the whole approach is different. I'd say the movies I linked to can
    explain it far better than I can.
    With DBC, if contracts are enabled (in Eiffel you can disable contracts
    to get better performance), your method is tested everytime you call the
    method. It tests pre-conditions and/or post-conditions (providing you
    the old state for your post-conditions). Also after every call to a
    method the class invariant is tested to validate your object. That's the
    basic idea.

    One way that could probably be implemented (though, only as of ruby 1.9
    due to &block params) might look this:

    class DayOfTime
    extend DBC
    invariant {
    @hour.between?(0,23)
    @min.between?(0,59)
    @sec.between?(0,59)
    }

    define:)min=) {
    pre_condition { |value|
    value.kind_of?(Numeric)
    value.between?(0,59)
    }
    body { |value|
    @value = value
    }
    post_condition { |old, value|
    @hour == old.hour
    @min == value
    @sec == old.second
    }
    }
    end

    Regards
    Stefan
    --
    Posted via http://www.ruby-forum.com/.
    Stefan Rusterholz, Sep 5, 2007
    #8
  9. Trans

    Trans Guest

    On Sep 5, 2:13 am, Stefan Rusterholz <> wrote:
    > Peter Cooper wrote:
    > > And if you get onto, say, Formula 1 cars.. some "repairs" can even take
    > > place on the road and the telemetry provides almost as much info to the
    > > engineers while the car is running out on the track as they can get in
    > > the
    > > garage. I've always wondered if/how to make software as easy to diagnose
    > > with real-time telemetry and the ability to make changes to a running
    > > program's parameters..

    >
    > Take a look at Eiffels Design By Contract. I'm a huge fan of it. Allows
    > you to move most testing code into the contract, the unit tests are then
    > merely running your code with contracts enabled.
    > For an idea on how that works, take a look at the movies with "Design by
    > Contract" in the title here:http://eiffel.com/developers/presentations/
    > Would be a very nice feature if ruby had that, even though I'd introduce
    > additional syntax and I've no idea how to integrate it nicely.


    http://jroller.com/dscataglini/entry/jim_weirich_test_drivern_development

    T.
    Trans, Sep 5, 2007
    #9
  10. Trans

    micathom Guest

    > Would be a very nice feature if ruby had that, even though I'd introduce
    > additional syntax and I've no idea how to integrate it nicely.


    There are a few libraries around that try to do this (google for "ruby
    dbc" or similar). Most of these efforts are not entirely convincing.
    Unfortunately, the ruby community seems somewhat dogmatic in arguing
    against dbc. Maybe because it's not enough of a duck.
    micathom, Sep 5, 2007
    #10
  11. micathom wrote:
    >> Would be a very nice feature if ruby had that, even though I'd introduce
    >> additional syntax and I've no idea how to integrate it nicely.

    >
    > There are a few libraries around that try to do this (google for "ruby
    > dbc" or similar). Most of these efforts are not entirely convincing.
    > Unfortunately, the ruby community seems somewhat dogmatic in arguing
    > against dbc. Maybe because it's not enough of a duck.


    I know, but I found none of them convincing enough. Might change with
    ruby2, as there are more possibilities then.
    IMHO DBC is orthogonal to duck typing. It changes just what you test
    for. Instead having a contract that states "argument must be a duck" you
    just will have "argument must quack like a duck". I fail to see how that
    goes against duck typing.

    Regards
    Stefan
    --
    Posted via http://www.ruby-forum.com/.
    Stefan Rusterholz, Sep 5, 2007
    #11
  12. Trans

    micathom Guest

    > I fail to see how that goes against duck typing.

    Me too.
    micathom, Sep 5, 2007
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    8
    Views:
    885
  2. Colin Paul Gloster
    Replies:
    48
    Views:
    1,869
    Colin Paul Gloster
    Apr 10, 2007
  3. Thomas Dodds

    Databind an embedded control in an embedded datagrid

    Thomas Dodds, Jul 26, 2004, in forum: ASP .Net Datagrid Control
    Replies:
    0
    Views:
    394
    Thomas Dodds
    Jul 26, 2004
  4. dayo
    Replies:
    11
    Views:
    336
    Ilya Zakharevich
    Dec 16, 2005
  5. Num GG
    Replies:
    2
    Views:
    348
    Num GG
    Nov 17, 2008
Loading...

Share This Page