[RDOC] Using unit-tests as examples for a documentation

Discussion in 'Ruby' started by benny, Nov 25, 2004.

  1. benny

    benny Guest

    dear list,

    suddenly this idea jumped on my brain:

    when using unit tests we got this pretty methods like
    assert_equal()
    and
    assert_match()

    in which we are telling what we expect to get back from a method.
    For this methods we invent more or less realistical examples for
    how we could use the tested method.
    We only need to select realistic examples to let them be meaningful and if
    they are, they could in fact serve as examples for a documentation as well.

    In a similar way we use the "don'ts" - the prohibited usage of our method
    as flesh for the
    assert_raise()

    tests (although I am not sure if negative examples are that useful for a
    documentation).

    As we all know: in the documentation nothing beats a good example!

    Now the idea: Why not use the examples in the unit-tests (at least the
    "positive ones") as link to a popup window "examples" in the
    HTML-documentation created by rdoc?

    For this to work properly we might need a convention for the locations of
    the unit-tests (e.g. as given in Progr. RubyV.2, page 159*) and an optional
    switch for rdoc to tell it to search for unit-tests and make the examples
    for the corresponding methods.

    what do you think about it? might that be possible or would there be so much
    divergence in the modelling of unit tests that rdoc won't have a chance to
    tie that all together?

    I see a big advantage coming out of this approach.
    Consider you were using unit-tests anyway on a regular basis. then you would
    run them at least anytime you make changes to your old code. so you will
    have to fix the unit-tests as well if you do some changes that affect the
    usage of your method. once changed the automatically build examples would
    be up to date if you are running rdoc the next time. (and even an outdated,
    "forgotten" documentation might stay useful when there are working examples
    coming with it).

    Surely that would affect the way we organize unit-test name parameter
    therein etc. but that should not be that bad.


    regards, benny

    * the example given is:

    roman
    |-----lib/
    | |---- roman.rb
    | |----- other files....
    |----test/
    | |-----test_roman.rb
    | |----- other tests......
     
    benny, Nov 25, 2004
    #1
    1. Advertising

  2. benny

    Dave Thomas Guest

    On Nov 25, 2004, at 15:32, benny wrote:

    > Now the idea: Why not use the examples in the unit-tests (at least the
    > "positive ones") as link to a popup window "examples" in the
    > HTML-documentation created by rdoc?


    This has been raised before. The problem is that it's hard to work out
    which tests test which methods: method naming conventions would help,
    but even so you quite often have three test methods testing one
    production method.

    An alternative might be to embed the UTs in the comment for the method,
    and then have TestUnit extract and run them automatically. I personally
    quite like that approach, although it would need some special markup
    that RDoc doesn't yet support.


    Cheers

    Dave
     
    Dave Thomas, Nov 25, 2004
    #2
    1. Advertising

  3. benny

    benny Guest

    Dave Thomas wrote:

    >
    > On Nov 25, 2004, at 15:32, benny wrote:
    >
    >> Now the idea: Why not use the examples in the unit-tests (at least the
    >> "positive ones") as link to a popup window "examples" in the
    >> HTML-documentation created by rdoc?

    >
    > This has been raised before.

    sorry, I didn't notice that. Don't want to bore the list.

    > The problem is that it's hard to work out
    > which tests test which methods: method naming conventions would help,
    > but even so you quite often have three test methods testing one
    > production method.

    but that could be made by a really simple convention:
    the name of each test method for method "foo" should be

    test_foo_individual_name

    Anyway this would be an *option*, i.e. test-methods not following that
    convention would silently slip away (which might also be a useful feature).

    >
    > An alternative might be to embed the UTs in the comment for the method,
    > and then have TestUnit extract and run them automatically. I personally
    > quite like that approach, although it would need some special markup
    > that RDoc doesn't yet support.


    I don't think it is good to embed the testing methods into the
    documentation. Think of packaging your sources: it might be useful to
    separate the tests from the code for smaller packages. the user when
    generating the documentation would then not have the examples in the docs
    but the developer might offer them on the online docs.

    also think of a replacement for a set of given classes methods. you might
    want to use the same tests on different development branches to ensure the
    compatibility. for using rdoc in this case you start it in the
    corresponding branch and make a simple symlink named "test" to the
    unit-test directories.

    regards,
    benny


    >
    >
    > Cheers
    >
    > Dave
     
    benny, Nov 25, 2004
    #3
  4. Dave Thomas wrote:

    > On Nov 25, 2004, at 15:32, benny wrote:
    >> Now the idea: Why not use the examples in the unit-tests (at least the
    >> "positive ones") as link to a popup window "examples" in the
    >> HTML-documentation created by rdoc?

    >
    > This has been raised before. The problem is that it's hard to work out
    > which tests test which methods: method naming conventions would help,
    > but even so you quite often have three test methods testing one
    > production method.
    >
    > An alternative might be to embed the UTs in the comment for the method,
    > and then have TestUnit extract and run them automatically. I personally
    > quite like that approach, although it would need some special markup
    > that RDoc doesn't yet support.


    I'm currently using indentation as a sample code marker in
    test-extract.rb and would welcome official tags for this. (Maybe
    prefixing each line of code with optional whitespace and "|"?)
    Integrating support for this into RDoc might also have other benefits
    for output formatting and so on.

    Here's a sample of what I am currently doing:

    > # Causes a pattern which would otherwise match
    > # greedily to match non-greedily.
    > #
    > # This can only be applied to #count and #optional
    > # based constructs.
    > #
    > # re_greedy = Regexp::English.new do
    > # literal("<") + something + literal(">")
    > # end
    > # re_non_greedy = Regexp::English.new do
    > # literal("<") + something.minimal + literal(">")
    > # end
    > # str = "<html><body>"
    > # re_greedy.match(str)[0] # => "<html><body>"
    > # re_non_greedy.match(str)[0] # => "<html>"
    > #
    > # This method will do nothing when applied to a Node
    > # which is already minimal:
    > #
    > # re_minimal = Regexp::English.something.minimal
    > # re_minimal.minimal == re_minimal # => true
    > #
    > # However, it will raise a NameError Exception when
    > # you try to call it on a Regexp Node that can't be
    > # made non-greedy:
    > #
    > # re = Regexp::English.literal("foo")
    > # re.minimal # raises NameError
    > def minimal
    > <implementation snipped>
    > end


    The next version of test/unit will also feature a new syntax for this.
    That one will use sample { assert_equal(code, result) } style test
    cases directly before the method IIRC.
     
    Florian Gross, Nov 25, 2004
    #4
  5. benny

    Dave Thomas Guest

    On Nov 25, 2004, at 16:52, Florian Gross wrote:
    > I'm currently using indentation as a sample code marker in
    > test-extract.rb and would welcome official tags for this. (Maybe
    > prefixing each line of code with optional whitespace and "|"?)
    > Integrating support for this into RDoc might also have other benefits
    > for output formatting and so on.


    I'm not sure what you're asking for here--could you explain at bit
    more? (RDoc will already set this indented code as code, won't it?)


    Cheers

    Dave
     
    Dave Thomas, Nov 26, 2004
    #5
  6. Dave Thomas wrote:

    > I'm not sure what you're asking for here--could you explain at bit more?
    > (RDoc will already set this indented code as code, won't it?)


    I'm thinking along the lines of a "sample code" icon or something
    similar that can be clicked to display or hide the embedded sample code
    and possibly also setup code that needs to be run for actually making
    the sample code work for the HTML output.

    Maybe something like in this simplified example:

    class KeyedList
    ...

    # Removes all items from a keyed list.
    #
    # % aList = KeyedList["foo", "bar", "qux"]
    # | aList.clear
    # | aList.size # => 0
    def clear
    size.times { pop }
    end
    end

    Where "%" would by default be hidden and could be uncovered by clicking
    the icon. I'm not sure about the symbols so feel free to change them to
    something that makes more sense for you.

    I think the biggest point would however be formatting samples
    consistently so that they can be distinguished easily from other things
    that make sense in a whitespace preserved area.
     
    Florian Gross, Nov 26, 2004
    #6
  7. benny

    benny Guest

    Florian Gross wrote:

    >
    >
    > Maybe something like in this simplified example:
    >
    > class KeyedList
    > ...
    >
    > # Removes all items from a keyed list.
    > #
    > # % aList = KeyedList["foo", "bar", "qux"]
    > # | aList.clear
    > # | aList.size # => 0
    > def clear
    > size.times { pop }
    > end
    > end

    could you please explain how that syntax would fit into Test::Unit?

    benny
     
    benny, Nov 26, 2004
    #7
  8. benny wrote:

    > Florian Gross wrote:
    >># Removes all items from a keyed list.
    >>#
    >># % aList = KeyedList["foo", "bar", "qux"]
    >># | aList.clear
    >># | aList.size # => 0

    > could you please explain how that syntax would fit into Test::Unit?


    I have written a tool that scans such samples out of source code and
    turns them into test cases.
     
    Florian Gross, Nov 27, 2004
    #8
  9. benny

    Dave Thomas Guest

    On Nov 25, 2004, at 15:32, benny wrote:

    > Now the idea: Why not use the examples in the unit-tests (at least the
    > "positive ones") as link to a popup window "examples" in the
    > HTML-documentation created by rdoc?


    I've been thinking a bit about this. I'm nervous about imposing my own
    rules on how tests are organized and how methods are named: I'd like
    RDoc to be as transparent as possible.

    So here's another take on it. You can already link from the method to
    its test using standard RDoc hyperlinks. How about adding a slight bit
    of new linking smarts: if you hyperlink to a class/method where the
    class name starts Test and the method name starts test_, RDoc will
    hyperlink to it in a different window. That way you can read the
    mainline code, and scroll to the appropriate test in the test window by
    clicking. You comment block might contain:

    # burble burble burble. Burble burble burble. Burble burble burble
    # burble burble. See the unit test in TestThingie#test_create

    def create
    ...
    end

    It'll use a named window, so all test code will share just one
    additional window.


    Cheers

    Dave
     
    Dave Thomas, Nov 27, 2004
    #9
  10. Dave Thomas wrote:

    > On Nov 25, 2004, at 15:32, benny wrote:
    >> Now the idea: Why not use the examples in the unit-tests (at least the
    >> "positive ones") as link to a popup window "examples" in the
    >> HTML-documentation created by rdoc?

    >
    > I've been thinking a bit about this. I'm nervous about imposing my own
    > rules on how tests are organized and how methods are named: I'd like
    > RDoc to be as transparent as possible.


    Well, it would just be a style suggestion. There's no reason to enforce
    tests being inlined into the documentation of methods, but there are use
    cases where it makes good sense to do so. (Specification by example
    comes to mind.) I'm not sure I understand the point about method names.

    > So here's another take on it. You can already link from the method to
    > its test using standard RDoc hyperlinks. How about adding a slight bit
    > of new linking smarts: if you hyperlink to a class/method where the
    > class name starts Test and the method name starts test_, RDoc will
    > hyperlink to it in a different window. That way you can read the
    > mainline code, and scroll to the appropriate test in the test window by
    > clicking. You comment block might contain:
    >
    > # burble burble burble. Burble burble burble. Burble burble burble
    > # burble burble. See the unit test in TestThingie#test_create


    I think this is too indirect. I'd really like to see the sample code
    inlined in the documentation, but formatted differently so it stands
    out. Being able to toggle the visibility of sample setup code would also
    be a nice thing, but it isn't as important to me. Of course this is just
    my personal opinion and other's are likely to have other thoughts about
    this.
     
    Florian Gross, Nov 27, 2004
    #10
  11. benny

    Dave Thomas Guest

    On Nov 27, 2004, at 10:47, Florian Gross wrote:

    > I think this is too indirect. I'd really like to see the sample code
    > inlined in the documentation, but formatted differently so it stands
    > out. Being able to toggle the visibility of sample setup code would
    > also be a nice thing, but it isn't as important to me. Of course this
    > is just my personal opinion and other's are likely to have other
    > thoughts about this.


    Are you talking about sample code or unit tests here. For sample code,
    I don't see why you'd want to hide it--wouldn't it be part of the
    normal comment of the method?


    Cheers

    Dave
     
    Dave Thomas, Nov 27, 2004
    #11
  12. Dave Thomas wrote:

    > On Nov 27, 2004, at 10:47, Florian Gross wrote:
    >
    >> I think this is too indirect. I'd really like to see the sample code
    >> inlined in the documentation, but formatted differently so it stands
    >> out. Being able to toggle the visibility of sample setup code would
    >> also be a nice thing, but it isn't as important to me. Of course this
    >> is just my personal opinion and other's are likely to have other
    >> thoughts about this.

    >
    > Are you talking about sample code or unit tests here. For sample code, I
    > don't see why you'd want to hide it--wouldn't it be part of the normal
    > comment of the method?


    I'm talking about cases where both are the same thing. (The overlap is
    really quite big. It works out well and lets you automatically keep the
    documentation and the code in synchronization. It offers quite some
    benefits to treat test and sample code as the same thing though it does
    not make sense for all test code to be used for samples.) However test
    code can sometimes need complex set-up code for which it could make
    sense to be hidden by default. I think a Ruby tutorial / book used the
    technique of providing samples and hiding the parts that were explained
    only in later chapters.
     
    Florian Gross, Nov 27, 2004
    #12
  13. benny

    benny Guest

    Dave Thomas wrote:

    >
    > On Nov 25, 2004, at 15:32, benny wrote:
    >
    >> Now the idea: Why not use the examples in the unit-tests (at least the
    >> "positive ones") as link to a popup window "examples" in the
    >> HTML-documentation created by rdoc?

    >
    > I've been thinking a bit about this. I'm nervous about imposing my own
    > rules on how tests are organized and how methods are named: I'd like
    > RDoc to be as transparent as possible.
    >
    > So here's another take on it. You can already link from the method to
    > its test using standard RDoc hyperlinks. How about adding a slight bit
    > of new linking smarts: if you hyperlink to a class/method where the
    > class name starts Test and the method name starts test_, RDoc will
    > hyperlink to it in a different window. That way you can read the
    > mainline code, and scroll to the appropriate test in the test window by
    > clicking. You comment block might contain:
    >
    > # burble burble burble. Burble burble burble. Burble burble burble
    > # burble burble. See the unit test in TestThingie#test_create
    >
    > def create
    > ...
    > end
    >
    > It'll use a named window, so all test code will share just one
    > additional window.
    >
    >
    > Cheers
    >
    > Dave

    (hoping your still watching this thread)

    I find it a good idea. Maybe instead of worring about the best solution we
    should allow both: unit-test in the documentation and documentation in the
    unit-tests. your proposal has the advantage to be able to put some extra
    documentation about the given examples in this popup-window (and separate
    this from the main documentation, too).

    following your proposal the question arises how to mark the parts of the
    testing code that should be shown in the window / how to hide the
    preparation.(when I got you right, you thought, that we build the
    documentation also in the test-directory and link to it , but that would
    hide the concrete code, wouldn't it ? and writing the same code 2 times -
    for the comment block and the code itself is annoying)


    cheers, benny
     
    benny, Dec 12, 2004
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. NW Technicals

    How can I run the Unit Tests using nUnit

    NW Technicals, Mar 21, 2007, in forum: ASP .Net
    Replies:
    1
    Views:
    313
    NW Technicals
    Mar 21, 2007
  2. Andreas Schwarz
    Replies:
    6
    Views:
    253
    Randy W. Sims
    Jan 1, 2004
  3. Bil Kleb
    Replies:
    0
    Views:
    102
    Bil Kleb
    Sep 9, 2005
  4. Brian Schröder

    Extract Tests from rdoc Examples

    Brian Schröder, Oct 19, 2005, in forum: Ruby
    Replies:
    0
    Views:
    116
    Brian Schröder
    Oct 19, 2005
  5. dayo
    Replies:
    11
    Views:
    346
    Ilya Zakharevich
    Dec 16, 2005
Loading...

Share This Page