TDD/unit testing question

Discussion in 'Ruby' started by Peter Szinek, Dec 5, 2006.

  1. Peter Szinek

    Peter Szinek Guest

    Hello,

    Slightly OT, but since I think Unit testing/TDD is a commonly accepted
    methodology here, I'll give it a shot:

    I am writing a web extraction framework in Ruby which is now relatively
    big. I am doing kind of a semi-TDD (i.e. some things are done pure TDD,
    and I am writing tests later for stuff which are not) - so at the end
    everything should be covered by tests.

    I have also tons of black-box tests (i.e. for the specified input and
    output, yell if the output of the current run is different) and as the
    code grows, also a great load of unit tests.

    My problem is that since the code is being changed a lot (sometimes
    quite brutal refactoring, even dropping some classes, joining others,
    throwing out inheritance hierarchy to replace it with composition or
    vice versa) my development time is 20-30% (maybe more) of updating all
    the tests (which is kinda annoying) and the remaining to actually write
    the code.

    Anyway, I have trapped so much errors when running all the tests between
    refactorings that the overall development time is much more faster than
    if I had to bug-hunt for all the obscure weirdness that popped up and
    catched easily by the tests. I guess this will be even more true as the
    code will grow.

    So my question is not if I have to follow these practices or not but rather

    1) Am I doing something wrong? (i.e. that I spend so much time with
    rewriting/modifying/updating the tests? or is this normal)
    2) Are there some tools/methods/ideas to speed things up? Or this is
    just good so?
    3) Maybe I am just too new to these techniques and a skilled TDDist
    marches much faster?

    I have seen people (now I am concretely talking about Java coders, but
    language does not matter) who left TDD/writing lots of unit tests
    because of this (and of course because of a boss who wanted a solution
    quickly and did not care about long term problems )

    What do you think?

    Peter

    __
    http://www.rubyrailways.com
     
    Peter Szinek, Dec 5, 2006
    #1
    1. Advertising

  2. Peter Szinek

    James Mead Guest

    On 05/12/06, Peter Szinek <> wrote:
    > 1) Am I doing something wrong? (i.e. that I spend so much time with
    > rewriting/modifying/updating the tests? or is this normal)
    > 2) Are there some tools/methods/ideas to speed things up? Or this is
    > just good so?
    > 3) Maybe I am just too new to these techniques and a skilled TDDist
    > marches much faster?


    It's difficult to give specific comments without actually seeing the
    codebase and tests, but here are some ideas that might help...

    - Don't make a big change to the code and then go through fixing loads
    of broken tests. This is always disheartening and makes you feel like
    you are spending a lot of time fixing tests. Instead break the big
    change down into smaller steps - make a small change to the tests and
    then fix the code. You should be alternating back and forth between
    test and code frequently. Don't get too far away from a green bar i.e.
    try to have as few broken tests as possible at any one time.

    - If you find that making a change to a single class is breaking loads
    of seemingly unrelated tests, this is probably an indication that too
    much code is being covered by each test. It may also indicate a high
    level of coupling in your design. Try to keep your unit tests as
    fine-grained as possible and ensure they really are black-box tests
    i.e. not coupled to the implementation. Using Mock Objects in your
    unit tests is a very useful technique to keep the tests as focussed as
    possible. If you need to write coarser-grained tests, try to write
    them against a relatively stable interface. e.g. the public API you
    will expose.

    --
    James.
    http://blog.floehopper.org
    http://mocha.rubyforge.org - Mock Objects for Ruby
     
    James Mead, Dec 5, 2006
    #2
    1. Advertising

  3. Peter Szinek

    Eric Hodel Guest

    On Dec 5, 2006, at 06:10 , James Mead wrote:
    > On 05/12/06, Peter Szinek <> wrote:
    >> 1) Am I doing something wrong? (i.e. that I spend so much time with
    >> rewriting/modifying/updating the tests? or is this normal)
    >> 2) Are there some tools/methods/ideas to speed things up? Or this is
    >> just good so?
    >> 3) Maybe I am just too new to these techniques and a skilled TDDist
    >> marches much faster?

    >
    > - If you find that making a change to a single class is breaking loads
    > of seemingly unrelated tests, this is probably an indication that too
    > much code is being covered by each test. It may also indicate a high
    > level of coupling in your design. Try to keep your unit tests as
    > fine-grained as possible and ensure they really are black-box tests
    > i.e. not coupled to the implementation. Using Mock Objects in your
    > unit tests is a very useful technique to keep the tests as focussed as
    > possible. If you need to write coarser-grained tests, try to write
    > them against a relatively stable interface. e.g. the public API you
    > will expose.


    When I was new to TDD and unit testing this was my biggest problem,
    especially the coupling side. I've evolved my coding style to
    produce shorter, simpler methods that are easier to test (usually
    less than ten lines, very rarely over 25 lines). Also, I heavily
    refactor tests to move re-usable test code into utility methods.

    --
    Eric Hodel - - http://blog.segment7.net

    I LIT YOUR GEM ON FIRE!
     
    Eric Hodel, Dec 5, 2006
    #3
  4. Peter Szinek

    Trans Guest

    Hi --

    Peter Szinek wrote:
    > My problem is that since the code is being changed a lot (sometimes
    > quite brutal refactoring, even dropping some classes, joining others,
    > throwing out inheritance hierarchy to replace it with composition or
    > vice versa) my development time is 20-30% (maybe more) of updating all
    > the tests (which is kinda annoying) and the remaining to actually write
    > the code.
    >
    > Anyway, I have trapped so much errors when running all the tests between
    > refactorings that the overall development time is much more faster than
    > if I had to bug-hunt for all the obscure weirdness that popped up and
    > catched easily by the tests. I guess this will be even more true as the
    > code will grow.
    >
    > So my question is not if I have to follow these practices or not but rather
    >
    > 1) Am I doing something wrong? (i.e. that I spend so much time with
    > rewriting/modifying/updating the tests? or is this normal)
    > 2) Are there some tools/methods/ideas to speed things up? Or this is
    > just good so?
    > 3) Maybe I am just too new to these techniques and a skilled TDDist
    > marches much faster?
    >
    > What do you think?


    I sympthize. I think TDD makes more sense for applications in which the
    general structure of the program is a given. If youre still working on
    the "what's the best API" then you are probably better off just putting
    the TDD on the back burner until you have it mostly worked out. Then
    you can go back and put in tests. I know that TDD concept promotes
    putting the test before the code, and really one should strive to do so
    (I don;t do it enough myself) but that doesn't mean you always have to
    do so. Really, the most important thing is that you end up with tests.

    T.
     
    Trans, Dec 6, 2006
    #4
  5. Peter Szinek

    Rob Sanheim Guest

    On 12/5/06, Trans <> wrote:
    > I sympthize. I think TDD makes more sense for applications in which the
    > general structure of the program is a given. If youre still working on
    > the "what's the best API" then you are probably better off just putting
    > the TDD on the back burner until you have it mostly worked out. Then
    > you can go back and put in tests. I know that TDD concept promotes
    > putting the test before the code, and really one should strive to do so
    > (I don;t do it enough myself) but that doesn't mean you always have to
    > do so. Really, the most important thing is that you end up with tests.


    The problem with this approach is that when you go back to try and add
    the tests, you realize your code is not test-friendly and testing is
    much harder then it should be. I'd imagine this is more a problem in
    less flexible languages then Ruby, but TDD/BDD definitely drive a
    certain simple, testable design that you don't arrive at otherwise.

    I think a good api can be developed using TDD, and in fact looking at
    the tests used to create an api is one of the best ways to learn it.

    - Rob
     
    Rob Sanheim, Dec 6, 2006
    #5
  6. Peter Szinek

    Bil Kleb Guest

    Trans wrote:
    > Hi --


    Hi.

    > I sympthize. I think TDD makes more sense for applications in which the
    > general structure of the program is a given.


    I respectfully disagree. I find TDD beneficial even when
    just trotting down to the corner store.

    > If youre still working on
    > the "what's the best API" then you are probably better off just putting
    > the TDD on the back burner until you have it mostly worked out.


    This is known as Technical Debt, and IMO, is pure evil
    as in the movie /Time Bandits/.

    > Then you can go back and put in tests. I know that TDD concept promotes
    > putting the test before the code, and really one should strive to do so
    > (I don;t do it enough myself) but that doesn't mean you always have to
    > do so. Really, the most important thing is that you end up with tests.


    But, it'll probably be more costly to do so after the fact -- see
    Beck's /When Should We Test?/ article in the Files section of
    the extremeprogramming Yahoo group.

    Regards,
    --
    Bil Kleb
    http://fun3d.larc.nasa.gov
     
    Bil Kleb, Dec 6, 2006
    #6
  7. Peter Szinek

    Bil Kleb Guest

    Eric Hodel wrote:
    > On Dec 5, 2006, at 06:10 , James Mead wrote:
    >>
    >> - If you find that making a change to a single class is breaking loads
    >> of seemingly unrelated tests, this is probably an indication that too
    >> much code is being covered by each test. It may also indicate a high
    >> level of coupling in your design.

    >
    > When I was new to TDD and unit testing this was my biggest problem,
    > especially the coupling side. I've evolved my coding style to produce
    > shorter, simpler methods that are easier to test (usually less than ten
    > lines, very rarely over 25 lines). Also, I heavily refactor tests to
    > move re-usable test code into utility methods.


    I had a similar experience. TDD reveals design smells in your code.

    Regards,
    --
    Bil Kleb
    http://fun3d.larc.nasa.gov
     
    Bil Kleb, Dec 6, 2006
    #7
  8. > I think a good api can be developed using TDD, and in fact looking at
    > the tests used to create an api is one of the best ways to learn it.


    Yeah, I think this is true. I really only started using TDD very
    recently, but one of the big things that swayed is me that TDD seems
    to be at its best when building APIs, or, more accurately, all the
    APIs I like the most seem to have been written using TDD.

    Anyway, from the original post:

    > I am writing a web extraction framework in Ruby which is now relatively
    > big. I am doing kind of a semi-TDD (i.e. some things are done pure TDD,
    > and I am writing tests later for stuff which are not) - so at the end
    > everything should be covered by tests.
    >
    > I have also tons of black-box tests (i.e. for the specified input and
    > output, yell if the output of the current run is different) and as the
    > code grows, also a great load of unit tests.


    OK, just want to say, there's a definite logical flaw in making
    generalizations about TDD from a project which is "kind of a
    semi-TDD." The broader question of whether or not TDD rocks (I think
    it does) is certainly an interesting question, but at the same time,
    as certainly a different question.

    Anyway, in terms of the actual practical issue, I think the easiest
    interpretation of this is that you're writing too many tests, and that
    some of them are probably testing the wrong things. Any resource on
    testing will reference the idea that TDD consists of writing a test
    and then writing the simplest code that could possibly satisfy that
    test. I think the hidden implication there is that you should also
    write the simplest test that could possibly articulate your
    requirements.

    In practical terms, what I'd do in your situation is, if I made a
    change and it blew up some tests, I'd examine the tests. Any tests
    which depended on implementation details and didn't really have much
    to do with the goals of the application itself, I'd simply throw away.
    That's just a waste of time. Get rid of it. Any tests which were about
    the API internals, rather than just the way a client programmer should
    use the API, I'd either throw those tests away too, or possibly
    separate them.

    This is especially true as you're changing the internals, refactoring,
    throwing away objects, etc. What you're really looking at might be a
    boundary between what elements of the API you expose and what elements
    can change without the client programmer ever even being aware of it.
    Even if the client programmer is you, this distinction is still valid,
    it's basically the core question of OOP. Anyway, I can't see your
    code, I'm just guessing here, but I'd say make that separation more
    explicitly, maybe even to the point of having client tests and
    internal API tests, the goal being to have less tests overall, and for
    those tests you retain to be cleaner and a little more categorized.

    Client programmer tests and functional ("black-box") tests, those are
    important. They make sure that the results you get are the results you
    want. API internal tests, if they're blowing up on details of the
    implementation, simply throw them away. If they're testing for objects
    that don't exist any more, they're just useless distractions. You
    should retain some amount of tests for internals of the API, but
    ideally, those are the only tests that should change when you're
    refactoring. Refactoring means changing the design without changing
    the functionality. If you refactor and a test of the functionality
    blows up, that's really bad news. If you refactor and a test of the
    API internals blows up, that's nothing.

    What it actually sounds like is spaghetti tests. You should really
    only test for things that you need to know. The tests that go to
    specific bits and pieces of the implementation, those should only be
    in a part of the testing code reserved for testing the implementation.
    Higher-level tests should only test higher-level behavior.

    Ideally, if you change only one thing in your application, you should
    only see one test fail. In reality that won't necessarily happen,
    certainly not right away in this particular case, but it becomes a lot
    more likely if you refactor both the tests and the application at the
    same time. In that case, the finer-grained your tests get, the
    finer-grained your application will get along with it. What you want
    to do is have very simple tests running against very simple code, so
    that as soon as a single test breaks, you know exactly what went
    wrong.

    I literally JUST got started using TDD, but I had been putting it off
    for years, and now, I totally 100% advocate it. Being able to
    recognize instantly exactly what went wrong is a REALLY nice thing.
    What makes legacy spaghetti so horrible to work with is the exact
    opposite: it takes you forever to get even a vague idea of where the
    mysterious thing that went wrong might be. Get fine-grained tests and
    fine-grained code and you'll be very happy.

    --
    Giles Bowkett
    http://www.gilesgoatboy.org
    http://gilesbowkett.blogspot.com
    http://gilesgoatboy.blogspot.com
     
    Giles Bowkett, Dec 6, 2006
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John Maclean
    Replies:
    1
    Views:
    350
    Martin P. Hellwig
    Apr 13, 2010
  2. MS Techie
    Replies:
    1
    Views:
    848
    MS Techie
    Dec 23, 2008
  3. Ulrich Eckhardt

    unit-profiling, similar to unit-testing

    Ulrich Eckhardt, Nov 16, 2011, in forum: Python
    Replies:
    6
    Views:
    341
    Roy Smith
    Nov 18, 2011
  4. Bill Mosteller
    Replies:
    0
    Views:
    234
    Bill Mosteller
    Oct 22, 2009
  5. Avi
    Replies:
    0
    Views:
    500
Loading...

Share This Page