Debugging in the large, modern practice?

Discussion in 'Ruby' started by Hugh Sasse, Oct 12, 2006.

  1. Hugh Sasse

    Hugh Sasse Guest

    I think the following may be a badly formed question, but if you'd
    bear with me....

    I have a large application (which is actually a Rails app) which is
    behaving oddly (I can change items in a DB twice, but 4 times
    fails), and using all the conventional approaches I have learned for
    debugging (printing things out, logging to files, ...) it is taking
    me an age to track the problem down. I have no good reason to assert
    that the database or Rails is at fault, it is more likely to be my
    code, but the interactions with the other code make debugging more
    difficult.

    So, my question is this: Given that since I started working in
    computing there have been major strides in software development,
    such as Object Oriented programming becoming mainstream, development
    of concepts like refactoring, development of practices such as the
    Agile methodologies, not to mention developments in networking and
    databases, what are the parallel developments in debugging large
    systems? By large, I mean sufficiently large to cause problems in
    the mental modelling of the dynamic nature of the process, and
    involving considerable quantities of other people's code.

    The experience I have gained seems to be insufficient to meet the
    kinds of demands that cannot be unique to my situation, so there
    must be better approaches out there already if others are meeting
    such demands.

    Given the prevalence of metaprogramming in Ruby, I'll phrase this
    another way, as a meta-question: what are good questions to ask to
    progress along the road of improving one's ability to debug large
    systems?

    Thank you,
    Hugh
     
    Hugh Sasse, Oct 12, 2006
    #1
    1. Advertisements

  2. Debugging large systems is indeed hard -- lots to remember at once
    with too many moving parts. It's much easier to debug small things.

    Alongside all the other strides you mention, testing has improved no
    end as I am sure you are aware. Tools like autotest [1] make it easy
    to run tests against your system all the time, so you notice sooner
    rather than later when it doesn't behave the way you (via your tests)
    expect it to.

    It's daunting when confronted with a large pile of someone else's
    code, especially if that code doesn't have tests, but you have to
    start somewhere. You can write tests against the third-party code
    which over time become your own personal (executable) documentation
    of its API.

    With your specific problem, perhaps you could write a test for the
    operation you are trying to do. Start with one that passes. Now add
    more tests until one fails -- should be easy as it sounds like you
    can reliably make the system fail. Now you can iteratively try to
    write intermediate tests between the one that passes and the one that
    fails, until you isolate the problem to a very small change
    somewhere. Think of it as a binary search via tests of the problem
    space. Hopefully you will be able to converge on the problematic
    needle in the haystack of code.

    And then when you have isolated and fixed the problem, you have a
    nice set of tests which ensure it won't reappear later on -- a
    benefit which manually inspecting log files doesn't confer.

    You probably know all this already, so apologies ;-)

    Good luck,
    Andy Stewart


    [1] http://www.zenspider.com/ZSS/Products/ZenTest/
     
    Andrew Stewart, Oct 12, 2006
    #2
    1. Advertisements

  3. Though I can't speak to your larger question as to history of
    debugging and its progress, I'm finding that writing (unit,
    functional) tests is really helping me understand my own applications
    better as well as understand the frameworks and other code they rely
    on. Tests can fail unexpectedly, and digging a bit to find the reason
    for the exhibited behavior can be very edifying. And as the scope of
    the test decreases, so does the number of parts that are contributing
    to the behavior. I've just had a good couple of days doing nothing
    but writing unit tests and I'm really happy with how I feel more
    comfortable with the code and its behavior.

    Michael Glaesemann
    grzm seespotcode net
     
    Michael Glaesemann, Oct 12, 2006
    #3
  4. Invariant checks ("Design by Contract"), mostly. Adding invariant
    checks to your system will often unearth weird cases - making them
    easily reproducable and making them fail early.

    Testing (which has already been mentioned) can handle some of the same
    niche, by removing coupling and allowing you to externally check for
    invariants being kept. However, it is usually much easier to add
    invariant checks to a system than to add tests.

    Eivind.
     
    Eivind Eklund, Oct 12, 2006
    #4
  5. Plus, when you find the one that fails you will no every subsystem
    involved up to there. You can begin to go write tests for those too
    now, ensuring they function as expected too. Hopefully you would
    slowly zero in on the problem.

    James Edward Gray II
     
    James Edward Gray II, Oct 12, 2006
    #5
  6. I am not being pedantic here, but have you not tried a debugger?
    Standard out is ok for only certain tasks, particularly if there is
    an extended time aspect to the problem.
    -Remote debugging (ie. servet, web service, .NET debugging in your
    IDE as if it was a locally running program)
    -Automated Unit Testing (JUnit, NUnit, RUnit)
    -Unit Test coverage measuring (rcov, JCoverage, Clover)
    -Automated UI testing (of web UIs, excellent examples being WATIR,
    Selenium etc.)

    If you are getting something as fundamental as somethign screwing up
    your DB saves, I would probably start looking at whether you are
    properly covered by your unit tests. If you find gaping holes in your
    test coverage I would write more tests to expose the problem.
    Simple answer: break up large systems into smaller systems
    that can be tested independently (and possibly even replaceable). Its
    not so much a component concept as just the age old practice of
    modular systems.

    I am not sure how this might apply to large Ruby apps, my gut feeling
    is there are not many huge ruby apps out there. The biggest I have
    heard is around 30000 LOC, which is an awful lot for Ruby. Equivalent
    functionality in a more conventional language (Java, C++, NET etc.)
    would be bigger.

    Rails apps are a bit different though, as Rails is an opinionated framework,
    and logically dictates where your code should be. You also don't end up
    writing much Ruby code. If you are, you're probably not leaning on the
    stack enough. My gut feeling is that code bloat in Rails will very quickly
    reveal itself that you are doing something wrong (possibly when test code
    starts to get difficult to write). To get 30000 LOC in a Rails app, when it
    is written well, your app would be huge, massively featured and have a
    pretty huge UI (lots of RHTML), it might be possible to achieve that if you
    are localising views a lot (while globalise nicely lets you avoid this as
    much as possible, things like date controls, right->left reading order,
    might make it easier to localise at the template level.
     
    Richard Conroy, Oct 12, 2006
    #6
  7. A couple of questions:

    1. How large is "large"? Is there some kind of "code size metric" that
    the Ruby community uses, and a tool or tools to measure it?

    2. You say "your code". How much of this application have you personally
    written, how much is "Rails and Ruby and the rest of the
    infrastructure", and how much is "the rest of it"?
    The "traditional CASE tools" -- IDEs, software configuration and project
    management tool sets, the waterfall model, the CMM levels, and of course
    prayer, threats, outsourcing and pizza. :)
    Again, without knowing both the scope of your specific project nor the
    size of the team that built/is building it, it's difficult to answer.
    There are bazillions of failed silver bullets to choose from. My
    personal opinion is that you're being too hard on yourself and that
    anybody who claims to have a tool or a process or a programming language
    that is *significantly* better than today's common practices is either
    deceived, deceiving or both.
    I think first of all, you have to *want* to debug large chunks of other
    peoples' code. It's an acquired taste. I acquired it at one point in my
    career but found it unsatisfying. If you *don't* want to debug large
    chunks of other peoples' code, there are ways you can structure your
    team and processes to minimize how much of it you have to do.

    And I would caution you that, although testing and test-driven
    development are certainly important and worthwhile, testing can only
    show the *presence* of defects, not the absence of defects.

    At the point in my career when I was at the peak of my ability to debug
    other peoples' code, I came up with a simple rule. You'll probably need
    to adjust the time scales to suit your situation, but in my case, the
    rule was: If I don't find the problem in one day, it's not my mistake,
    but a mistake in someone else's work. And if it takes more than a week,
    it's not a software problem, it's a hardware problem. :)

    Good luck ... may the source be with you. :)
     
    M. Edward (Ed) Borasky, Oct 12, 2006
    #7
  8. Hugh Sasse

    Kent Sibilev Guest

    My first bet would be on improving the test case suite. But in the
    process I think that a good debugger is a very valuable tool too. You
    can try ruby-debug, which I find very helpful sometimes:

    http://datanoise.com/articles/2006/09/14/debugging-rails-application
     
    Kent Sibilev, Oct 12, 2006
    #8
  9. Hugh Sasse

    Hugh Sasse Guest

    I've delayed replying for a bit because lots of good stuff is
    coming in, and I didn't want to stem the flow or divert it, ...

    I'm not yet clear where to set the breakpoints, and so on, because
    I don't understand enough about the nature of the failure. But
    I probably need to re-examine the Ruby debugger, because my knowledge
    of it's capabilities is somewhat limited.... So, up to now, no,
    I have avoided this approach.
    Hence the logging...
    I'm not on .NET, I'm on Solaris and also Cygwin. Now, some
    people will want to spit molten sulphur at me for using Cygwin.
    Given that I'm more fluent in Unix than Windows, I'd be glad of
    an alternative if there is one, but I don't think that's the
    problem here anyway. Oh, and the usual vanishingly
    small budget constraints for us seem to apply. "There might be
    money next year"
    Using Watir already. Other testing could do with improvment.
    Well, the logs show the save! commands are getting called OK, with
    the right values. I've said I've no reason to suspect the database.
    Leaving reason aside, my emotions tell me that maybe I should
    suspect, having read rather uncomplimentary stuff about MySQL,
    advocating PostgreSQL, but I don't know enough about the DB issues
    to be sure. This is my first serious DB project. So for now I
    suspect my own code. So I'll need to refactor to improve testing.
    I don't think it's that big.
    I'm sure my use of rails is less than optimal. I still don't fully
    undestand it.
    Thank you.
    Hugh
     
    Hugh Sasse, Oct 12, 2006
    #9
  10. Hugh Sasse

    Hugh Sasse Guest

    Yes, this is something I need to know more about. In Programming Pearls
    I've seen lots of mention of invariants, e.g. for loops, but though
    I think I understood what was being said, I found that I didn't 'grok'
    the statements deeply. I think I need to understand more about how to
    think in this way. Some of this is probably covered by "What could
    I put in an assert statement that would always succeed (for this loop
    or whatever", but I suspect the concept runs deeper.
    Thank you,
    Hugh
     
    Hugh Sasse, Oct 12, 2006
    #10
  11. Hugh Sasse

    Hugh Sasse Guest

    Well, I'm trying to find out about techniques for use where
    complexity and size are problem. What is large for me may not
    be large for others. (I mean, there are some really sharp
    programmers on this list!)
    I have written all the stuff that is not rails or ruby libraries
    out of stdlib. And it took me a months to write, but under
    pressure, so it probably needs rewriting in many places.[Parallel history of debugging vs other CS stuff?]
    I clearly need to find out what IDEs exist for Ruby under
    Unix/Cygwin, and what they'll do for me. I mainly use vim,
    but then I suspect that I'm "so amazingly primative that
    still think [syntax highlighting is] a pretty neat idea". I will
    need to be able to configure any such IDE for large print, and
    light text on dark, which will be a factor in what I choose.
    I am the team. I'd love to be able to code-review this stuff with
    others here, but there's only me. So if it won't fit in my head
    then....
    Yes, I've read this sort of thing in The Mythical Man Month, but I'm
    wondering about all the tech I don't know about. For example, much
    of XP seemed to be established practice that I didn't know about
    until I found out about RUnit circa 2001. So since all these other
    fields I mentioned have progressed, it seems more likely that stuff
    has passed me by than that it doesn't exist yet. I don't understand
    Aspect Oriented Programming yet, for example.
    Some have it thrust upon them. I feel like someone with a spade,
    who's thinking, "They've got steamrollers and cranes, there must
    be mechanical diggers by now".
    Is that, 'for large values of "team"', only?
    I know these ones are present, all right! :)
    I probably need to use the tools more effectively for this case, to
    reach those values.
    Thank you,
    Hugh
     
    Hugh Sasse, Oct 12, 2006
    #11
  12. Hugh Sasse

    Hugh Sasse Guest

    Hugh Sasse, Oct 12, 2006
    #12


  13. On Linux, and probably other UNIX-like operating systems, FreeRide is
    available. I don't know if FreeRide will run under CygWin, but the
    Windows One-Click installer contains a native Windows version of
    FreeRide. Most of the folks here who run Ruby on Windows don't like the
    CygWin version of Ruby, although if you want to run C extensions, it's
    probably still a bit more convenient. In any event, I've had pretty good
    luck mixing native Windows tools with CygWin -- if there's an equivalent
    in CygWin, you usually have to do things with your path, but that's
    about it.
    Well ... knowledge is meant to be shared ... teach someone Rails :).
    Or generative programming, or domain engineering, or domain specific
    languages or language workbenches. Oh, yeah ... design by contract ...
    proving programs correct ... the list is endless.

    I'll claim (again) that the two most important factors in programmer
    productivity are detailed knowledge of the application domain and
    *familiarity* with the syntax and semantics of the programming language
    and other development tools. A *bad* environment can destroy
    productivity, but a good one doesn't guarantee it -- you need an
    acceptable to good environment and detailed familiarity with it.
    I suspect an IDE is the closest to a mechanical digger.
    Well ... more like large values of budget, I think. :)
    In my case, the product in question was a smallish operating system
    which shall remain nameless. The tools were an assembler, documentation
    of the OS table structure and core dumps. There was in fact a "kernel
    debugger" but I rarely used it -- it was too risky on a live system. And
    that was after having been to a training on the OS from the vendor and
    two years of immersion in it at the assembly language level (this was BC
    -- Before "C").
     
    M. Edward (Ed) Borasky, Oct 13, 2006
    #13
  14. I found Bertrand Meyer's "OO Software Construction" provided a lot
    interesting insights in this area and others as well. It's a large book and
    it also covers Eiffel "along the way" but BM definitively has strong
    opinions and well thought out positions that are worthwhile to consider -
    even if you do not agree to all of them. I found the read very fruitful.

    Kind regards

    robert
     
    Robert Klemme, Oct 14, 2006
    #14
  15. Maybe set_trace_func with a complete application trace (of the part that
    fails) helps in understanding what methods get called. Maybe you then
    realize immediately where the bug is (some method called too often or not at
    all). If not you might get an idea where to place your breakpoints or
    detailed debug output.
    Just some questions into the blue: are there triggers involved on DB level?
    Do you use stored procedures vs. simple INSERT, UPDATE, DELETE, SELECT?

    Kind regards

    robert
     
    Robert Klemme, Oct 14, 2006
    #15
  16. Coffee!
    ;-)

    robert
     
    Robert Klemme, Oct 14, 2006
    #16
  17. Hugh Sasse

    Hugh Sasse Guest

    Thank you. I have a copy of that but have only dipped into it so far.
    Hugh
     
    Hugh Sasse, Oct 16, 2006
    #17
  18. Hugh Sasse

    Hugh Sasse Guest

    That's a good idea, thank you.
    I'm using the ActiveRecord API to do all this find() and save() and save!().
    Since the application is not concurrent at the moment I'm not doing anything
    explicit with transactions, and I'm not using stored procedures either.
    I'm not sure when to use stored procedures, for that matter.
    Thank you,
    Hugh
     
    Hugh Sasse, Oct 16, 2006
    #18
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.