how to reduce bugs due to incorrect indentation

Discussion in 'Python' started by msustik@gmail.com, Feb 6, 2014.

  1. Guest

    I had a bug in a Python script recently. The code in question was something along the lines of:

    if a == 1:
    x = y
    else:
    x = z
    y = z + y
    z = z + 1

    While editing this file I accidentally pushed TAB on the line with 'y = z + y'.

    My changes were elsewhere and I did not notice the above one line change when I looked at the diffs before commit. I should have noticed it...

    It was rare that a was 1 and therefore the problem did not show up for a while. (I know I should have had tests exercising all cases...)

    When the bug showed up, it was kind of difficult to remember what was the original intent. Fortunately, looking at old versions allowed me to find the problem commit and the bug.

    Any suggestion on how to avoid this type of error in the future?

    Thanks!
     
    , Feb 6, 2014
    #1
    1. Advertising

  2. On Thu, Feb 6, 2014 at 2:08 PM, Dan Sommers <> wrote:
    > On Wed, 05 Feb 2014 19:02:09 -0800, msustik wrote:
    >
    >> My changes were elsewhere and I did not notice the above one line
    >> change when I looked at the diffs before commit. I should have noticed
    >> it...
    >>
    >> It was rare that a was 1 and therefore the problem did not show up for
    >> a while. (I know I should have had tests exercising all cases...)
    >>
    >> When the bug showed up, it was kind of difficult to remember what was
    >> the original intent. Fortunately, looking at old versions allowed me
    >> to find the problem commit and the bug.
    >>
    >> Any suggestion on how to avoid this type of error in the future?

    >
    > I think you just mentioned the best suggestions: (a) have a good
    > collection of unit tests (where "good" means data and branch coverage),
    > and (b) look carefully, or have someone else look carefully, at every
    > commit.


    Yep, I'd agree with that. Additionally, a habit of small and
    conceptual commits helps hugely, even if you don't look at them too
    carefully as they go through; get familiar with 'git gui blame' or
    whatever the equivalent is for your system (or just browsing through
    'gitk filename.py' or equiv) and you can go back in time to pin down
    bugs. I had to fight heavily to get my boss to understand this,
    because he said he _never_ went back through the repo's history;
    meanwhile, I was doing so frequently, and knowing exactly what code
    was changed, when, and why. His attitude to bugs was "don't make any,
    and you're a bad programmer if you do". My attitude to bugs was, and
    is, "be honest with yourself, you will make them, so design your
    systems to handle that".

    When your commits are small and tidy, you can often find bugs really
    REALLY easily just by looking at the commit that changed some
    particular line. With gitk, I'll often make a dummy edit to a line,
    then highlight the red "this line deleted" line in the uncommitted
    changes display, right-click, "Show origin of this line".
    Alternatively, 'git blame' or other annotation can work, but I
    generally find that annotating a whole file is overkill and trying to
    ask for annotations of one small section is tedious. But whichever way
    you do it, you should be able to VERY quickly go from "Hmm, I wonder
    why this is indented like this?" to "Here's the commit that made it
    like that", and then you can easily decide whether it was right or
    not.

    Of course, your commit messages have to be useful. I can't honestly
    say my own commit messages are perfect, but at least they're not like
    these:

    https://github.com/douglascrockford/JSLint/commits/master

    Note to future readers: That link will take you to the most recent
    commits on that repo. It may be (it may be!) that, by the time you
    look at it, he'll have changed his practice and started writing
    exemplary messages. If so, just page back through the history and find
    how it looked in 2014. :)

    ChrisA
     
    Chris Angelico, Feb 6, 2014
    #2
    1. Advertising

  3. Asaf Las Guest

    On Thursday, February 6, 2014 5:02:09 AM UTC+2, wrote:
    > I had a bug in a Python script recently. The code in question was
    > something along the lines of:
    > if a == 1:
    > x = y
    > else:
    > x = z
    > y = z + y
    > z = z + 1
    >
    > While editing this file I accidentally pushed TAB on the line with
    > 'y = z + y'.
    > My changes were elsewhere and I did not notice the above one line
    > change when I looked at the diffs before commit. I should have noticed
    > it...
    >
    > It was rare that a was 1 and therefore the problem did not show up
    > for a while. (I know I should have had tests exercising all cases...)
    > When the bug showed up, it was kind of difficult to remember what
    > was the original intent. Fortunately, looking at old versions allowed
    > me to find the problem commit and the bug.
    > Any suggestion on how to avoid this type of error in the future?
    > Thanks!


    I see.

    The only solution to mimic C style curly brackets comes to my mind is
    to use hash to mark start and stop of your block as:

    if a == 1:#
    x = y
    #
    else:#
    x = z
    #
    y = z + y
    z = z + 1

    and PEP ... Nulla poena sine lege :)

    /Asaf
     
    Asaf Las, Feb 6, 2014
    #3
  4. Terry Reedy Guest

    On 2/5/2014 10:02 PM, wrote:

    > if a == 1:
    > x = y
    > else:
    > x = z
    > y = z + y
    > z = z + 1
    >
    > While editing this file I accidentally pushed TAB on the line with 'y = z + y'.


    In this particular case, remove the indentation with
    x = y if a == 1 else z
    and indenting the next line is syntax error.
    --
    Terry Jan Reedy
     
    Terry Reedy, Feb 6, 2014
    #4
  5. On 2014-02-06, <> wrote:
    > I had a bug in a Python script recently. The code in question was something along the lines of:
    >
    > if a == 1:
    > x = y
    > else:
    > x = z
    > y = z + y
    > z = z + 1
    >
    > While editing this file I accidentally pushed TAB on the line
    > with 'y = z + y'.
    >
    > Any suggestion on how to avoid this type of error in the future?


    The best advice is to pay closer attention to what you're doing. Look
    at the code while you're editing, not your fingers. Before you commit
    the change, spend some time looking carefully at it to verify that
    changes were intentional.

    --
    Grant Edwards grant.b.edwards Yow! I left my WALLET in
    at the BATHROOM!!
    gmail.com
     
    Grant Edwards, Feb 6, 2014
    #5
  6. Guest

    Thanks for all the suggestions!
    Best,
    -Matyas
     
    , Feb 6, 2014
    #6
  7. schreef:
    > I had a bug in a Python script recently. The code in question was something along the lines of:
    >
    > if a == 1:
    > x = y
    > else:
    > x = z
    > y = z + y
    > z = z + 1
    >
    > While editing this file I accidentally pushed TAB on the line with 'y = z + y'.
    >
    > My changes were elsewhere and I did not notice the above one line change when I looked at the diffs before commit. I should have noticed it...
    >
    > It was rare that a was 1 and therefore the problem did not show up for a while. (I know I should have had tests exercising all cases...)
    >
    > When the bug showed up, it was kind of difficult to remember what was the original intent. Fortunately, looking at old versions allowed me to find the problem commit and the bug.
    >
    > Any suggestion on how to avoid this type of error in the future?


    My suggestion: configure your editor to insert the appropriate amount of
    spaces instead of a tab when you press the tab key.


    Best regards,
    Roel

    --
    "Met een spitsvondig citaat bewijs je niets."
    -- Voltaire
     
    Roel Schroeven, Feb 6, 2014
    #7
  8. On Thu, Feb 6, 2014 at 3:29 PM, Roel Schroeven <> wrote:
    > schreef:
    >>
    >> I had a bug in a Python script recently. The code in question was
    >> something along the lines of:
    >>
    >> if a == 1:
    >> x = y
    >> else:
    >> x = z
    >> y = z + y
    >> z = z + 1
    >>
    >> While editing this file I accidentally pushed TAB on the line with 'y = z
    >> + y'.
    >>
    >> My changes were elsewhere and I did not notice the above one line change
    >> when I looked at the diffs before commit. I should have noticed it...
    >>
    >> It was rare that a was 1 and therefore the problem did not show up for a
    >> while. (I know I should have had tests exercising all cases...)
    >>
    >> When the bug showed up, it was kind of difficult to remember what was the
    >> original intent. Fortunately, looking at old versions allowed me to find the
    >> problem commit and the bug.
    >>
    >> Any suggestion on how to avoid this type of error in the future?

    >
    >
    > My suggestion: configure your editor to insert the appropriate amount of
    > spaces instead of a tab when you press the tab key.


    +1 - tabs are evil.
     
    Larry Martell, Feb 6, 2014
    #8
  9. Ethan Furman Guest

    On 02/06/2014 12:36 PM, Larry Martell wrote:
    > On Thu, Feb 6, 2014 at 3:29 PM, Roel Schroeven wrote:
    >> schreef:
    >>>
    >>> While editing this file I accidentally pushed TAB on the line with 'y = z
    >>> + y'.

    >>
    >> My suggestion: configure your editor to insert the appropriate amount of
    >> spaces instead of a tab when you press the tab key.

    >
    > +1 - tabs are evil.


    Tabs are not evil, and an argument can be made that tabs are better (a decent editor can be configured to show x many
    spaces per tab, then users could decide how much indentation they preferred to see... but I digress).

    Using spaces instead of tabs would also have not prevented the error that Msustik encountered, and for that matter we
    don't know whether he was using tabs or spaces in his source file, only that he hit the Tab key -- surely you are not
    suggesting everyone rip out their tab key and just hit the space bar four times for each level of indentation? ;)

    --
    ~Ethan~
     
    Ethan Furman, Feb 6, 2014
    #9
  10. On Thu, Feb 6, 2014 at 4:32 PM, Ethan Furman <> wrote:
    > On 02/06/2014 12:36 PM, Larry Martell wrote:
    >>
    >> On Thu, Feb 6, 2014 at 3:29 PM, Roel Schroeven wrote:
    >>>
    >>> schreef:
    >>>>
    >>>>
    >>>> While editing this file I accidentally pushed TAB on the line with 'y =
    >>>> z
    >>>> + y'.
    >>>
    >>>
    >>> My suggestion: configure your editor to insert the appropriate amount of
    >>> spaces instead of a tab when you press the tab key.

    >>
    >>
    >> +1 - tabs are evil.

    >
    >
    > Tabs are not evil, and an argument can be made that tabs are better (a
    > decent editor can be configured to show x many spaces per tab, then users
    > could decide how much indentation they preferred to see... but I digress).
    >
    > Using spaces instead of tabs would also have not prevented the error that
    > Msustik encountered, and for that matter we don't know whether he was using
    > tabs or spaces in his source file, only that he hit the Tab key -- surely
    > you are not suggesting everyone rip out their tab key and just hit the space
    > bar four times for each level of indentation? ;)


    The Tab key is not evil, it's the tab character (Ctrl-I). I have been
    bitten by this many time when I had to work on a program written by
    another. They had their tab stops set at 5 or 6, mine is set at 4, or
    they did not have expandtab set, but I did. So you get either a script
    that looks misaligned, but works, or one that does not look misaligned
    but doesn't work. When I have to pick up someone else's script the
    first thing I do is replace the tabs with spaces.
     
    Larry Martell, Feb 6, 2014
    #10
  11. On Fri, Feb 7, 2014 at 9:09 AM, Larry Martell <> wrote:
    > The Tab key is not evil, it's the tab character (Ctrl-I). I have been
    > bitten by this many time when I had to work on a program written by
    > another. They had their tab stops set at 5 or 6, mine is set at 4, or
    > they did not have expandtab set, but I did. So you get either a script
    > that looks misaligned, but works, or one that does not look misaligned
    > but doesn't work. When I have to pick up someone else's script the
    > first thing I do is replace the tabs with spaces.


    All you've proven is that *mixing* spaces and tabs is evil. It's like
    arguing that oil is evil because, when you mix it with water, weird
    stuff happens. But that doesn't mean I want to fry my bacon in water.

    Mmm, bacon.

    Sorry. I'm back now. Ahem. Arguably, a better fix is to replace spaces
    with tabs, because they're more obvious. But mainly, just be
    consistent. Whatever one file uses, it uses exclusively. It'd be
    pretty easy to create a git commit hook that checks files for leading
    indentation and rejects the commit if it's mismatched; I would guess
    the same is true in Mercurial.

    But none of this would solve the OP's original issue. Whether it's a
    tab or spaces, unexpectedly indenting a line of code is a problem.
    It's no different from accidentally hitting Ctrl-T in SciTE and
    reordering two lines, when one line depends on the other. It's a bug.
    So you look at your commits before you make them (to give yourself a
    chance to catch it quickly), and you make sure you can always look
    back over your commits (in case you didn't catch it quickly). Much
    better than blaming the characters involved. Poor innocent U+0009.

    ChrisA
     
    Chris Angelico, Feb 6, 2014
    #11
  12. Asaf Las Guest

    On Friday, February 7, 2014 12:30:17 AM UTC+2, Chris Angelico wrote:
    > On Fri, Feb 7, 2014 at 9:09 AM, Larry Martell <> wrote:
    >
    > > The Tab key is not evil, it's the tab character (Ctrl-I). I have been
    > > bitten by this many time when I had to work on a program written by
    > > another. They had their tab stops set at 5 or 6, mine is set at 4, or
    > > they did not have expandtab set, but I did. So you get either a script
    > > that looks misaligned, but works, or one that does not look misaligned
    > > but doesn't work. When I have to pick up someone else's script the
    > > first thing I do is replace the tabs with spaces.

    >
    > All you've proven is that *mixing* spaces and tabs is evil. It's like
    > arguing that oil is evil because, when you mix it with water, weird
    > stuff happens. But that doesn't mean I want to fry my bacon in water.
    > Mmm, bacon.
    > Sorry. I'm back now. Ahem. Arguably, a better fix is to replace spaces
    > with tabs, because they're more obvious. But mainly, just be
    > consistent. Whatever one file uses, it uses exclusively. It'd be
    > pretty easy to create a git commit hook that checks files for leading
    > indentation and rejects the commit if it's mismatched; I would guess
    > the same is true in Mercurial.
    >
    > But none of this would solve the OP's original issue. Whether it's a
    > tab or spaces, unexpectedly indenting a line of code is a problem.
    > It's no different from accidentally hitting Ctrl-T in SciTE and
    > reordering two lines, when one line depends on the other. It's a bug.
    > So you look at your commits before you make them (to give yourself a
    > chance to catch it quickly), and you make sure you can always look
    > back over your commits (in case you didn't catch it quickly). Much
    > better than blaming the characters involved. Poor innocent U+0009.
    > ChrisA


    pep8 pushed \t to dark side in Python.

    though it is better that spaces sometimes. let say someone's indented code
    with 2 spaces and user is comfortable with 4. if \t then it is done by
    editor's conf without touching code.

    /Asaf
     
    Asaf Las, Feb 6, 2014
    #12
  13. On Fri, Feb 7, 2014 at 10:01 AM, Asaf Las <> wrote:
    > pep8 pushed \t to dark side in Python.


    Only for the Python stdlib, and only because a decision has to be made
    one way or the other. I believe Guido stated at one point that there
    was only a very weak push toward "spaces only" rather than "tabs
    only", just that it had to be one of those.

    ChrisA
     
    Chris Angelico, Feb 7, 2014
    #13
  14. Guest

    On Thursday, February 6, 2014 12:29:36 PM UTC-8, Roel Schroeven wrote:
    >
    > My suggestion: configure your editor to insert the appropriate amount of
    >
    > spaces instead of a tab when you press the tab key.


    You misunderstood the problem, but managed to start a Tab war! :)

    My emacs inserts 4 spaces in python mode when I press the tab key. Python uses the indentation to decide how many lines following the else are executed in the else branch or after the else branch (always that is).

    By pressing inadvertently the Tab key I changed the semantics of the code while it is still syntactically correct and PEP8 compliant. This was indeed a user error and my question was towards practices reducing the chance of this happening again.

    Based on the responses I arrived to the conclusion that there is no better solution than trying to be careful and have good testing suites.

    It would be possible to disable the Tab key completely and type in the spaces all the time. (It is much less likely that one would press the space baraccidentally four times or hold it down to get 4 spaces by mistake.)

    Unfortunately this means giving up the indentation help of the editor and that will slow down coding. It will also lead to many indentation mistakes during development (most of which will be caught right away however. Maybe acoloring of the background based on tab position could assist in this.

    I also considered adding an extra blank line after the if-else block (similarly for loops) in the hope that it would reduce the chance of missing an inadvertent indentation after the block.

    However, this defeats somewhat the python paradigm that got rid of closing braces and endif-s etc. used in other languages to allow more compact code.

    -Matyas


    >
    >
    >
    >
    >
    > Best regards,
    >
    > Roel
    >
    >
    >
    > --
    >
    > "Met een spitsvondig citaat bewijs je niets."
    >
    > -- Voltaire
     
    , Feb 7, 2014
    #14
  15. On Fri, Feb 7, 2014 at 12:20 PM, <> wrote:
    > It would be possible to disable the Tab key completely and type in the spaces all the time. (It is much less likely that one would press the space bar accidentally four times or hold it down to get 4 spaces by mistake.)
    >
    > Unfortunately this means giving up the indentation help of the editor andthat will slow down coding. It will also lead to many indentation mistakesduring development (most of which will be caught right away however. Maybea coloring of the background based on tab position could assist in this.
    >


    I don't know that it'd really help much anyway. You might reduce one
    chance of making errors by hitting a single key, but at the cost of
    stupid syntactic salt (indentation requires hitting a key four times?
    No thanks), and your fingers would just get used to
    whack-whack-whack-whack. No change.

    You can spend all your time trying to warp your coding style around
    preventing this bug or that bug from happening, or you can just
    acknowledge that bugs WILL happen and handle them after the event.
    (Hence, source control.) Suppose you come up with a solution to the
    accidental-indentation problem. What are you going to do about this
    one?

    def foo(bar):
    if not bar: bat = [0]
    for x in bar:
    print(len(bar),x)

    Now, why is your empty-list handling not working? Oh, there was a
    typo. How are you going to deal with that? Well, you could bring in
    C-style variable declarations; then you'd get an immediate error
    ('bat' is undeclared), but somehow I don't think most Python
    programmers would prefer this :) Now personally, I do quite like
    declared variables, because they allow infinitely-nested scoping, and
    I find that feature worth the effort of declaring all my locals; but
    it's a tradeoff, and I wouldn't go to that level of effort *just* to
    catch typos in variable names. What if there had been a 'bat' at a
    higher scope? Then the typo just means the code does something else
    wrong. No fundamental difference.

    There was a scheme posted to this list a little while ago to have
    variable names shown in different colors, which might have helped. (I
    disagree with the author's idea that similar names should be in
    similar colors - I think that similar names should be in DISsimilar
    colors, specifically to catch this sort of error. But anyway.) That's
    a theory that might help... but it still might not. And what if your
    error is in a literal string that later gets parsed? No, there's no
    way that you can catch everything beforehand.

    Bugs happen. Find 'em, fix 'em.

    ChrisA
     
    Chris Angelico, Feb 7, 2014
    #15
  16. Chris Angelico schreef:
    > But none of this would solve the OP's original issue. Whether it's a
    > tab or spaces, unexpectedly indenting a line of code is a problem.


    I had misread. I thought the problem was that the OP did want to indent,
    but accidentally used the tab key instead of the space bar to do so,
    introducing the dreaded mixing of tabs and spaces. Remapping the tab key
    would solve that problem.

    But the real problem was different, so my advice was indeed not a
    solution to the problem. I should've read better.

    --
    The saddest aspect of life right now is that science gathers knowledge
    faster than society gathers wisdom.
    -- Isaac Asimov

    Roel Schroeven
     
    Roel Schroeven, Feb 7, 2014
    #16
  17. schreef:
    > On Thursday, February 6, 2014 12:29:36 PM UTC-8, Roel Schroeven wrote:
    >> My suggestion: configure your editor to insert the appropriate amount of
    >>
    >> spaces instead of a tab when you press the tab key.

    >
    > You misunderstood the problem, but managed to start a Tab war! :)


    Indeed I did; sorry for both :)

    --
    The saddest aspect of life right now is that science gathers knowledge
    faster than society gathers wisdom.
    -- Isaac Asimov

    Roel Schroeven
     
    Roel Schroeven, Feb 7, 2014
    #17
  18. Hi,

    On 7.2.2014. 2:20, wrote:
    > Based on the responses I arrived to the conclusion that there
    > is no better solution than trying to be careful and have good
    > testing suites.
    >
    > It would be possible to disable the Tab key completely
    > ...[snipped]...
    > Maybe a coloring of the background based on tab position
    > ...[snipped]...
    > I also considered
    > ...[snipped]...


    YMMV, but for me, just reading through this fun thread took more
    time then ever debugging issues caused by bad Python code
    indentation. :-D

    So, my suggestion would be to just ignore the problem and deal
    with any resulting issues as they occur.

    Clean coding & development practices, some of which have been
    mentioned earlier in this thread and are useful for many other
    reasons as well, will additionally reduce the chance of such
    errors causing any non-trivial issues.

    Best regards,
    Jurko Gospodnetić
     
    Jurko Gospodnetić, Feb 8, 2014
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. DAvid

    reduce the CLB

    DAvid, Oct 24, 2004, in forum: VHDL
    Replies:
    2
    Views:
    621
  2. Keith Thompson

    groups.google.com indentation bugs [semi-OT]

    Keith Thompson, Jan 9, 2005, in forum: C Programming
    Replies:
    10
    Views:
    565
    Flash Gordon
    Feb 1, 2005
  3. Fresh
    Replies:
    2
    Views:
    638
    Bo Persson
    Apr 22, 2008
  4. Josef 'Jupp' Schugt

    Still use 'ruby-bugs' for Ruby bugs?

    Josef 'Jupp' Schugt, Nov 4, 2004, in forum: Ruby
    Replies:
    2
    Views:
    163
    Tom Copeland
    Nov 4, 2004
  5. Jesse B.
    Replies:
    2
    Views:
    192
    Josh Cheek
    Mar 27, 2010
Loading...

Share This Page