Unexpected BigDecimal rounding issue

Discussion in 'Ruby' started by Samuel Lown, May 6, 2009.

  1. Samuel Lown

    Samuel Lown Guest

    Hi,

    Some more not so fun rounding issues.

    Using Floats:
    => true

    Cool, so you'd think it would work with BigDecimal:
    => "30.0000000000000000000000002"

    I'm clearly missing something here as I thought BigDecimal was supposed
    to fix this type of thing.

    Tested in:
    ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
    ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0]

    Can anyone offer an explanation?

    Cheers, sam
     
    Samuel Lown, May 6, 2009
    #1
    1. Advertisements

  2. Hi,

    At Wed, 6 May 2009 10:01:54 +0900,
    Samuel Lown wrote in [ruby-talk:335867]:
    BigDecimal is another kind of floating point number, which uses
    decimal base instead of binary, but has finite digits.
    Therefore, it is impossible to represent exactly a recurring
    decimal theoretically.
     
    Nobuyoshi Nakada, May 6, 2009
    #2
    1. Advertisements

  3. BigDecimal is still a kind of float, albeit a decimal float rather
    than a binary float.

    So it helps with problems when the fractional part of a number can be
    expressed exactly as a finite string of decimal digits, but just as
    certain fractional parts can't be expressed exactly as a finite string
    of binary digits, there are some which have the same problem when the
    base is 10.

    For example 1/3 cannot be expressed as a decimal float, no matter how
    many digits 0.33333.....

    Similarly 3000/116 (which is the same as 30/1.16) can't be expressed
    by a finite sequence of decimal digits.

    It comes out as

    25.862068965517241379310344827[5862068965517241379310344827]...

    where the digits in the brackets repeat infinitely.


    --
    Rick DeNatale

    Blog: http://talklikeaduck.denhaven2.com/
    Twitter: http://twitter.com/RickDeNatale
    WWR: http://www.workingwithrails.com/person/9021-rick-denatale
    LinkedIn: http://www.linkedin.com/in/rickdenatale
     
    Rick DeNatale, May 6, 2009
    #3
  4. Samuel Lown

    Samuel Lown Guest

    Many thanks Nobuyoshi and Rick, that makes complete sense now.

    I guess the safest rule of thumb is if you're doing comparisons for
    humans, use humanized numbers (i.e. integers :)

    Cheers, sam
     
    Samuel Lown, May 6, 2009
    #4
  5. Yeah I think they would be better off with rationals

    Blog: http://random8.zenunit.com/
    Learn: http://sensei.zenunit.com/
    Twitter: http://twitter.com/random8r

     
    Julian Leviston, May 6, 2009
    #5
  6. Or define a maximum error, i.e. do not rely on identical values but
    rather the difference of the quotient.

    Kind regards

    robert
     
    Robert Klemme, May 6, 2009
    #6
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.