floating point problem... floating indeed :(

Discussion in 'Ruby' started by teeshift, Dec 1, 2006.

  1. teeshift

    teeshift Guest

    I recently wrote a code and haven't been found the bug and start
    suspecting that it was
    the floating computation that causing the problem. I did a little
    experiement and found the
    following phenomena:

    I am using

    Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
    GNU/Linux
    Ubuntu is the OS.

    #####################
    #!/usr/bin/ruby1.9

    a = 1.0

    a.to_f

    while(a < 15.0)
    a = a + 0.1
    print a,"\t",a * a,"\n"
    end
    #####################

    and the output is...(as following) quite odd. Anyone can explain it?
    or a way to avoid it (while
    keeping the precision)


    1.1 1.21
    1.2 1.44
    1.3 1.69
    1.4 1.96
    1.5 2.25
    1.6 2.56
    1.7 2.89
    1.8 3.24
    1.9 3.61
    2.0 4.0
    2.1 4.41
    2.2 4.84
    2.3 5.29000000000001
    2.4 5.76000000000001
    2.5 6.25000000000001
    2.6 6.76000000000001
    2.7 7.29000000000001
    2.8 7.84000000000001
    2.9 8.41000000000001
    3.0 9.00000000000001
    3.1 9.61000000000001
    3.2 10.24
    3.3 10.89
    3.4 11.56
    3.5 12.25
    3.6 12.96
    3.7 13.69
    3.8 14.44
    3.9 15.21
    4.0 16.0
    4.1 16.81
    4.2 17.64
    4.3 18.49
    4.4 19.36
    4.5 20.25
    4.6 21.16
    4.7 22.09
    4.8 23.04
    4.9 24.01
    5.0 25.0
    5.1 26.01
    5.2 27.04
    5.3 28.09
    5.4 29.16
    5.5 30.25
    5.6 31.36
    5.7 32.49
    5.8 33.64
    5.9 34.81
    6.0 35.9999999999999
    6.1 37.2099999999999
    6.19999999999999 38.4399999999999
    6.29999999999999 39.6899999999999
    6.39999999999999 40.9599999999999
    6.49999999999999 42.2499999999999
    6.59999999999999 43.5599999999999
     
    teeshift, Dec 1, 2006
    #1
    1. Advertising

  2. On 11/30/06, teeshift <> wrote:
    > I recently wrote a code and haven't been found the bug and start
    > suspecting that it was
    > the floating computation that causing the problem. I did a little
    > experiement and found the
    > following phenomena:
    >
    > I am using
    >
    > Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
    > GNU/Linux
    > Ubuntu is the OS.


    See this writeup for some details on why this happens.
    http://en.wikipedia.org/wiki/IEEE_754
     
    Wilson Bilkovich, Dec 1, 2006
    #2
    1. Advertising

  3. teeshift

    Chris Pearl Guest

    Someone else might wish to give a more concrete explanation, but as a
    rule, floating-point calculations - using the usual Float datatype -
    are not accurate, and prone to rounding errors such as the above.

    If you're dealing with limited precision (i.e. X digits after the
    decimal point) you should consider using the Decimal datatype instead.

    On 12/1/06, teeshift <> wrote:
    > I recently wrote a code and haven't been found the bug and start
    > suspecting that it was
    > the floating computation that causing the problem. I did a little
    > experiement and found the
    > following phenomena:
    >
    > I am using
    >
    > Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
    > GNU/Linux
    > Ubuntu is the OS.
    >
    > #####################
    > #!/usr/bin/ruby1.9
    >
    > a = 1.0
    >
    > a.to_f
    >
    > while(a < 15.0)
    > a = a + 0.1
    > print a,"\t",a * a,"\n"
    > end
    > #####################
    >
    > and the output is...(as following) quite odd. Anyone can explain it?
    > or a way to avoid it (while
    > keeping the precision)
    >
    >
    > 1.1 1.21
    > 1.2 1.44
    > 1.3 1.69
    > 1.4 1.96
    > 1.5 2.25
    > 1.6 2.56
    > 1.7 2.89
    > 1.8 3.24
    > 1.9 3.61
    > 2.0 4.0
    > 2.1 4.41
    > 2.2 4.84
    > 2.3 5.29000000000001
    > 2.4 5.76000000000001
    > 2.5 6.25000000000001
    > 2.6 6.76000000000001
    > 2.7 7.29000000000001
    > 2.8 7.84000000000001
    > 2.9 8.41000000000001
    > 3.0 9.00000000000001
    > 3.1 9.61000000000001
    > 3.2 10.24
    > 3.3 10.89
    > 3.4 11.56
    > 3.5 12.25
    > 3.6 12.96
    > 3.7 13.69
    > 3.8 14.44
    > 3.9 15.21
    > 4.0 16.0
    > 4.1 16.81
    > 4.2 17.64
    > 4.3 18.49
    > 4.4 19.36
    > 4.5 20.25
    > 4.6 21.16
    > 4.7 22.09
    > 4.8 23.04
    > 4.9 24.01
    > 5.0 25.0
    > 5.1 26.01
    > 5.2 27.04
    > 5.3 28.09
    > 5.4 29.16
    > 5.5 30.25
    > 5.6 31.36
    > 5.7 32.49
    > 5.8 33.64
    > 5.9 34.81
    > 6.0 35.9999999999999
    > 6.1 37.2099999999999
    > 6.19999999999999 38.4399999999999
    > 6.29999999999999 39.6899999999999
    > 6.39999999999999 40.9599999999999
    > 6.49999999999999 42.2499999999999
    > 6.59999999999999 43.5599999999999
    >
    >
    >
     
    Chris Pearl, Dec 1, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. H aka N
    Replies:
    15
    Views:
    15,676
    Ben Jones
    Mar 2, 2006
  2. wASP
    Replies:
    11
    Views:
    617
  3. Motaz Saad
    Replies:
    7
    Views:
    6,496
  4. Replies:
    4
    Views:
    1,293
    Default User
    Feb 22, 2006
  5. Saraswati lakki
    Replies:
    0
    Views:
    1,350
    Saraswati lakki
    Jan 6, 2012
Loading...

Share This Page