floating point problem... floating indeed :(

T

teeshift

I recently wrote a code and haven't been found the bug and start
suspecting that it was
the floating computation that causing the problem. I did a little
experiement and found the
following phenomena:

I am using

Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
GNU/Linux
Ubuntu is the OS.

#####################
#!/usr/bin/ruby1.9

a = 1.0

a.to_f

while(a < 15.0)
a = a + 0.1
print a,"\t",a * a,"\n"
end
#####################

and the output is...(as following) quite odd. Anyone can explain it?
or a way to avoid it (while
keeping the precision)


1.1 1.21
1.2 1.44
1.3 1.69
1.4 1.96
1.5 2.25
1.6 2.56
1.7 2.89
1.8 3.24
1.9 3.61
2.0 4.0
2.1 4.41
2.2 4.84
2.3 5.29000000000001
2.4 5.76000000000001
2.5 6.25000000000001
2.6 6.76000000000001
2.7 7.29000000000001
2.8 7.84000000000001
2.9 8.41000000000001
3.0 9.00000000000001
3.1 9.61000000000001
3.2 10.24
3.3 10.89
3.4 11.56
3.5 12.25
3.6 12.96
3.7 13.69
3.8 14.44
3.9 15.21
4.0 16.0
4.1 16.81
4.2 17.64
4.3 18.49
4.4 19.36
4.5 20.25
4.6 21.16
4.7 22.09
4.8 23.04
4.9 24.01
5.0 25.0
5.1 26.01
5.2 27.04
5.3 28.09
5.4 29.16
5.5 30.25
5.6 31.36
5.7 32.49
5.8 33.64
5.9 34.81
6.0 35.9999999999999
6.1 37.2099999999999
6.19999999999999 38.4399999999999
6.29999999999999 39.6899999999999
6.39999999999999 40.9599999999999
6.49999999999999 42.2499999999999
6.59999999999999 43.5599999999999
 
W

Wilson Bilkovich

I recently wrote a code and haven't been found the bug and start
suspecting that it was
the floating computation that causing the problem. I did a little
experiement and found the
following phenomena:

I am using

Linux source 2.6.15-27-386 #1 PREEMPT Sat Sep 16 01:51:59 UTC 2006 i686
GNU/Linux
Ubuntu is the OS.

See this writeup for some details on why this happens.
http://en.wikipedia.org/wiki/IEEE_754
 
C

Chris Pearl

Someone else might wish to give a more concrete explanation, but as a
rule, floating-point calculations - using the usual Float datatype -
are not accurate, and prone to rounding errors such as the above.

If you're dealing with limited precision (i.e. X digits after the
decimal point) you should consider using the Decimal datatype instead.
 

Members online

Forum statistics

Threads
473,734
Messages
2,569,441
Members
44,832
Latest member
GlennSmall

Latest Threads

Top