bad maths

M

Martin larsson

Hi

i = 0
100.times {
p i += 0.6
}

results in:
5.6
5.7
5.8
5.9
5.99999999999999
6.09999999999999
6.19999999999999

on my XP maching with ruby 1.8.1-13. I used rubyinstaller to install it
if that matters

anyone more have this problem? anyway to fix it?
 
M

Martin larsson

Martin said:
Hi

i = 0
100.times {
p i += 0.6
}

results in:
5.6
5.7
5.8
5.9
5.99999999999999
6.09999999999999
6.19999999999999

on my XP maching with ruby 1.8.1-13. I used rubyinstaller to install
it if that matters

anyone more have this problem? anyway to fix it?
sry, i was testing around with different numbers so i copied wrong
result for that litte script, just change 0.6 to 0.1 and the the result
is right
 
H

Hal Fulton

Martin said:
Hi

i = 0
100.times {
p i += 0.6
}

results in:
5.6
5.7
5.8
5.9
5.99999999999999
6.09999999999999
6.19999999999999

on my XP maching with ruby 1.8.1-13. I used rubyinstaller to install it
if that matters

anyone more have this problem? anyway to fix it?

Look for a thread about a week or so ago. :)

I suggest you round off the numbers for output and/or for
calculations. And use integer arithmetic wherever you can.


Hal
 
L

Lloyd Zusman

Martin larsson said:
sry, i was testing around with different numbers so i copied wrong
result for that litte script, just change 0.6 to 0.1 and the the result
is right

First, look up "floating point arithmetic" on Google (include the
double quotes in your search).

This link is also quite useful:

http://docs.sun.com/source/806-3568/ncg_goldberg.html

Then, read the recent thread in this list called "Major addition bug?".
This thread starts with the following message ID:

<[email protected]>
 
B

Bret Jolly

Martin larsson said:
Martin larsson wrote:
[repeated adding of 0.1 not coming out exact]

The problem is that the computer thinks in binary and
0.1 has an infinitely long binary representation. (You
won't see this "problem" if you instead add 0.5, which
has a finitely long representation) This is not a ruby
problem, but a limitation of the computer's floating point.
In general, you should regard floating point numbers as
being slightly inexact. If you need exactness, require
the mathn library and use rationals. Imitating your example,
you'd get something like:

require 'mathn'
i = 5
11.times {puts i += 1/10} # => exact rational results

As an alternative, you could limit the precision of your
output, which effectively rounds the numbers to output
precision:

i = 5.5
5.times { puts "%5.3f" % (i += 0.1)}
#output follows:
5.600
5.700
5.800
5.900
6.000

Regards, Bret
Bret Jolly (Jou Lii Bair)
 
B

Bret Jolly

A small addendum to my previous answer. If you need exact decimal
arithmetic (as you do if the decimals represent dollars and cents)
you should use the BigDecimal class. That works in base 10 rather
than base 2. (Actually, it works in base 10000, which is mathematically
equivalent to base 10.) Imitating your example:
require 'bigdecimal'
a = BigDecimal("5.5", 20) # the 20 indicates the precision
5.times {puts (a += BigDecimal("0.1")).to_s}
# output follows:
0.56E1
0.57E1
0.58E1
0.59E1
0.6E1

No nonsense about 5.99999... here.

The BigDecimal class is useful when you want the arithmetic to
be decimal rather than binary, and especially when you need extra
precision beyond what floating-point can achieve. It is, however,
slower than floating point, a bit trickier to use (since you have
to set precision or make sure the implicit settings are acceptable)
and its method library is less rich than what the Math module
provides for Float.

Regards, Bret
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top