Rounding error, (100.0 * 9.95).to_i == 994

[Dan Janowski]

I can understand floating point errors, but this one seems too small
for that.
Especially given this:

irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994


irb(main):016:0> (100.0*9.95).to_i
=> 994
irb(main):017:0> (100*9.95).to_i
=> 994
irb(main):018:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):019:0> RUBY_VERSION
=> "1.8.1"
irb(main):020:0> RUBY_PLATFORM
=> "sparc-solaris2.8"


irb(main):020:0> (100.0*9.95).to_i
=> 994
irb(main):021:0> (100*9.95).to_i
=> 994
irb(main):022:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):023:0> RUBY_VERSION
=> "1.6.8"
irb(main):024:0> RUBY_PLATFORM
=> "powerpc-darwin7.0"

Dan

 

[Jamis Buck]

I can understand floating point errors, but this one seems too small for
that.
Especially given this:

irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994

a = 100 * 9.95
"%.16f" % a
=> "994.9999999999998863"

It's not a problem with Ruby, it's a problem with floating point numbers
in general.

- Jamis

 

[Walter Szewelanczyk]

I can understand floating point errors, but this one seems too small for
that.
Especially given this:

irb(main):023:0> v=100.0*9.95
=> 995
irb(main):024:0> v.to_i
=> 994


irb(main):016:0> (100.0*9.95).to_i
=> 994
irb(main):017:0> (100*9.95).to_i
=> 994
irb(main):018:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):019:0> RUBY_VERSION
=> "1.8.1"
irb(main):020:0> RUBY_PLATFORM
=> "sparc-solaris2.8"


irb(main):020:0> (100.0*9.95).to_i
=> 994
irb(main):021:0> (100*9.95).to_i
=> 994
irb(main):022:0> (100*9.95+0.0000000000001).to_i
=> 995
irb(main):023:0> RUBY_VERSION
=> "1.6.8"
irb(main):024:0> RUBY_PLATFORM
=> "powerpc-darwin7.0"

Dan

ri Float#to_i

------------------------------------------------------------- Float#to_i
flt.to_i => integer
flt.to_int => integer
flt.truncate => integer
------------------------------------------------------------------------
Returns _flt_ truncated to an +Integer+.

you may want to try

ri Float#round

------------------------------------------------------------ Float#round
flt.round => integer
------------------------------------------------------------------------
Rounds _flt_ to the nearest integer. Equivalent to:

def round

return floor(self+0.5) if self > 0.0

return ceil(self-0.5) if self < 0.0

return 0.0

end



1.5.round #=> 2

(-1.5).round #=> -2


Hope that helps,

Walt

--
Walter Szewelanczyk
IS Director
M.W. Sewall & CO. email : (e-mail address removed)
259 Front St. Phone : (207) 442-7994 x 128
Bath, ME 04530 Fax : (207) 443-6284

 

[Bill Atkins]

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

Bill

 

[David G. Andersen]

a = 100 * 9.95
"%.16f" % a
=> "994.9999999999998863"

It's not a problem with Ruby, it's a problem with floating point numbers
in general.

eep:~> perl
$f = 100.0 * 9.95;
printf("%d\n", $f);
994

if it really bothers you (and you don't want to use round),
there's an ugly solution:

require 'bigdecimal'
v = BigDecimal.new("9.95")
vone = v * 100
vone.to_i
=> 995

(but really, you just want to use round. ;-)

-dave

 

[Jamis Buck]

OK, that would make sense if v's value was 994.5, but it seems to be
995.0. Why should to_i round down like that?

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
as 995.0, rounding it.

- Jamis

 

[Dan Janowski]

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show
the exact value. Why shouldn't to_i benefit?

It seems to me that this type of intolerance would effectively make
to_i not only worthless but dangerous.

Dan

 

[Florian Gross]

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show the
exact value. Why shouldn't to_i benefit?

The value is even less exact. .to_s automatically rounds to 15 digits.
Internally a higher precision is used. (Though it is still not perfect.)

Basically Ruby 'blurs' the values on output so that you need not bother
about manually rounding them all the time.

Your problem is that you're using .to_i. That will give you 5.0 even for
values like 5.99999999999999.

I hope my explanation was of some help.

 

[Jason DiCioccio]

Hello!

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck

But it isn't 995.0. It is 994.99999999883 or something. IRB displays it
as 995.0, rounding it.

- Jamis

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever )? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

Thanks!
-JD-

 

[David G. Andersen]

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever )? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

You can't represent all floating point numbers accurately in
binary. The problem in this case isn't actually in the storage
of 9.95 or 995 in floating point; it comes in when you do the
multiplication. You might take a peek at

http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm

It has a nice overview of some of the pitfalls of floating point,
and the links go on from there...

-Dave

 

[Florian Gross]

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever )? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

Floating point numbers are expressed in base 2. 9.95 has an infinite
number of digits in base 2 representation just like the result of 1/3
has in base 10.

However floating point numbers clearly can't be stored with an infinite
amount of bits which is why the above gets slightly imprecise.

You can have a look at all the digits of a float like this:

irb(main):172:0> "%.32f" % 9.95
=> "9.94999999999999930000000000000000"

 

[Jamis Buck]

Hello!

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck



I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever )? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

http://mathworld.wolfram.com/RoundoffError.html

It's a problem caused by the fact that floating point numbers cannot be
represented exactly, and when you perform operations on those inexact
numbers, the different between the approximation and the true becomes
greater and greater.

- Jamis

 

[Volkard Henkel]

Hello Jason,

Hello!

--On Thursday, October 28, 2004 03:48:56 +0900 Jamis Buck

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever )? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

9.95 can not exactly put into a floeating point number with base 2.
with base 10 it's no problem.
9.95==9*10^0+9*10^-1+5*10^-2
but with bas 2 this will become a periaodic dual fraction.
so 9.95 is the problem. its internally in the computer slightly lower
than the real 9.95
multiplying by 100.0 (which can be expresses exactly) does not repair
the problem of 9.95

every language is plagued by this problem, if it uses the floating
point arithmetic given by the processor (and doing this is a good idea
because the fine speed). we as programmers have to know about this
problem and live with it. mostly its a good idea to ignore it.
sometimes we don't calc 100.0 Dollars * 9.95 Dollars but do 10000
Pence * 995 Pence to avoid the problem (because irregular
pence-vanishing with doubles is not amusing, but regular
pence-vanishing with intagers can be managed).

 

[Jason DiCioccio]

Thanks for your responses. I actually did hear about this a long time ago
I think, but I had forgotten the specifics. It's really not often that I
end up dealing with floating point numbers. Most things are done (for me)
with Integers it seems. I'd imagine if I were designing a game engine or
doing scientific work, that would not be the case .

Thanks again!
-JD-

--On Thursday, October 28, 2004 05:09:48 +0900 Volkard Henkel

 

[Hal Fulton]

I would agree if it were not for this:

$ ruby -e 'puts (100.0*9.95).to_s'
995
$ ruby -e 'puts (100.0*9.95).to_i.to_s'
994

Ruby seems to be able to manage the FP error rate since FP.to_s show the
exact value. Why shouldn't to_i benefit?

It seems to me that this type of intolerance would effectively make to_i
not only worthless but dangerous.

I sympathize, but I think the point is this:
- to_i doesn't round, it truncates;
- you have #round if you want to use it;
- to_s has to either round or truncate; rounding
seems logical
- therefore expr.to_s and expr.to_i.to_s may differ

It's a matter of opinion, but I think it's working properly.
Just another of hundreds of design issues Matz has faced.


Hal

 

[Hal Fulton]

I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
issue with floating point numbers is making this come out to
994.999999999983 (or wahtever )? If every language is plagued by this
problem, then I'd be curious to know the history behind it..

It's not history, it's math.

Here's the short explanation.

Remember repeating decimals, which you learned about in elementary school?
For example, 1/3 = 0.33333... can't be expressed in a finite number of
digits.

Likewise 0.95 can't be expressed in a finite number of (binary) digits.
When you convert it to binary, it repeats infinitely.


Hal

 

[Hal Fulton]

The problem in this case isn't actually in the storage
of 9.95 or 995 in floating point; it comes in when you do the
multiplication.

I'm not sure it's *just* in the multiplication... 0.95 can't
be stored exactly in binary, can it?

Hal

 

[trans. (T. Onoma)]

| > I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
| > issue with floating point numbers is making this come out to
| > 994.999999999983 (or wahtever )? If every language is plagued by this
| > problem, then I'd be curious to know the history behind it..
| >
| It's not history, it's math.
|
| Here's the short explanation.
|
| Remember repeating decimals, which you learned about in elementary school?
| For example, 1/3 = 0.33333... can't be expressed in a finite number of
| digits.

Not exactly, '1/3' is finite. Rational numbers can be expressed. Irrationals
cannot. But current standards do not attempt to deal with repeating decimals,
even though they could. The reason for this seems to be a matter of history
related to an ability to test inequalities quickly.

IEEE754 is now 20 years old, and is showing its age. I have actually been
thinking of working on a improved version myself. But there's many details to
deal with, so who knows... maybe.

T.

 

[Hal Fulton]

| > I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
| > issue with floating point numbers is making this come out to
| > 994.999999999983 (or wahtever )? If every language is plagued by this
| > problem, then I'd be curious to know the history behind it..
| >
| It's not history, it's math.
|
| Here's the short explanation.
|
| Remember repeating decimals, which you learned about in elementary school?
| For example, 1/3 = 0.33333... can't be expressed in a finite number of
| digits.

Not exactly, '1/3' is finite. Rational numbers can be expressed. Irrationals
cannot. But current standards do not attempt to deal with repeating decimals,
even though they could. The reason for this seems to be a matter of history
related to an ability to test inequalities quickly.

Expressed as a single number in either base 10 or 2, 0.333... requires an
infinite number of digits (which is all I said).

Knowing that it is rational, you could separately store the two numbers expressing
that ratio. But that is completely different from traditional floating-point storage,
and the simple traditional methods of (e.g.) addition and subtraction would not
work.

IEEE754 is now 20 years old, and is showing its age. I have actually been
thinking of working on a improved version myself. But there's many details to
deal with, so who knows... maybe.

Good idea, go for it.

Hal

 

[Jamis Buck]

| > I'm confused now.. 100.0 * 9.95 is clearly 995. So what exactly is the
| > issue with floating point numbers is making this come out to
| > 994.999999999983 (or wahtever )? If every language is plagued by this
| > problem, then I'd be curious to know the history behind it..
| >
| It's not history, it's math.
|
| Here's the short explanation.
|
| Remember repeating decimals, which you learned about in elementary school?
| For example, 1/3 = 0.33333... can't be expressed in a finite number of
| digits.

Not exactly, '1/3' is finite.

... That's not what I learned in school. How many decimal digits does it
take to express 1/3, then?

- Jamis