Decimals not equalling themselves (e.g. 0.2 = 0.2000000001)

[CNiall]

I am very new to Python (I started learning it just yesterday), but I
have encountered a problem.

I want to make a simple script that calculates the n-th root of a given
number (e.g. 4th root of 625--obviously five, but it's just an example
), and because there is no nth-root function in Python I will do this
with something like x**(1/n).

However, with some, but not all, decimals, they do not seem to 'equal
themselves'. This is probably a bad way of expressing what I mean, so
I'll give an example:0.33000000000000002

As you can see, the last two decimals are very slightly inaccurate.
However, it appears that when n in 1/n is a power of two, the decimal
does not get 'thrown off'. How might I make Python recognise 0.2 as 0.2
and not 0.20000000000000001?

This discrepancy is very minor, but it makes the whole n-th root
calculator inaccurate. :\

 

[Diez B. Roggisch]

I am very new to Python (I started learning it just yesterday), but I
have encountered a problem.

I want to make a simple script that calculates the n-th root of a given
number (e.g. 4th root of 625--obviously five, but it's just an example
), and because there is no nth-root function in Python I will do this
with something like x**(1/n).

However, with some, but not all, decimals, they do not seem to 'equal
themselves'. This is probably a bad way of expressing what I mean, so
I'll give an example:
0.33000000000000002

As you can see, the last two decimals are very slightly inaccurate.
However, it appears that when n in 1/n is a power of two, the decimal
does not get 'thrown off'. How might I make Python recognise 0.2 as 0.2
and not 0.20000000000000001?

This discrepancy is very minor, but it makes the whole n-th root
calculator inaccurate. :\

Welcome to the wonderful world of IEEE754. Just because other languages
shield you from the gory details they still are there. Python chose to
not do that, instead showing the rounding errors introduced and making
the developer decide how to deal with these.

http://pyfaq.infogami.com/why-are-floating-point-calculations-so-inaccurate

Diez

 

[Diez B. Roggisch]

I am very new to Python (I started learning it just yesterday), but I
have encountered a problem.

I want to make a simple script that calculates the n-th root of a given
number (e.g. 4th root of 625--obviously five, but it's just an example
), and because there is no nth-root function in Python I will do this
with something like x**(1/n).

There is - pow.
5


Diez

 

[Jorgen Grahn]

CNiall schrieb: ....
....

Welcome to the wonderful world of IEEE754. Just because other languages
shield you from the gory details they still are there. Python chose to
not do that, instead showing the rounding errors introduced and making
the developer decide how to deal with these.

Which other languages try to hide how floating-point numbers behave?

The correct way of dealing with this (you probably agree) is never to
expect infinite precision from floats, and design your code/algorithms
accordingly -- never use "if f1==f2:" and so on.

Floating-point is a tricky area. Lots of programmers (including me)
know too little about it.

/Jorgen

 

[Diez B. Roggisch]

Which other languages try to hide how floating-point numbers behave?

PHP and Java, amongst others. The implicitly apply a formatting when
producing a string-representation.

The correct way of dealing with this (you probably agree) is never to
expect infinite precision from floats, and design your code/algorithms
accordingly -- never use "if f1==f2:" and so on.

Floating-point is a tricky area. Lots of programmers (including me)
know too little about it.

It sure is and I don't know too much myself. But IMHO python does
something right when making the programmer aware of the problems that
can appear.

Diez

 

[Dan Bishop]

I am very new to Python (I started learning it just yesterday), but I
have encountered a problem.

I want to make a simple script that calculates the n-th root of a given
number (e.g. 4th root of 625--obviously five, but it's just an example
), and because there is no nth-root function in Python I will do this
with something like x**(1/n).

However, with some, but not all, decimals, they do not seem to 'equal
themselves'. This is probably a bad way of expressing what I mean, so
I'll give an example:
 >>> 0.5
0.5
 >>> 0.25
0.25
 >>> 0.125
0.125
 >>> 0.2
0.20000000000000001
 >>> 0.33
0.33000000000000002

As you can see, the last two decimals are very slightly inaccurate.
However, it appears that when n in 1/n is a power of two, the decimal
does not get 'thrown off'.

This is because your computer stores numbers in binary, and thus those
numbers can be represented exactly.

decimal 0.5 = binary 0.1
decimal 0.25 = binary 0.01
decimal 0.125 = binary 0.001

However, numbers that have "nice" representations in decimal are often
non-terminating in binary.

decimal 0.2 = binary 0.0011 0011 0011 0011 0011 0011...
decimal 0.33 = binary 0.01 01010001111010111000
01010001111010111000...

It's just like in decimal where 1/3 = 0.333333..., so if you're
limited to a finite number of digits, you get rounding error.
(Interesting that you picked 0.33 as one of your examples.)

How might I make Python recognise 0.2 as 0.2
and not 0.20000000000000001?

With floats, you can't. As far as your computer is concerned, there's
no such number as 0.2. The closest you can get with IEEE 754 double
precision is

binary 0.001100110011001100110011001100110011001100110011001101 (53
significant bits)
= decimal 0.200000000000000011102230246251565404236316680908203125
(exactly)

If absolutely necessary, you can use the decimal.Decimal("0.2").
However, I do *NOT* recommend always using decimal arithmetic, for the
following reasons:

(1) You still have to deal with round-off errors resulting from finite
precision. For example:
Decimal("1.999999999999999999999999999")

(2) There's nothing special about the number ten. We chose it as our
number base not because of any of its mathematical properties, but
simply because of the number of fingers we have. The designers of our
computers recognized this fact and chose to optimize the hardware for
base-two instead, therefore...

(3) Decimal arithmetic is at least an order of magnitude slower binary
arithmetic.

If your complaint is merely about all the "noise" digits displayed,
simply use a smaller number of significant digits.

'0.2'

This discrepancy is very minor, but it makes the whole n-th root
calculator inaccurate. :\

I think you may have unrealistic expectations for accuracy.

Unlike ints, which are used for counted (and therefore exact)
quantities, floats are typically used for measured (and therefore
inexact) quantities.

The error in approximating 0.2 by its double-precision approximation
is less than 56 parts per quintillion. There are very few physical
measurements that require that level of accuracy. Suppose you went to
the butcher's and ordered a pound of beef, and got only
0.9999999999999999444888487687 pound. Would you even notice? And if
you did, how exactly would you expect them to compensate you for the
less than a trillionth of a cent that you got shortchanged?

 

[Fuzzyman]

I am very new to Python (I started learning it just yesterday), but I
have encountered a problem.

I want to make a simple script that calculates the n-th root of a given
number (e.g. 4th root of 625--obviously five, but it's just an example
), and because there is no nth-root function in Python I will do this
with something like x**(1/n).

However, with some, but not all, decimals, they do not seem to 'equal
themselves'. This is probably a bad way of expressing what I mean, so
I'll give an example:
 >>> 0.5
0.5
 >>> 0.25
0.25
 >>> 0.125
0.125
 >>> 0.2
0.20000000000000001
 >>> 0.33
0.33000000000000002

As you can see, the last two decimals are very slightly inaccurate.
However, it appears that when n in 1/n is a power of two, the decimal
does not get 'thrown off'. How might I make Python recognise 0.2 as 0.2
and not 0.20000000000000001?

This discrepancy is very minor, but it makes the whole n-th root
calculator inaccurate. :\

You're using floating point numbers and not decimals. For the
precision of decimals use the Python standard library decimal module.

As others have noted, the accuracy issue with floating point numbers
is enshrined in their implementation at the platform level and has
nothing to do with Python.

Michael Foord

 

[Steven D'Aprano]

What are they teaching in computer science classes these days?

I don't know about these days, but 20-odd years ago there was no
discussion of floating point accuracy in the Comp Sci classes I did at
Melbourne Uni. I did a class in computational mathematics, run by the
maths department, and it discussed a lot of issues about accuracy in
float calculations. However, if they mentioned anything about e.g. 0.2
not being exactly representable in binary, I slept through it.

Maybe that's why I failed that class. *wry grin*

 

[Ken Starks]

I am very new to Python (I started learning it just yesterday), but I
have encountered a problem.

I want to make a simple script that calculates the n-th root of a given
number (e.g. 4th root of 625--obviously five, but it's just an example
), and because there is no nth-root function in Python I will do this
with something like x**(1/n).

However, with some, but not all, decimals, they do not seem to 'equal
themselves'. This is probably a bad way of expressing what I mean, so
I'll give an example:
0.33000000000000002

As you can see, the last two decimals are very slightly inaccurate.
However, it appears that when n in 1/n is a power of two, the decimal
does not get 'thrown off'. How might I make Python recognise 0.2 as 0.2
and not 0.20000000000000001?

This discrepancy is very minor, but it makes the whole n-th root
calculator inaccurate. :\

As everyone else has pointed out, this is likely to be a
'floating point' error. what they don't mention is that you
don't actually have to use floating point at all.

For example, some numbers, m, can be represented as the product of
rational powers of primes. Its not that easy to add and subtract
them, but multiplication, division, raising to the nth power and
taking the nth root, are all easy. (m and n positive integers).

There certainly are algorithms that will evaluate the 'fourth
root of 625' as precisely 5.

More generally, there are algorithms that will guarantee to return
either
+ the exact answer as a float (if it is one)
+ the nearest or second nearest float to the actual answer
depending on your choice of:
- round towards zero
- round away from zero
- round towards positive infinity
- round towards negative infinity
- round depending on parity of next digit

It is the representation of the numbers during the intermediate
stages that is critical. You trade-off speed against accuracy.

You may well find, if you do this, that the results returned
by built-in mathematical functions DO NOT return either the
nearest or the second nearest float to the actual answer. It
depends on the underlying C library that was used, and its
programmers' choices.

So your home-rolled 'nth power' function would add a little
something to the standard functionality of the language,
and writing it would add to your understanding of the
language too.

 

[Gabriel Genellina]

When I was an undergrad the only courses that dealt with FP
issues were classes on numerical analysis. I don't even know
if numerical analysis classes were required for CS majors. I
think most of us in that class were engineering majors.

And even if those topics were always covered in CS classes, a CS degree
isn't required to program in Python - fortunately.

 

[schinckel]

And even if those topics were always covered in CS classes, a CS degree  
isn't required to program in Python - fortunately.

I had a class today which dealt with Decimal <-> IEE754 conversion,
and
whilst 0.1 was an example that was converted, and a representation was
generated, no mention was made of the precision issue.

I'm hoping that it was just that we ran out of time, and the lecturer
will discuss it in detail next time.

Matt.

 

[Gabriel Genellina]

I had a class today which dealt with Decimal <-> IEE754 conversion,
and
whilst 0.1 was an example that was converted, and a representation was
generated, no mention was made of the precision issue.

I'm hoping that it was just that we ran out of time, and the lecturer
will discuss it in detail next time.

Presumably you encountered that 0.1 (decimal) = 0.000110011001100... (binary), and as you can see it has infinite periodic bits. IEEE754 stores only a finite number of bits for the mantissa, all the remaining (infinite) bits are dropped; a representation error is unavoidable.

 

[Dave]

However, with some, but not all, decimals, they do not seem to 'equal
themselves'.

Back in my days studying electrical engineering I was pointed to this
reference about floating point arithmetic - http://citeseer.ist.psu.edu/goldberg91what.html

HTH,
Dave