Floating point bug?

D

Dan Bishop

Surely you jest.

He's not joking at all.
Your example is exact to 28 digits. Your attempted
trick is to use a number that never ends (1/3=0.3333...).

It does end in base 3, 6, 9, 12, etc.

You have to remember that base-ten wasn't chosen because it has
mathematical advantages over other bases, but merely because people
counted on their fingers. In light of this fact, why is one-fifth
more deserving of an exact representation than one-third is?
 
C

Carl Banks

What do you mean by "practically unusable?"

So big that a fraction takes 10 minutes to reduce to simplest form and/
or your hard disk starts thrashing.

It can easily happen with seemingly innocuous calculations.

I heard similar arguments
made against big integers at one point ("Primitive types are usually big
enough, why risk performance?") but I fell in love with them when I
first saw them in Smalltalk, and I'm glad Python supports them natively.

It's not the same argument, though.

Repeated calculations don't make bignums too large to manage unless
they're growing it exponentially.

With rationals, the accumulating calculations usually makes the
denominator grow out of control (unless there's some mitigating
factor, like in Paul Rubin's example where there were only a ever few
denominators.)


Carl Banks
 
J

Jeff Schwab

Carl said:
So big that a fraction takes 10 minutes to reduce to simplest form and/
or your hard disk starts thrashing.

It can easily happen with seemingly innocuous calculations.



It's not the same argument, though.

Repeated calculations don't make bignums too large to manage unless
they're growing it exponentially.

With rationals, the accumulating calculations usually makes the
denominator grow out of control (unless there's some mitigating
factor, like in Paul Rubin's example where there were only a ever few
denominators.)

OK, thanks for explaining. It doesn't seem to me intuitively like
something that would be a problem, but I'm willing to take your word for it.
 
M

mensanator

So big that a fraction takes 10 minutes to reduce to simplest form and/
or your hard disk starts thrashing.

Out of curiosity, just how big is that?
I ask because I routinely use extremely big
ints and rationals and haven't seen that.

And I'm talking about numbers with, say,
50000 decimal digits up to numbers with
over a billion bits.
It can easily happen with seemingly innocuous calculations.

Where I've had problems is not the size
of the numbers in the repeated calculations
but the number of calculations, say, millions.
It's not the same argument, though.

Repeated calculations don't make bignums too large to manage unless
they're growing it exponentially.

My work grows exponentially.
With rationals, the accumulating calculations usually makes the
denominator grow out of control (unless there's some mitigating
factor, like in Paul Rubin's example where there were only a ever few
denominators.)

I don't see any problem there either.

It's entirely possible _I_ won't be able
to use the new Fraction data type. My work
has already left the built-in Python big
ints in the dust in favor of gmpy, so I
have no expectation that the new Fraction
type will be of any use to me.
 
C

Carl Banks

OK, thanks for explaining. It doesn't seem to me intuitively like
something that would be a problem, but I'm willing to take your word for it.

Consider what happens when you add two fractions:

1/2 + 1/5

To do that, you have to take the LCD of the denomintor, in this case
10, so you get

5/10 + 2/10 = 7/10

Now imagine that you're adding a lot of different numbers with a lot
of different bases. That LCD's going to be pretty big. To make
matters worse, imagine taking this number as the divisor in a later
calculation: a new denominator would appear (7). So you get
denominators that you didn't even input, which can make LCDs go
higher.

Any iteration with repeated divisions and additions can thus run the
denominators up. This sort of calculation is pretty common (examples:
compound interest, numerical integration).

The thing I don't like about rationals is that they give a false sense
of security. They are performing reasonably, and then you make a
slight change or some circumstance changes slightly and suddenly they
blow up.


Carl Banks
 
S

Steven D'Aprano

Consider what happens when you add two fractions:

1/2 + 1/5

To do that, you have to take the LCD of the denomintor, in this case 10,
so you get

5/10 + 2/10 = 7/10

Now imagine that you're adding a lot of different numbers with a lot of
different bases. That LCD's going to be pretty big.

It *will* be pretty big, or it *could be* pretty big?

The worst case comes from something like this:

a/2 + b/3 + c/4 + d/5 + e/6 + ...

where the denominator could be as high as n! (for n = the number of
fractions you're adding). Since n! gets large quickly, you don't want
that.

But a less-naive implementation shouldn't be that bad: the lowest common
denominator of a/2 + c/4 is 4, not 8. Still, multiplying all the
relatively-prime denominators will still get large quickly, just not
quite as quickly as n!.

Which is where a good fraction implementation should allow you to specify
the largest denominator you wish to see. After all, the difference
between:

975328755018/6827301285133

and

1/7

is less than 2e-13, or a relative error of approximately 0.0000000001%.
If you care about a difference of that magnitude, you're probably already
using numpy.

An interesting question would be, how large a denominator would you need
to beat the precision of floats? My back-of-the-envelope calculation
suggests that limiting your denominator to 4503599627370496 is enough to
give you a precision as good as floats:

# on my PC, the machine accuracy is 2.2204460492503131e-16
# this is the smallest float which, when added to 1.0,
# doesn't give 1.04503599627370496.0

The advantage is that while 1000.0+eps gives 1000.0 (which it shouldn't),
(1000*4503599627370496 + 1)/4503599627370496 is a perfectly reasonable
fraction to work with. Ugly to the human eye perhaps, but if your PC is
grinding to a halt on such a fraction, maybe you should take this as a
sign that 64K *isn't* enough memory for everybody.

*wink*


[snip]
The thing I don't like about rationals is that they give a false sense
of security. They are performing reasonably, and then you make a slight
change or some circumstance changes slightly and suddenly they blow up.

Or, to put it another way, rationals and floats both are dangerous, but
in different ways. The performance of rationals can fall drastically, and
floating point calculations can suddenly become inaccurate. You make your
choice and take your chances.
 
L

Lie

Consider what happens when you add two fractions:
1/2 + 1/5

To do that, you have to take the LCD of the denomintor, in this case
10, so you get

5/10 + 2/10 = 7/10

Now imagine that you're adding a lot of different numbers with a lot
of different bases. That LCD's going to be pretty big. To make
matters worse, imagine taking this number as the divisor in a later
calculation: a new denominator would appear (7). So you get
denominators that you didn't even input, which can make LCDs go
higher.

Any iteration with repeated divisions and additions can thus run the
denominators up. This sort of calculation is pretty common (examples:
compound interest, numerical integration).

Wrong. Addition and subtraction would only grow the denominator up to
a certain limit
Or, to put it another way, rationals and floats both are dangerous, but
in different ways. The performance of rationals can fall drastically, and
floating point calculations can suddenly become inaccurate. You make your
choice and take your chances.

When I mean safety, fraction is safe in calculation integrity safety,
it is always safe to do calculations in fraction (although at one time
the huge calculation might stall the system, the calculation will
always be correct all the time). But actually there is a way to ensure
performance safety in Fraction class, fraction might grow
uncontrollably only if the data is multiplied or divided. If you're
just doing addition and subtraction, the denominator growth is limited
to a certain range given a limited range of data.
 
S

Steve Holden

Lie said:
Wrong. Addition and subtraction would only grow the denominator up to
a certain limit


When I mean safety, fraction is safe in calculation integrity safety,
it is always safe to do calculations in fraction (although at one time
the huge calculation might stall the system, the calculation will
always be correct all the time). But actually there is a way to ensure
performance safety in Fraction class, fraction might grow
uncontrollably only if the data is multiplied or divided. If you're
just doing addition and subtraction, the denominator growth is limited
to a certain range given a limited range of data.

Surely this assumes that the denominators are being reduced after each
operation, which would certainly add to the time the operations take.

The simplest and fasted implementation of rational addition and
subtraction uses a common denominator which is the product of both. To
avoid denominator growth at each operation you have to start extracting
common factors, which is bound to slow you down.

So, presuming your "certain limit" is the product of all mutually prime
denominators that have been involved, you are slowing things down to
maintain that limitation.

regards
Steve
 
C

Carl Banks

Wrong. Addition and subtraction would only grow the denominator up to
a certain limit

I said repeated additions and divisions.

Anyways, addition and subtraction can increase the denominator a lot
if for some reason you are inputing numbers with many different
denominators.


Carl Banks
 
L

Lie

I said repeated additions and divisions.

Repeated Addition and subtraction can't make fractions grow
infinitely, only multiplication and division could.
Anyways, addition and subtraction can increase the denominator a lot
if for some reason you are inputing numbers with many different
denominators.

Up to a certain limit. After you reached the limit, the fraction would
always be simplifyable.

If the input numerator and denominator have a defined limit, repeated
addition and subtraction to another fraction will also have a defined
limit.
 
S

Steve Holden

Lie said:
Repeated Addition and subtraction can't make fractions grow
infinitely, only multiplication and division could.
On what basis is this claim made?

(n1/d1) + (n2/d2) = ((n1*d2) + (n2*d1)) / (d1*d2)

If d1 and d2 are mutually prime (have no common factors) then it is
impossible to reduce the resulting fraction further in the general case
(where n1 = n2 = 1, for example).
Up to a certain limit. After you reached the limit, the fraction would
always be simplifyable.
Where does this magical "limit" appear from?
If the input numerator and denominator have a defined limit, repeated
addition and subtraction to another fraction will also have a defined
limit.

Well I suppose is you limit the input denominators to n then you have a
guarantee that the output denominators won't exceed n!, but that seems
like a pretty poor guarantee to me.

Am I wrong here? You seem to be putting out unsupportable assertions.
Please justify them or stop making them.

regards
Steve
 
L

Lie

On what basis is this claim made?

(n1/d1) + (n2/d2) = ((n1*d2) + (n2*d1)) / (d1*d2)

If d1 and d2 are mutually prime (have no common factors) then it is
impossible to reduce the resulting fraction further in the general case
(where n1 = n2 = 1, for example).



Where does this magical "limit" appear from?


Well I suppose is you limit the input denominators to n then you have a
guarantee that the output denominators won't exceed n!, but that seems
like a pretty poor guarantee to me.

Am I wrong here? You seem to be putting out unsupportable assertions.
Please justify them or stop making them.

Well, I do a test on my own fraction class. I found out that if we set
a limit to the numerators and denominators, the resulting output
fraction would have limit too. I can't grow my fraction any more than
this limit no matter how many iteration I do on them. I do the test is
by something like this (I don't have the source code with me right
now, it's quite long if it includes the fraction class, but I think
you could use any fraction class that automatically simplify itself,
might post the real code some time later):

while True:
a = randomly do (a + b) or (a - b)
b = random fraction between [0-100]/[0-100]
print a

And this limit is much lower than n!. I think it's sum(primes(n)), but
I've got no proof for this one yet.
 
M

Mark Dickinson

And this limit is much lower than n!. I think it's sum(primes(n)), but
I've got no proof for this one yet.

It's the least common multiple of the integers 1 through n, or
equivalently the product over all primes p <= n of the highest power
of p not exceeding n. So for n = 100, it's:

64 * 81 * 25 * 49 * 11 * 13 * 17 * ... rest of primes up to 100.

For general n, this number is of roughly the same order of magnitude
as e**n.

See

http://www.research.att.com/~njas/sequences/A003418

for more.

Mark
 
L

Lie

It's the least common multiple of the integers 1 through n, or
equivalently the product over all primes p <= n of the highest power
of p not exceeding n. So for n = 100, it's:

64 * 81 * 25 * 49 * 11 * 13 * 17 * ... rest of primes up to 100.

For general n, this number is of roughly the same order of magnitude
as e**n.

Ah, yes, I meant product(primes(n)), please forgive my honest mistake
which is partially caused by me not supposed to still be awake at this
time of the day. And thanks for Mark for noticing the mistake, and
here is the code I used:

import fraction
import random

frac = fraction.frac
ops = (frac.__add__, frac.__sub__)

a = frac(random.randrange(1, 10), random.randrange(1, 10))
b = frac(random.randrange(1, 10), random.randrange(1, 10))

while True:
o = ops[random.randrange(0, 2)]
a = o(a, b)
b = frac(random.randrange(1, 10), random.randrange(1, 10))
print a

I decided to keep the num/den limit low (10) because higher values
might obscure the fact that it do have limits. And through further
observations, I think it is sufficient if the limit is imposed in the
denominator only (numerator can have any values it wanted to,
denominator growth is determined only by the limit of denominator
growth).

I think I'll also post the code for the fraction class I used, if you
have other fraction class that can automatically simplify, you could
use that instead as this class suffers from a naive GCD
implementation:

==== fraction.py ====
from __future__ import division


def GCD(a, b):
if b == 0: return a
return GCD(b, a % b)


class frac:

''' Fraction Class



A fraction class.



Attributes:

num -> the numerator of a fraction

den -> the denominator of a fraction



Methods:

add(a, b) -> add fraction a to fraction b and return a
new fraction

sub(a, b) -> subtract fraction b from fraction a and
return a new fraction

mul(a, b) -> multiply fraction a with fraction b and
return a new fraction

div(a, b) -> divides fraction b from fraction a and
return a new fraction

invert(a) -> invert the fraction (switch the numerator
and denominator)

neg(a) -> negates the fraction (negates numerator)

powr(a, b) -> raise fraction a to the power of b

simplify(a, b) -> simplify fraction to its canonical
representation



__init__(a, b) -> creates a new fraction

__str__(a, b) -> returns a string representation. Mixed
fraction if possible

__repr__(a, b) -> returns a string representation. Always
return vulgar fraction



Operators:

Conversion to fractions will be done automatically whenever
possible, in-place

operation is fully supported. Both regular and reflected
operation is supported.

a + b -> Calls frac.add(a, b)

a - b -> Calls frac.sub(a, b)

a * b -> Calls frac.mul(a, b)

a / b -> Calls frac.div(a, b)

a // b -> Floors the resulting fraction from frac.div(a, b)

-a -> Negates a fraction

+a -> Returns a copy of the fraction

~a -> Return the reciprocal of a



Comparisons:

Implemented through __cmp__(a, b), no rich comparison.

a == b -> a equals b

a != b -> a not equal b

a > b -> a greater than b

a < b -> a less than b

a >= b -> a greater than or equal to b

a <= b -> a less than or equal to b



Casting:

__complex__ -> Converts the fraction to floats and return the
result as complex number

__int__ -> Returns the whole part of the fraction in
Integer

__long__ -> Returns the whole part of the fraction in Long

__float__ -> Returns the value of the fractino in Float



Exceptions:

ZeroDenominatorError

-> A fraction cannot have zero as its denominator



Bugs:

- At the meantime, the fraction class doesn't mix well if used

together with floating type numbers. Inline operators and

initializers implicitly assumed that numbers are either
integers or

fraction. So if there are operations involving floating
point or

you initialized a fraction with a floating point number, the
result

would be something like a "floating point fraction".

'''



def __init__(self, a = 0, b = 1):

dev = GCD(a, b)

if b > 0:

self.num = a // dev

elif b < 0:

self.num = -a // dev

else:

raise frac.ZeroDenominatorError



self.den = abs(b) // dev



def simplify(self):

dev = GCD(self.num, self.den)

self.num //= dev

self.den //= dev

return self



def add(a, b):

return frac(a.num * b.den + b.num * a.den, a.den * b.den)

def sub(a, b):

return frac(a.num * b.den - b.num * a.den, a.den * b.den)

def mul(a, b):

return frac(a.num * b.num, a.den * b.den)

def div(a, b):

return frac(a.num * b.den, a.den * b.num)

def invert(a):

return frac(a.den, a.num)

def neg(a):

return frac(-a.num, a.den)

def powr(x, y):

return frac(x.num ** y, x.den ** y)



def __add__(self, other):

return self.add(other)

def __radd__(self, other):

return other.add(self)



def __sub__(self, other):

return self.sub(other)

def __rsub__(self, other):

return other.sub(self)



def __mul__(self, other):

return self.mul(other)

def __rmul__(self, other):

return other.mul(self)



def __div__(self, other):

return self.div(other)

def __rdiv__(self, other):

return other.div(self)



def __truediv__(self, other):

return self.div(other)

def __rtruediv__(self, other):

return other.div(self)



def __floordiv__(self, other):

ret = self.div(other)

return ret.num // ret.den

def __rfloordiv__(self, other):

ret = other.div(self)

return ret.num // ret.den



def __iadd__(a, b):

a.num, a.den = a.num * b.den + b.num * a.den, a.den * b.den

return a.simplify()

def __isub__(a, b):

a.num, a.den = a.num * b.den - b.num * a.den, a.den * b.den

return a.simplify()

def __imul__(a, b):

a.num, a.den = a.num * b.num, a.den * b.den

return a.simplify()

def __idiv__(a, b):

a.num, a.den = a.num * b.den, a.den * b.num

return a.simplify()

def __itruediv__(a, b):

a.num, a.den = a.num * b.den, a.den * b.num

return a.simplify()

def __ifloordiv__(self, other):

self /= other

return self.num // self.den



def __str__(self):
''' String Function

Convert the function to its human-friendly representation.
Tries
to convert smartly.
'''

ret = ''

if self.num < 0: ret = '-'

w, n, d = abs(self.num // self.den), abs(self.num) % self.den,
self.den

if w != 0:

if n != 0 and d != 1:

ret += '%s %s/%s' % (w, n, d)

else:

ret += str(w)

else:

if n != 0 and d != 1:

ret += '%s/%s' % (n, d)

else:

ret = '0'

return ret



def __repr__(self):

return str(self.num) + '/' + str(self.den)



def __complex__(self):

return complex(float(self))

def __int__(self):

return int(self.num / self.den)

def __long__(self):

return long(self.num / self.den)

def __float__(self):

return self.num / self.den



def __neg__(self):

return frac.neg(self)

def __pos__(self):

return frac(self.num, self.den)

def __abs__(self):

return frac(abs(self.num), self.den)

def __invert__(self):

return frac.invert(self)

def __pow__(x, y):

return powr(x, y)



def __coerce__(self, other):

try:

self.num, self.den

except AttributeError:

self = frac(self)

try:

other.num, other.den

except AttributeError:

other = frac(other)

return self, other



def __cmp__(self, other):

a = self.num * other.den

b = other.num * self.den

return cmp(a, b)



class ZeroDenominatorError(Exception):

''' Exception for having a zero as the denominator in a
Fraction Class

'''

def __init__(self):

pass

def __str__(self):

return "A fraction cannot have zero as denominator
(frac.den != 0)"
 
M

Mensanator

It's the least common multiple of the integers 1 through n, or
equivalently the product over all primes p <= n of the highest power
of p not exceeding n. �So for n = 100, it's:
64 * 81 * 25 * 49 * 11 * 13 * 17 * ... rest of primes up to 100.
For general n, this number is of roughly the same order of magnitude
as e**n.

Ah, yes, I meant product(primes(n)), please forgive my honest mistake
which is partially caused by me not supposed to still be awake at this
time of the day. And thanks for Mark for noticing the mistake, and
here is the code I used:

import fraction
import random

frac = fraction.frac
ops = (frac.__add__, frac.__sub__)

a = frac(random.randrange(1, 10), random.randrange(1, 10))
b = frac(random.randrange(1, 10), random.randrange(1, 10))

while True:
� � o = ops[random.randrange(0, 2)]
� � a = o(a, b)
� � b = frac(random.randrange(1, 10), random.randrange(1, 10))
� � print a

I decided to keep the num/den limit low (10) because higher values
might obscure the fact that it do have limits. And through further
observations, I think it is sufficient if the limit is imposed in the
denominator only (numerator can have any values it wanted to,
denominator growth is determined only by the limit of denominator
growth).

I think I'll also post the code for the fraction class I used, if you
have other fraction class that can automatically simplify, you could
use that instead as this class suffers from a naive GCD
implementation:

Out of curiosity, of what use is denominator limits?

The problems where I've had to use rationals have
never afforded me such luxury, so I don't see what
your point is.
 
C

casevh

Out of curiosity, of what use is denominator limits?

The problems where I've had to use rationals have
never afforded me such luxury, so I don't see what
your point is

In Donald Knuth's The Art of Computer Programming, he described
floating slash arithmetic where the total number of bits by the
numerator and denominator was bounded. IIRC, a use case was matrix
inversion.

casevh
 
S

Steven D'Aprano

On what basis is this claim made?

(n1/d1) + (n2/d2) = ((n1*d2) + (n2*d1)) / (d1*d2)

If d1 and d2 are mutually prime (have no common factors) then it is
impossible to reduce the resulting fraction further in the general case
(where n1 = n2 = 1, for example).



Where does this magical "limit" appear from?


Well I suppose is you limit the input denominators to n then you have a
guarantee that the output denominators won't exceed n!, but that seems
like a pretty poor guarantee to me.

Am I wrong here? You seem to be putting out unsupportable assertions.
Please justify them or stop making them.
Well, I do a test on my own fraction class. I found out that if we set a
limit to the numerators and denominators, the resulting output fraction
would have limit too. I can't grow my fraction any more than this limit
no matter how many iteration I do on them. I do the test is by something
like this (I don't have the source code with me right now, it's quite
long if it includes the fraction class, but I think you could use any
fraction class that automatically simplify itself, might post the real
code some time later):

while True:
a = randomly do (a + b) or (a - b)
b = random fraction between [0-100]/[0-100] print a

And this limit is much lower than n!. I think it's sum(primes(n)), but
I've got no proof for this one yet.


*jaw drops*


Please stop trying to "help" convince people that rational classes are
safe to use. That's the sort of "help" that we don't need.


For the record, it is a perfectly good strategy to *artificially* limit
the denominator of fractions to some maximum value. (That's more or less
the equivalent of setting your floating point values to a maximum number
of decimal places.) But without that artificial limit, repeated addition
of fractions risks having the denominator increase without limit.
 
S

Steven D'Aprano

Out of curiosity, of what use is denominator limits?

The problems where I've had to use rationals have never afforded me such
luxury, so I don't see what your point is.

It ensures that your fraction's denominator doesn't grow indefinitely, at
the cost of some precision. In principle, fraction denominators can grow
exponentially. In practice, probably less quickly, but still quickly
enough that people on this list have reported that adding two fractions
lead to millions of digits in each denominator and massive paging as
their computer struggled to perform the addition.

The beauty of fractions is that they give you infinite precision. The
danger of fractions is that it takes a lot of memory to store infinitely
precise numbers :)

Frankly, I think that for most real-world data, it's unlikely to be a
problem, but Guido's experiences with ABC were the opposite. But then we
don't know how naive the ABC fraction libraries were. For all I know they
did this:

1/2 + 1/2 = 4/4
4/4 - 1/2 = 4/8
4/8 + 1/2 = 16/16
etc.
 
S

Steven D'Aprano

I decided to keep the num/den limit low (10) because higher values might
obscure the fact that it do have limits.

You do realise that by putting limits on the denominator, you guarantee
that the sum of the fractions also has a limit on the denominator? In
other words, your "test" is useless.

With denominators limited to 1 through 9 inclusive, the sum will have a
denominator of 2*3*5*7 = 210. But that limit is a product (literally and
figuratively) of your artificial limit on the denominator. Add a fraction
with denominator 11, and the sum now has a denominator of 2310; add
another fraction n/13 and the sum goes to m/30030; and so on.
 
M

Mensanator

You do realise that by putting limits on the denominator, you guarantee
that the sum of the fractions also has a limit on the denominator? In
other words, your "test" is useless.

With denominators limited to 1 through 9 inclusive, the sum will have a
denominator of 2*3*5*7 = 210.

Th limit will be 2*2*2*3*3*5*7. As MD said, "equivalently
the product over all primes p <= n of the highest power
of p not exceeding n".
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,262
Messages
2,571,049
Members
48,769
Latest member
Clifft

Latest Threads

Top