sys.float_info.epsilon

[Tim Rowe]

I'm reading Mark Summerfield's "Programming Python 3.0" at the moment,
and I'm puzzled by some of his uses of sys.float_info.epsilon. I
appreciate the issues of comparing floating point numbers, but I'm
puzzled by code like:
...
x = float(input(msg))
if abs(x) < sys.float_info.epsilon:
...

What could the float() conversion return that would give different results for:
if abs(x) < sys.float_info.epsilon
and (to my mind, more obvious):
if abs(x) == 0.0

I didn't realise that float() could return anything with an absolute
value less than sys.float_value.epsilon other than 0.0 (which I think
all representations can represent exactly). What am I missing here?

 

[Mark Dickinson]

I didn't realise that float() could return anything with an absolute
value less than sys.float_value.epsilon other than 0.0 (which I think
all representations can represent exactly).  What am I missing here?

There are many positive floating-point values smaller than
sys.float_info.epsilon.

sys.float_info.epsilon is defined as the difference between 1.0 and
the next largest representable floating-point number. On your system,
the next largest float is almost certainly 1 + 2**-52, so
sys.float_info.epsilon will be exactly 2**-52, which is around
2.2e-16. This number is a good guide to the relative error
from rounding that you can expect from a basic floating-point
operation.

The smallest positive floating-point number is *much* smaller:
again, unless you have a very unusual platform, it's going to
be 2**-1074, or around 4.9e-324. In between 2**-1074 and
2**-52 there are approximately 4.4 million million million
different floating-point numbers. Take your pick!

If you want a specific example, how about float('6.626e-34').

Mark

 

[Mark Dickinson]

You are missing the whole thing that mes floating point tricky.
I _believe_ that the epsilon is the smallest positive x such that
    1.0 != 1.0 + x

Nitpick alert: this isn't quite the same thing, since that
definition is affected by rounding. For example, if you're
using IEEE 754 doubles but (somehow) you've got a round-half-up
rounding mode then this makes epsilon 2**-53 rather than 2**-52,
because 1 + 2**-53 rounds to 1 + 2**-52 != 1.

(This confused the hell out of me many years ago when I was
reading Numerical Recipes, and trying to figure out why
on earth IEEE 754 and the VAX format had different epsilons
even though both had 53-bit mantissas. The answer is that
IEEE 754 uses round-half-even as standard, and VAX uses
round-half-up. Oh, and the authors of Numerical Recipes
used the 1 != 1+x definition...)

Python's float_info.epsilon comes straight from C's
DBL_EPSILON, which is defined very carefully in the C99 standard
as:

"""the difference between 1 and the least value greater than 1
that is representable in the given floating point type"""

(taking the 'given floating point type' to be double).

Mark

 

[Tim Rowe]

There are many positive floating-point values smaller than
sys.float_info.epsilon.

sys.float_info.epsilon is defined as the difference between 1.0 and
the next largest representable floating-point number. On your system,
the next largest float is almost certainly 1 + 2**-52, so
sys.float_info.epsilon will be exactly 2**-52, which is around
2.2e-16. This number is a good guide to the relative error
from rounding that you can expect from a basic floating-point
operation.

The smallest positive floating-point number is *much* smaller:
again, unless you have a very unusual platform, it's going to
be 2**-1074, or around 4.9e-324. In between 2**-1074 and
2**-52 there are approximately 4.4 million million million
different floating-point numbers. Take your pick!

Ok, that makes a lot of sense, thanks. I was thinking in terms of
Ada's floating point delta (rather than epsilon), which as I remember
it, if specified, applies uniformly across the whole floating point
range, not just to a particular point on the scale.

That just leaves me puzzled as to why Mark Summerfield used it instead
of a check against zero on user input. There's a later division by
2*x, so small values of x matter; there's no protection against
overflow (the numerator could perfectly well be somewhere up near
sys.float_info.max; a quick check in the shell tells me that Python3
will happily do sys.float_info.max/(2*sys.float_info.epsilon) and will
give me the answer "inf") so presumably he's trying to protect against
divide by zero. So my next question is whether there is any x that can
be returned by float() such that x != 0 but some_number / (2 * x)
raises a ZeroDivisionError?

 

[Tim Rowe]

2009/2/4 Scott David Daniels <[email protected]>:

Thanks for that. It makes me feel guilty to point out that:

addition is not associative in real numbers

should presumably be "addition is not associative in floating point numbers".

 

[Tim Rowe]

And, of course he is right (and didn't even whomp on my typo of "makes"
as "mes in the first line quoted above).

A typo for "makes" didn't bother me. Non-associativity of the real
numbers under addition risked making my whole world fall apart

 

[Mark Dickinson]

That just leaves me puzzled as to why Mark Summerfield used it instead
of a check against zero on user input.

No idea: you'd have to ask Mark Summerfield. If there's
an email address published in his book, I'm sure he
wouldn't object to the question.

So my next question is whether there is any x that can
be returned by float() such that x != 0 but some_number / (2 * x)
raises a ZeroDivisionError?

Nope. If x is nonzero, then 2*x is definitely nonzero.
It could be an infinity, or a nan, or the '2*x'
computation could raise an exception, or perhaps
cause the interpreter to crash (it shouldn't, but
you never know...), but it's never going to be zero.

Well, okay, *never* is a strong word: if 2*x is
subnormal, and you're operating on a platform that
for whatever reasons flushes subnormal results
to zero, then it could happen. But not in *real*
life.

Mark