Float comparison

[CBFalconer]

You must be using words with meanings other than what everyone else is
using.

What is there is *exactly* what has been stored there, by definition.
C99 6.2.4p2:

Yes, but that is not necessarily what was offered to be stored. For
example, use double to compute:

double x = 1.0 + (FLT_EPSILON/2.0)

which is calculable, and is not equal to the double 1.0. Now store
the result in a float.

float y = x;

Now try to extract something other than 1.0 from y.

I haven't read it for many years, but there is a whole section in
Knuth about floating point errors, and keeping track.

 

[CBFalconer]

and if a number can be stored exactly why not treat it is if it
were exact? On some implementations you can get an integer type
with a greater range than int (or long) by using floating point
types to store integer values. And they are exact.

Here you are using knowledge outside of the FP value. You know you
always stored integers, and so (within maximum value) you can get
those intgers back. But there are infinitely many other values
that each quasi integer could represent, without that extraneous
processing knowledge.

 

[Ben Bacarisse]

That's fine, except for the correct/incorrect classification, which
does not apply.

double a, b, c;

a = 1.0; b = a + DBL_EPSILON/2; c = 0.0;
/* randomize rand */
if ((rand() & 1)) {
c = a; a = b; b = c;
c = 0.0;
}

now ignore the processing, and just examine the values in a and b.
I challenge you to tell them apart. Apply any necessary includes,
etc.

What is it about this exampled this depends on the variables having
floating point types? You could make the same remarks, and draw the
same conclusions, about int and _Bool. Do C variables of any type
hold exact values in your view of the world?

 

[CBFalconer]

.... snip ...

I know of code which carefully uses doubles to do integer
arithmetic, staying within the range where integers can be
represented exactly, in order to use the greater range of exact
integral values that a double can represent on that specific
implementation than an int. In this case the stored doubles
always have an exact number of pennies in them (well, it also
uses exact numbers of other things).

But now you are using extraneous information, beyond that held in
the FP value. You are using the fact that you know than only
integers have ever been stored. This is not intrinsic to the FP
value.

 

[Keith Thompson]

Yes, but that is not necessarily what was offered to be stored.

[...]

Are you backing off from your claim "what has been stored", and
replacing it with a claim about "what was offered to be stored"? And
what does "offered to be stored" mean? I don't believe that concept,
or anything resembling it, appears in the standard.

Here's the bottom line. You have asserted repeatedly that a stored
floating-point value (such as a value stored in an object of type
double) has a particular meaning, namely a range of values that
includes the exact stored value.

I assert that that's one possible interpretation, and that the
interpretation that it represents just the exact stored value is
equally valid. (In the latter interpretation, the mathematical value
1.0/3.0, for example, cannot be stored, only some value close to it.)

After being repeatedly asked to do so, you have consistently failed to
produce a citation from the standard that either supports your
interpretation or refutes mine. Why is that?

 

[Ben Bacarisse]

CBFalconer <[email protected]> writes:

What is it about this exampled this depends on the variables having

 

[CBFalconer]

.... snip ...

After being repeatedly asked to do so, you have consistently
failed to produce a citation from the standard that either
supports your interpretation or refutes mine. Why is that?

But I have. I referred you to a section about float.h.

 

[Keith Thompson]

But I have. I referred you to a section about float.h.

I see nothing in 5.2.4.2.2 that supports your claim that a stored
floating-point value represents a range of mathematical real values
rather than one specific mathematical real value. In particular,
see 5.2.4.2.2p2:

A floating-point number (x) is defined by the following model:

x = s b^e summation(k=1, p) f(sub k) b^-k, emin <= e <= emax

That's a clumsy rendition of the mathematical notation used in the
standard; you'll need to use a PDF (or hardcopy) version to see it.

The model defines a single exact value, not a range of values, for any
floating-point number.

 

[Flash Gordon]

Flash Gordon wrote:
.... snip ...

But now you are using extraneous information, beyond that held in
the FP value. You are using the fact that you know than only
integers have ever been stored. This is not intrinsic to the FP
value.

Variables do NOT exist in isolation, not do values. They exist within
programs.

You have snipped without addressing the following points:

1) By EXACTLY the same argument all integer types represent a range of
values (I gave examples of real situations where this applies)

2) The only way to determine the range of values represented, whether it
is stored as an integer or floating point, IS to use the additional
information and do an error analysis.

I.e. WITHOUT performing an error analysis all you can do is say that a
variable may or may not represent a number somewhere on the complex
plain and the error may be infinite.

I.e. the definition of the floating point model in C tels you NOTHING
about the range of numbers that any variable might represent.

Another example

double x = sin(11);

Does the foating point model tell you what value x will contain?
Given the value of x without knowing that it is the result of the sin
function, can you use the floating point model (and NOTHING else) to
derive a range that includes the mathematical value of sin(11) ?

Just to give you a clue, the answer to BOTH questions is NO. If you
claims were true the answer to both questions would be yes.

 

[James Kuyper]

christian.bau wrote:
....

With the model that representation = real number, the analysis is
quite simple. For example, when using IEEE 754 arithmetic, you perform
the arithmetic with real numbers giving the exact real result, then
apply a simple rounding algorithm that produces a real number that has
a representation, and that is the representation of the result.

There is, in general, no way to "perform the arithmetic ... giving the
exact real result", and as a result there's no rounding step, either. In
general, what you have to do is implement an algorithm that inherently
produces only an a approximation to the exact result, and what you have
to do to conform to IEEE 754 is design the algorithm to ensure that the
approximation is sufficiently good.

 

[Keith Thompson]

christian.bau wrote:
...

There is, in general, no way to "perform the arithmetic ... giving the
exact real result", and as a result there's no rounding step,
either. In general, what you have to do is implement an algorithm that
inherently produces only an a approximation to the exact result, and
what you have to do to conform to IEEE 754 is design the algorithm to
ensure that the approximation is sufficiently good.

Actually, it is possible to "perform the arithmetic ... giving the
exact real result", though it can be resource-intensive. All
mathematical expressions expressible in C yield rational numbers. To
get exact real results corresponding to those expressions, you can use
a representation consisting of two arbitrary-precision integers. This
doesn't work for expressions involving functions that yield irrational
results (sqrt, sin, et al), but it can work for everything involving
literals and built-in operators.

 

[Richard Bos]

As far as I can tell, that's exactly accurate. But as soon as you do
any floating point math on it, accuracy is implementation-defined...?
(5.2.4.2.2p5)

Implementation-defined, _but_ with certain minima. For example, adding x
to x must always give you 2.0. Adding x to 1.5e+20 is not guaranteed to
give you either of the two naively expected values (though it must give
you something close to them).

Richard

 

[Flash Gordon]

Keith Thompson wrote:

Actually, it is possible to "perform the arithmetic ... giving the
exact real result", though it can be resource-intensive. All
mathematical expressions expressible in C yield rational numbers. To
get exact real results corresponding to those expressions, you can use
a representation consisting of two arbitrary-precision integers. This
doesn't work for expressions involving functions that yield irrational
results (sqrt, sin, et al), but it can work for everything involving
literals and built-in operators.

In theory you can, in practice you can't. Try calculating Pi and you
will eventually run out of memory.

 

[Phil Carmody]

Any just who says what "sufficiently good" is. Me who knows what the problem
being solved is? The standard that hasn't a clue? Me needs to add the
reciprocal of a googolplex to a googolplex and have the representation different
than that for a googolplex. I'm so glad to know IEEE 754 will give me a number
that is "sufficiently good" for my purpose.

How about this, I'm looking for mersenne primes. Do you think the IEEE754
"sufficiently good" is in fact sufficiently good?

It is sufficiently good. Demonstrably so. Not provably so, but as long
as you accept a probability of error below certain bounds, and have
an ability to detect errors, you're laughing.

Phil

 

[BartC]

Huh? What do you mean. Seems to me that if you multiply two
'ranges' together you will get a result with a third 'range'. Do
the algebra.

I think he means: if the range of any floating point value is say 0 to 10%
(eg. 10.0 represents real values 10 to 11), then
multiplying 10 x 10 gives 100, which with a range of 10% gives 100 to 110.

However (10...11) x (10...11) has results in the range 100 to 121. 121 is
not in the range 100 to 110.

 

[James Kuyper]

Oddly enough, IEEE 754 is the source that tells you how good an
approximation is good enough to qualify as conforming to IEEE 754. I
realize that it might seem to you to be more appropriate to choose some
other authority, such as Article 3 of the US constitution, or maybe
Martha Stewart, but for some odd reason IEEE decided to be boringly
predictable instead.

Whether sufficiently good to qualify as conforming to IEEE 754 is good
enough for your purposes is something only you can be sure of. However,
IEEE 754's requirements on the accuracy of various operations are pretty
nearly as tight as it's reasonably possible for them to be. If it's
requirements aren't good enough for the task you're trying to perform,
you probably want to do something that can't be done in a fixed-sized
floating point format.

I wouldn't recommend using floating point math (which is what IEEE 754
is about) to do a search for prime numbers.

 

[CBFalconer]

I think he means: if the range of any floating point value is say 0 to 10%
(eg. 10.0 represents real values 10 to 11), then
multiplying 10 x 10 gives 100, which with a range of 10% gives 100 to 110.

However (10...11) x (10...11) has results in the range 100 to 121. 121 is
not in the range 100 to 110.

This is a chimera. The errors are small, compared with the
values. The sort of effect you are worrying about is of the form
error_ratio squared, and thus negligible.

 

[CBFalconer]

I see nothing in 5.2.4.2.2 that supports your claim that a stored
floating-point value represents a range of mathematical real
values rather than one specific mathematical real value. In
particular, see 5.2.4.2.2p2:

A floating-point number (x) is defined by the following model:

x = s b^e summation(k=1, p) f(sub k) b^-k, emin <= e <= emax

That's a clumsy rendition of the mathematical notation used in
the standard; you'll need to use a PDF (or hardcopy) version to
see it.

The model defines a single exact value, not a range of values,
for any floating-point number.

Repeat. In 5.2.4.2.2:

[#10] The values given in the following list shall be
replaced by implementation-defined constant expressions with
(positive) values that are less than or equal to those
shown:

-- the difference between 1 and the least value greater
than 1 that is representable in the given floating
point type, b1-p

FLT_EPSILON 1E-5
DBL_EPSILON 1E-9
LDBL_EPSILON 1E-9

Combine that with the method of expressing non-1.0 values.

 

[CBFalconer]

christian.bau wrote:
...

There is, in general, no way to "perform the arithmetic ...
giving the exact real result", and as a result there's no
rounding step, either. In general, what you have to do is
implement an algorithm that inherently produces only an a
approximation to the exact result, and what you have to do to
conform to IEEE 754 is design the algorithm to ensure that the
approximation is sufficiently good.

That's fair enough, but you are worrying about what you can
conclude from a program. I am worrying about what you can conclude
from a stored FP value, not about whether it was mis-calculated.
You need to worry about both, but the value range is a restriction
upon which you can't improve.

 

[CBFalconer]

.... snip ...

Another example

double x = sin(11);

Does the foating point model tell you what value x will contain?
Given the value of x without knowing that it is the result of the
sin function, can you use the floating point model (and NOTHING
else) to derive a range that includes the mathematical value of
sin(11) ?

Just to give you a clue, the answer to BOTH questions is NO. If
you claims were true the answer to both questions would be yes.

The critical point here is what does sin(11) return. If it returns
a double, then you can safely assume that x holds the value that
sin(11) will return in future (assuming sin is a single valued
function). If you assert that sin(11) returns the sin of an angle
of 11 degrees, there are two accuracies involved. One is the
accuracy of the sin function. It is pointless arranging an
algorithm that could return values more accurate than that of the
double (assuming it is returning a double).

I think the C restrictions are such that we can assume sin returns
a single valued function representing the sine of an angle TO SOME
ACCURACY.