Float comparison

[Ben Bacarisse]

It doesn't specify it because it is obvious. Please describe how
you would store the exact value of 1.0/3.0 in a floating point
format. Do you deny the usefulness of that quantity?

There is a world of difference between the correct statement "not
all values can be stored exactly" and the incorrect on "no stored
values are exact".

 

[Ben Bacarisse]

... snip ...

I maintain that your example is using 'other facts, such as
overall program structure'. I am talking about what the user
can get by examining the contents of the object x.

I'd say you had this exactly backwards. It is a fact that a
floating point "cell" (what's the word for thingy-that-holds-a-
floating-point-value?) holds an exact value. It's a certain
pattern of bits that doesn't change. In a certain context you
may choose to interprate this in one of (at least) two ways. It
can be an exact value (1.0 is exactly 1 [1]) or a range of
values. Without knowing how the program is using the floating
point variables you can't tell if it is meant to be exact or
not.

The point is that you don't KNOW what has been stored in that
object by simply examining the object.

What an odd statement. You know exactly what is stored in it by
simply looking. You often don't know what should be there, but you
certainly do know what is there.

You do KNOW the range of
values it can represent.

You don't even know that if I grasp what you mean by "represent".
After a long calculation, if the answer is "1.000..." (all zeros) does
that mean the answer really is between 1-eps and 1+eps (the two
neighbouring floats)? Of course not. To know the range of values
this answer "represents" (I use quotes because that is not how I use
the word) you have to do a full error analysis of the algorithm.

 

[Keith Thompson]

... snip ...

I maintain that your example is using 'other facts, such as
overall program structure'. I am talking about what the user
can get by examining the contents of the object x.

I'd say you had this exactly backwards. It is a fact that a
floating point "cell" (what's the word for thingy-that-holds-a-
floating-point-value?) holds an exact value. It's a certain
pattern of bits that doesn't change. In a certain context you
may choose to interprate this in one of (at least) two ways. It
can be an exact value (1.0 is exactly 1 [1]) or a range of
values. Without knowing how the program is using the floating
point variables you can't tell if it is meant to be exact or
not.

The point is that you don't KNOW what has been stored in that
object by simply examining the object. You do KNOW the range of
values it can represent. Any further knowledge requires analyzing
the entire program that uses the object. That is probably a
hideously complicated project.

Yes, you *do* know what has been stored in an object by simply
examining the object. No analysis of the entire program is necessary.

Here's an example:

#include <stdio.h>
int main(void)
{
double x = 1.0 / 3.0;
printf("x = %.60f\n = %a\n", x, x);
return 0;
}

The output on my system is:

x = 0.333333333333333314829616256247390992939472198486328125000000
= 0x1.5555555555555p-2

I examined the object x and determined exactly what had been stored in
it, namely the double representation of the real value
0.333333333333333314829616256247390992939472198486328125, or,
equivalently, 0x1.5555555555555p-2 . (Note that the %a format is
C99-specific.)

You are *assuming* that the value of x must represent a range of
values. That's certainly one possible interpretation, but it's not
the only one. Please cite chapter and verse in the standard to
support this assumption.

I'll grant that, without analyzing the program, you can't necessarily
know what the stored value *means*. Which is exactly why you can't
assume that it represents a particular range of values.

[snip]

 

[Ben Bacarisse]

#include <float.h>
double a, b;

...
a = 1.0;
b = a + (DBL_EPSILON / 2)
...

Now a and b display identical values. They did not have identical
values stored in them.

This is true of int too, then, is it?

int a = 1;
int b = a + (1/2);

now both display 1 but they did not have identical values stored int
them? So in your view, neither floats not ints are exact. Both
"represent" a range that should include the actual intended "real
value"?

This sounds like the wrong way to look at both floats and ints.

 

[Keith Thompson]

Phil Carmody wrote: [...]

What is the difference between the "real value" of a FP variable,
and simply its value?

#include <float.h>
double a, b;

...
a = 1.0;
b = a + (DBL_EPSILON / 2)
...

Now a and b display identical values.

Very likely, yes, though I don't think it's guaranteed. I'll assume
that they're identical.

Well, actually, a and b don't *display* anything; you can display
their values using, for example, printf.

They did not have identical
values stored in them.

Of course they did. The value stored in a was the result of
valuating the expression
1.0
and that value is 1.0. The value stored in b was the result of
evaluating the expression
a + (DBL_EPSILON / 2)
and that value is also 1.0. The computation did not yield
the mathematically exact result of the addition, because the
mathematically exact result is not representable; instead, it
yielded the nearest representable value, namely 1.0. That value
was stored in b.

Two different computations, due to the limited precision of type
double, yielded exactly the same result. That identical result was
stored in two different objects.

Perhaps the program *tried* to store a value greater than 1.0 in b,
but it did not actually do so. This "real value" you're talking about
just doesn't exist.

[...]

 

[CBFalconer]

There is a world of difference between the correct statement "not
all values can be stored exactly" and the incorrect on "no stored
values are exact".

That's fine, except for the correct/incorrect classification, which
does not apply.

double a, b, c;

a = 1.0; b = a + DBL_EPSILON/2; c = 0.0;
/* randomize rand */
if ((rand() & 1)) {
c = a; a = b; b = c;
c = 0.0;
}

now ignore the processing, and just examine the values in a and b.
I challenge you to tell them apart. Apply any necessary includes,
etc.

 

[CBFalconer]

.... snip ...


What an odd statement. You know exactly what is stored in it by
simply looking. You often don't know what should be there, but
you certainly do know what is there.

Yes, you can find out what 'is' there. But that is not necessarily
what 'has been' stored there. All you know about the 'has been'
value is that it is within some range of the value that 'is' there.

 

[CBFalconer]

.... snip ...

Yes, you *do* know what has been stored in an object by simply
examining the object. No analysis of the entire program is
necessary.

Here's an example:

#include <stdio.h>
int main(void)
{
double x = 1.0 / 3.0;
printf("x = %.60f\n = %a\n", x, x);
return 0;
}

The output on my system is:

x = 0.333333333333333314829616256247390992939472198486328125000000
= 0x1.5555555555555p-2

I examined the object x and determined exactly what had been stored in
it, namely the double representation of the real value
0.333333333333333314829616256247390992939472198486328125, or,
equivalently, 0x1.5555555555555p-2 . (Note that the %a format is
C99-specific.)

A better output system would have cut off that train of digits
roughly at the first '1' in it. As it is, you don't know that 1.0
/ 3.0 was stored. It could equally well have been
1.0/3.000000000001 (for some number of zeroes). You know what IS
stored, not what WAS stored.

 

[luserXtrog]

It doesn't specify it because it is obvious.  Please describe how
you would store the exact value of 1.0/3.0 in a floating point
format.  Do you deny the usefulness of that quantity?

What about something like BCD but with base-60?
Ancient Sumeria already solved this one!

Of course, you'd be wasting 4 values from each 6bit digit...

 

[CBFalconer]

.... snip ...

Two different computations, due to the limited precision of type
double, yielded exactly the same result. That identical result
was stored in two different objects.

Perhaps the program *tried* to store a value greater than 1.0 in
b, but it did not actually do so. This "real value" you're
talking about just doesn't exist.

Oh, it exists, but not in that floating representation. And that
is how programs accumulate relatively large 'rounding errors'. You
have to keep track of what is going on. Watching every little
detail is excessive, and will not be done. Always remembering that
a floating point value represents a known range is very helpful.

 

[Flash Gordon]

Keith Thompson wrote:
.... snip ...

Oh, it exists, but not in that floating representation.

The program when running does not work with "real vaus" and then round
them to floating point representations. It works with floating point
representations only. So the "real value" never exists in the computer.

And that
is how programs accumulate relatively large 'rounding errors'.

Keith has already mentioned the need to do an error analysis if you want
to know what the possible error.

You
have to keep track of what is going on.

Earlier you said that you know what the range is *without* examining the
code. Which is it?

Watching every little
detail is excessive, and will not be done.

Wrong. Some people spend a long time doing exactly that because it is
very important and they are being paid to do it. It is called "Error
Analysis" and gets taught in universities. I've done done an approximate
error analysis on software that used integer arithmetic (for speed) in
the past, and in that program the integers where all storing approximate
values (altitude in units of 4 feet, the roll and pitch of an aircraft etc).

Always remembering that
a floating point value represents a known range is very helpful.

Apart from the fact that this is NOT true. You need to analyse th
program to know whether it is

I know of code which carefully uses doubles to do integer arithmetic,
staying within the range where integers can be represented exactly, in
order to use the greater range of exact integral values that a double
can represent on that specific implementation than an int. In this case
the stored doubles always have an exact number of pennies in them (well,
it also uses exact numbers of other things).

Now you have a description of real software in which floating point
variables always have exact values stored in them. I'm sure that over
the years I've seen other people mention similar things.

Everything you are saying applies to integer arithmetic in C as well,
because people DO use integers to hold approximate values (unless you
think planes fly at exact multiples of 4 feet and do things like jumping
from 80 feet to 84 feet without ever going through the intervening
altitudes).

So either integer types cannot store exact values and you need to do a
full error analysis to determine the range of values they might
represent or floating point variables can store exact values which are
exactly the mathematically correct values the program is intended to store.

 

[Keith Thompson]

Yes, you can find out what 'is' there. But that is not necessarily
what 'has been' stored there. All you know about the 'has been'
value is that it is within some range of the value that 'is' there.

You must be using words with meanings other than what everyone else is
using.

What is there is *exactly* what has been stored there, by definition.
C99 6.2.4p2:

An object exists, has a constant address, and retains its
last-stored value throughout its lifetime.

Given:

double x = 1.0 + DBL_EPSILON/2;

you seem to be assuming that the mathematical value 1.0 +
DBL_EPSILON/2 (slightly greater than 1.0, but not representable as a
double) is somehow more meaningful *with respect to the object x* than
the actual value, exactly 1.0, that's stored in x. In fact, that
mathematical value is never computed when the program executes, is
never stored in x, and does not exist in the program. The "+"
operator doesn't yield a mathematical real value; it yields a value of
type double and there is *no difference* between that value and 1.0.

 

[Guest]

and if a number can be stored exactly why not treat it is if
it were exact? On some implementations you can get an integer type
with a greater range than int (or long) by using floating point types
to store integer values. And they are exact.


I don't see how this can be wrong.

I don't have that message in front of me, and I am not going to go
to extensive efforts to dig it out.

a riddle inside a mystery wrapped in an enigma


this just seems plain wrong

Search for the chunk "EPSILON" in things dealing with float.h.

I've read most of the messages in this thread and I see nothing
to support your view. You've had planty of oportunity to pesent it.

why?


I won't dignify this foolishness with a reply.

you are making extraordinary claims about the behaviour of the C
language
it is therefor up to you to back it uo. As an enthusiast of the
standard
I'd expect you to leap at the chance to quote the standard.

 

[Guest]

Ans this statement means what exactly. I don't understand quantum
physics but don't feel the need to tell everyone.

I was hoping Phil or Keith would be motivated to explain what point
they were trying to make.

err, like the last one he listed?

No. Chuck was listing C answers. I was listing a conventional
mathematical
answer.

You don#t understand the difference between operations and values?
Wow. No wonder you don't understand what debuggers do.

I must not of been clear. [actually I *was* see later in post] I was
expressing surprise that anyone needed to be told there was a
difference
between opertations and values. I am quite clear about the
distinction.

why did you break the sentence there? It made my meaning less clear.

"Why would *anyone* need it explained to them that operations and
values are distict."

See? Quite clear what I meant. Your trollness is beginning to show...

Have you been drinking?

Have you ever programmed in anything other than C or one of its close
relatives? In some languages operations (functions) are a sort of
value.

 

[Bart van Ingen Schenau]

That's fine, except for the correct/incorrect classification, which
does not apply.

    double a, b, c;

    a = 1.0; b = a + DBL_EPSILON/2; c = 0.0;
    /* randomize rand */
    if ((rand() & 1)) {
       c = a; a = b; b = c;
       c = 0.0;
    }

now ignore the processing, and just examine the values in a and b.
I challenge you to tell them apart.  Apply any necessary includes,
etc.

Lets see if I understand your point by using a different type:

#define INT_EPSILON 1
int a, b, c;

a = 1; b = a + INT_EPSILON/2; c = 0;
/* randomize rand */
if ((rand() & 1)) {
c = a; a = b; b = c;
c = 0;
}

now ignore the processing, and just examine the values in a and b.
I challenge you to tell them apart. Apply any necessary includes,
etc.

Yes, I think I understand. Floating point numbers are no more precise
than integers.

Bart v Ingen Schenau

 

[James Kuyper]

The existence of the x_EPSILON values in float.h. Multiplied by
the actual stored value, they give the range.

Not exactly. The actual range can vary from value*x_EPSILON/FLT_RADIX to
value*x_EPSILON, depending upon the ratio between value and the nearest
power of FLT_RADIX. When value is an exact power of FLT_RADIX, both of
those expressions apply, the smaller one on the low side, the larger one
on the high side. The simplest way to determine the exact range is to
use the nextafter() function; if you're using C90 rather than C99, the
simplest way is a lot more complicated.

 

[Dik T. Winter]

> Keith Thompson wrote: ....
>
> The existence of the x_EPSILON values in float.h. Multiplied by
> the actual stored value, they give the range.

Is that so? In that case how is it possible that given two floats a and
b, that for some (mathematical) values a' and b' in the ranges of a and b,
the mathematical value of a' * b' is not in the range of the C result a * b?

 

[Dik T. Winter]

>
> It doesn't specify it because it is obvious.

What does not specify what?

> Please describe how
> you would store the exact value of 1.0/3.0 in a floating point
> format.

Why do you think I would? IEEE specifies precisly what happens when you
divide 1.0 by 3.0, using *exact* values.

 

[Dik T. Winter]

> Oh, it exists, but not in that floating representation. And that
> is how programs accumulate relatively large 'rounding errors'. You
> have to keep track of what is going on.

And precisely that is where assuming that a float f represents a range
bounded by a multiple of FLT_EPSILON will *not* help.

Moreover, in some applications this forward error analysis will not give
any useful result, think about the solution of a linear system with say
100 unknowns. You have to apply backward error analysis to get anything
useful out of it, e.g. the calculated result is the *exact* result of the
original system changed by perturbation with in such bounds. With forward
error analysis the conclusion would be: the actual answer can be anywhere.

 

[CBFalconer]

Is that so? In that case how is it possible that given two
floats a and b, that for some (mathematical) values a' and b' in
the ranges of a and b, the mathematical value of a' * b' is not
in the range of the C result a * b?

Huh? What do you mean. Seems to me that if you multiply two
'ranges' together you will get a result with a third 'range'. Do
the algebra.