Float comparison

[CBFalconer]

CBFalconer said:


Is the interval actually defined for anything other than 1.0?

Yes, for each actual FP implementation. For most, the range of
x*(1-EPSILON) through x*(1+EPSILON) applies to x values from 1.0 to
anything less than 2.0. Then you use EPSILON*2 for values from 2.0
through less than 4.0. Etc. Note that the range above
automatically incorporates the right EPSILON.

Most systems are always normalized, because that frees up the most
significant bit in the significand to supply the sign. I am
ignoring denormalized values, which are too small to be expressed
with the same precision, NaNs, and INFx. I am also ignoring the
end points.

 

[CBFalconer]

.... snip ...

Revise your picture slightly.

|----A----|----B----|----C----|
<---1---> <---2---> <---3--->

Anything in the interval 1 will be recorded as the value A,
interval 2 as value B, interval 3 as value C. 1, 2, and 3 are the
gazintas. The comesoutas are A, B, or C. Note that the value
(e.g. A) does not have to lie in the middle of the interval (eg 1).

Right, but I've been discussing the result itself, not what led to it.
It could just as well have been produced by treating a double object
as an array of unsigned char and using system-specific knowledge to
construct the desired representation.


I don't think CBF actually understands this.

No, I don't think you understand that anything in the interval 1
above (which I have been referring to with a different name, i.e.
range) will be stored as A. That cannot be told apart from
anything else in the interval 1. This has nothing to do with the
programming, just with the storage.

Floating-point operations generally yield approximate values; I think
everyone understands this.

I am NOT talking about the operations. I am talking about what the
FP object represents.

 

[CBFalconer]

.... snip ...

Let's get a definitive answer, Chuck! The statement: regardless
of what it "means", how it was arrived at, and what unrepresentable
intermediate values were computed, a value as stored is stored
exactly, as provable with the == operator.

See my earlier answer. I think half the problem is people are
talking about different things with the same words, and using
different words for the same things.

 

[CBFalconer]

.... snip ...

That doesn't quite cover it. C's floating-point model describes
the meaning of a floating-point value as a mathematical real
value. The "==" operator doesn't operate on mathematical real
values. Chuck could consistently answer "yes" to your question
while continuing to maintain that a stored floating-point value
denotes a range of real values; presumably "==" would yield a
true result if both operands denote the same real range.

Quite true. I said exactly that aa while ago.

 

[Keith Thompson]

That's ok--my bad reading comprehension actually caused me to misread
these, so they were canceled out.

Then I hope my correction wasn't canceled out by yet another
misreading. }

 

[Keith Thompson]

Well... you know what I meant.

Actually, I'm not entirely sure that I do.

But with limits, most notably p and e.

Those limits simply mean that there are plenty of real values that
cannot be represented as floating-point values. But each
floating-point value (ignoring, as always, infinities and NaNs)
corresponds to a single well-defined real value. I believe my
statement above ("C's floating-point model describes ...") is correct
either with or without the qualification you added.

Sure, but are there numbers the model describes that aren't exactly
representable on a given implementation*? Numbers like one-third (base
10), for instance, the model only approximates out to a given precision,
and that approximation I believe should be exactly storable in a base 10
system of that precision.

* for that implementation's particular parameters to the model

But I'd welcome discussion on this point.

I believe the answer to your question is no.

I'm not sure what you mean by "one-third (base 10)". Real numbers
don't have bases; a base is an attribute of a way of representing a
number, not of the number itself. One-third is one-third.

Assuming either a decimal floating-point system or the more common
binary floating-point system, there is no floating-point number that
corresponds to the mathematical real value one-third. There is a
floating-point number whose corresponding real value is closer to
one-third than any other floating-point number's corresponding real
value is. That FP number is the closest FP approximation to
one-third. Of course that approximation is exactly representable as a
floating-point number, since we chose it based on that criterion.

[...]

However, I don't think there's any place where the standard actually
says the number does or does not denote the range**, so in lieu of that,
I'd say it's up to human interpretation what numbers in that
unrepresentable space "mean". Maybe they mean "hamsters".

** though a proximity relationship between the number and the
surrounding unrepresentable space is implied by the sections on
conversions and by the model itself

I believe C99 5.2.4.2.2 does actually say that a floating-point number
denotes a single real value. Paragraph 2:

A floating-point number (x) is defined by the following model:

x = [mathematical expression]

The mathematical expression (which uses mathematical notation that I
won't try to reproduce here) is clearly being used to define a real
number, not a floating-point number; it uses constructs that don't
exist in C. Mathematically, it evaluates unambiguously to a single
real number, not to a range. The use of "x = " indicates that the
value of a floating-point number is that single real number.

IMHO that *should* settle the question (and it's something I hadn't
realized before I started this journey).

 

[Keith Thompson]

Keith Thompson wrote:
[...]
float x = 0.3333333432674407958984375;
float y = 3.33333343e-01;

The stored values in x and y are identical.

(x == y) is 1
Agreed (given certain reason able assumptions).

My point, however, is that the float value stored in x corresponds to
the mathematical real value 0.3333333432674407958984375 . The
mathemetical value 0.333333343 (notation tweaked for consistency)
cannot even be represented as a float value, though of course a close
approximation of it can.

Boo. 'float x = 1/3.;' to assign a value to the float x. If I print
the value with %.25f I get your wide number. If I print with '%.9f' I
get my narrower one. There is no difference in the value of either
expression when stored in a float.

Yes, when stored in a float. I'm talking about the relationship
between stored float values and mathematical real values.

That 0.333333343 cannot even be
represented as a float value is wrong.

No, as I said, a close approximation of 0.333333343 can be stored as a
float value, but the exact mathematical value 0.333333343 (i.e., the
rational number 333333343 / 1000000000) cannot (unless FLT_RADIX is a
power of ten).

Since the topic of this discussion is exact stored values, an
appoximation, even one that's sufficiently close for most purposes,
would not have been sufficient to make my point.

Again, my approximation is every bit as good as yours for float. In
fact mine is closer to 1/3. than yours.

Am I really being that unclear?

The point is that my "approximation" *isn't an approximation*. It is
the exact mathematical value represented by the floating-point vlaue
stored in x. In most contexts, the distinction is probably
unimportant. In the context of this discussion, the distinction is
central to the point I've been trying to make.

To put it another way, the stored value 0.3333333432674407958984375 is
exactly equal to the rational number 11184811 / 33554432 where the
denominator is 2**25. In a binary-radix floating-point system, each
representable number (ignoring infinities and NaNs) is exactly equal
to some rational number where the denominator is a power of 2.

The approximation 0.333333343, though it's perfectly fine in most
contexts, doesn't demonstrate this.

I do understand exactly what you are saying. I'm going to have to do
some more reading before I can comment further. Thank you very much.

All right. Let's see if I've got the point now.

float x = 1.2;
1.20000005e+00
20132660 / 16777216
1.2000000476837158203125

What do you think? The result is 23 digits wide, more than the 16 or
so guaranteed by double. Is that OK?

Looks good. Reduced to lowest terms (dividing the numerator and
denominator by 4), I get 5033165 / 4194304, where the denominator is
2**22.

Any time you print a fraction defined in one base in another base, if
both bases aren't powers of the same number, you'll need more digits
to represent the exact value than you'd need just to guarantee
uniqueness.

 

[Keith Thompson]

Maybe that is the problem. The C standard cannot override
mathematics and physics - it simply has to be subservient. If it
apparently is not, that indicates an omission in the standard.

No, it doesn't have to be subservient, and it isn't.

Physics, as I'm sure you'd be quick to say if the shoe were on the
other foot, is off-topic. Floating-point numbers can of course
be used to model physical quantities, though the model is only an
approximation in most cases.

As for mathematics, it encompasses far more than you seem willing
to acknowledge. The set of floating-point numbers of a given
type corresponds to a finite proper subset of the real numbers.
That is a perfectly valid mathematical concept.

Are you asserting that a model in which the FP numbers completely
cover a range of real numbers is mathematically valid, while a model
in which they cover only a finite subset of real numbers is not?

If the standard states that a value in a FP object represents an
object in the range x*(1-EPSILON) to x*(1+EPSILON), that is fair
enough. If it also says that the FP object value is exact, it is
wrong, or it is reusing words in a different manner, etc.

Since I can't easily render the formula in 5.2.4.4.2 in
plain text, I've grabbed a screenshot and made it available at
<http://www.mib.org/~kst/5.2.4.2.2.gif>. Please read it. It very
clearly says that a floating-point number is defined by a formula
that specifies a single unique real value, *not* a range.

If you want to argue that I've misinterpreted it, that the formula
in the standard is consistent with your interpretation, by all
means make that argument. If, on the other hand, you want to argue
that the model in the standard is flawed, then make that argument.
But if you want to claim your model is right, and that a model
that implementers and programmers have been using for years with
no problems I'm aware of beyond those that are inevitable for any
floating-point model, you're going to have to make a much better
argument than you've been making so far.

It's not impossible that you're right and the rest of us are wrong.
But if I were seeing this kind of nearly unaniminous disagreement
from people I respect, I'd take a moment to reexamine my assumptions.

 

[Beej Jorgensen]

I'm not sure what you mean by "one-third (base 10)".

Sorry--leftovers from paragraph rearrangement, but the idea was merely
to specify 1/3 as a number that could not be perfectly represented (in a
particular base).

Assuming either a decimal floating-point system or the more common
binary floating-point system, there is no floating-point number that
corresponds to the mathematical real value one-third. There is a
floating-point number whose corresponding real value is closer to
one-third than any other floating-point number's corresponding real
value is. That FP number is the closest FP approximation to
one-third. Of course that approximation is exactly representable as a
floating-point number, since we chose it based on that criterion.

I'm going to go ahead and agree with all this. I think I was mistaken
about what you were saying about the model... but let me get
clarification:

My question is (rephrased), does the standard's floating point model
define the actual mathematical value of 1/3 for a base 10 system, or
does the model merely define the approximation of 1/3? I was supposing
the model could not define the true value of 1/3 in that case (since it
only sums out to the precision, not forever) and therefore the model
must only define the approximation.

To generalize then, are there *any* numbers which the model defines that
are not exactly representable in an implementation for that
implementation's particular set of parameters to the model? I was
supposing the model did not define any such numbers. (But this is where
I'm fuzzy and will readily accept proof-by-counterexample.)

I was further supposing, then, that the floating point model did not
actually define any numbers past the representation's precision, which
would leave such things undefined(?). (But the standard does seem to
speak of unrepresentable numbers in terms of conversions.)

I believe C99 5.2.4.2.2 does actually say that a floating-point number
denotes a single real value. Paragraph 2:

Ok, I'll go with all that.

Nevertheless, I know there must be a way to close this gap in a mutually
agreeable way.

I'm going to latch onto something else CBF said a short time ago which
had to do with what you could "deduce" from a stored value.

Are you willing to say that, with no prior knowledge of how a floating
point value was arrived at, basically nothing about previous loss of
precision can be deduced from that stored value?

-Beej

 

[Phil Carmody]

Are you, Phil Carmody, a bit thick in the ways of usenet? Do you, Phil Carmody
think each post that quotes you, Phil Carmody, is directed in its entirety at
you, Phil Carmody, and only you, Phil Carmody?

Given that the above followed immediately after a paragraph which was
peppered with the word "you", it was a fair assumption to make.

Just so you, Phil Carmody, know the part partially quoted above was not directed
at Phil Carmody, but some other(s) [unnamed] also posting in this thread that
seem from their posts unable to realize that real numbers exist because the ISO
C standard doesn't define them.

The problem was that what you wrote was less useful than things
that other people such as Keith, Richard, and myself have already
written. I find it hard to imagine anyone who would have gained
anything from your post.

Only the ob CBF troll part was directed at Phil
Carmody. I thought the change from second person, the ob CBF troll part, to
third person for the rest should have been enough of a clue, but obviously not.

Thanks for proving that you are a complete fucking idiot so clearly.
Either that or you are a willful liar. Take your pick.

My handy hints:
1) Learn to read
2) Learn to write
3) Learn C
then you might be able to make valuable contribution here, until
then, you're just an empty vessel making distorted noises.

Phil

 

[CBFalconer]

No, it doesn't have to be subservient, and it isn't.

Yes it does. The physics (assuming accurate) is reality. If you
publish a 'standard' that claims your automobile gets 200 mpg, you
will get few believers. I'm not saying that is analagous. The
standard is actually quite accurate. It is not the holy word. The
problem here is interpretation.

Physics, as I'm sure you'd be quick to say if the shoe were on the
other foot, is off-topic. Floating-point numbers can of course
be used to model physical quantities, though the model is only an
approximation in most cases.

As for mathematics, it encompasses far more than you seem willing
to acknowledge. The set of floating-point numbers of a given
type corresponds to a finite proper subset of the real numbers.
That is a perfectly valid mathematical concept.

Are you asserting that a model in which the FP numbers completely
cover a range of real numbers is mathematically valid, while a model
in which they cover only a finite subset of real numbers is not?

Of course not. Don't distort MY comments.

Since I can't easily render the formula in 5.2.4.4.2 in
plain text, I've grabbed a screenshot and made it available at
<http://www.mib.org/~kst/5.2.4.2.2.gif>. Please read it. It very
clearly says that a floating-point number is defined by a formula
that specifies a single unique real value, *not* a range.

The formula is perfectly accurate. It reflects the value specified
by the FP content. It DOESN'T reflect the meaning of that value.

.... snip ...

It's not impossible that you're right and the rest of us are wrong.
But if I were seeing this kind of nearly unaniminous disagreement
from people I respect, I'd take a moment to reexamine my assumptions.

That's the only reason I am continuing to push this business.

 

[CBFalconer]

CBFalconer said:

Keith Thompson wrote:

[...]
float x = 0.3333333432674407958984375;
float y = 3.33333343e-01;
... snip ...

The approximation 0.333333343, though it's perfectly fine in
most contexts, doesn't demonstrate this.

Yes it does. Both the value above are members of the same range,

There are infinitely many ranges of which both values are members.

which is the range specified by the floating object when
initialized with either value.

No, the floating point object doesn't specify a range. It is able
to store a value, but only one value at a time. We covered this
before.

No. Maybe you misstated it before. The point is that a given FP
system always specifies a range covered by a particular FP value.
The mere fact that you say this indicates you haven't a clue as to
what is really going on.

It's evaluated as true when x has the same value as y.


The programming which initialized them doesn't matter as far as
their value is concerned. The thing that affects their value is,
would you believe, the last value to be stored there. If that's the
same for both, both will store the same value. In this case, the
value stored in y is the result of converting a double-precision
floating-point value such as (on my system)
0.333333343000000004163752009844756685197353363037109375 into a
single-precision floating point value.

You just attempted to specify the programming. Wrong. The only
thing available is the FP object itself.

 

[Flash Gordon]

... snip ...
So what? doubles are used to store reals, more or less. ints
are used to store integers. 1.0+DBL_EPSILON/2.0 is a real[1].
1.5 is NOT an integer.

[1] but not a storable real, in a double.
It's that "more or less" that bites you, isn't it? Your model
makes some sense if you ignore those pesky details where it
falls apart.

<snip>

In addition doubles *are* used to store exact integral values
and integer types *are* used to store approximations. Any claim
that you have stored a range when you have in fact stored the
exact value that you intended to store is clearly wrong and would
make error analysis impossible.

So what? You are referring to the programming that is using that
object. I am talking about what you can deduce from the object in
isolation.

That value isThe point is, that taken in isolation you do NOT know
that it is a range. Taken in isolation you only know what the actual
stored value is. is a single exact value specified by a well-defined
model which may represent either an exact quantity or a range which
could be (and ofen is) far larger than the ranges you are talking about.

The standard actually talks about values that "can be represented
exactly" in section 6.3.1.5 (paragraph 2).

The standard also states that the set of values of the style float is a
subset of the values of type double. This can only be true if they are
not ranges as you are describing since the range 0.00001 to 0.00002 is
not the same as the range 0.000001 to 0.000002 (I know those are not
really ranges in your model, but they are easy to type) so with your
model they can be almost completely disjoint sets (infinities and NaNs
may be the only point of intersection). This is in 6.2.5 paragraph 10
which /defines/ real floating types.

So various pieces of wording in the C standard are inconsistent with
your claim that the values stored are always approximations. They are,
however, entirely consistent with the branch of mathematics known as set
theory which allows you do have whatever sets you want.

 

[Flash Gordon]

I'm going to go ahead and agree with all this. I think I was mistaken
about what you were saying about the model... but let me get
clarification:

My question is (rephrased), does the standard's floating point model
define the actual mathematical value of 1/3 for a base 10 system, or

No. Nor does it provide a way to specify it, only a way to specify a
constant expression that is a close approximation to it.

does the model merely define the approximation of 1/3? I was supposing
the model could not define the true value of 1/3 in that case (since it
only sums out to the precision, not forever) and therefore the model
must only define the approximation.

No, it does not define the approximations, it defines the values which
exist in the set of representable numbers. Other parts of the standard
define (or leave implementation-defined) which you get if you divide 1.0
by 3.0 or if you have the double constant 0.1 in your source which
double that will be converted to.

To generalize then, are there *any* numbers which the model defines that
are not exactly representable in an implementation for that
implementation's particular set of parameters to the model? I was
supposing the model did not define any such numbers. (But this is where
I'm fuzzy and will readily accept proof-by-counterexample.)

The model defines the numbers which are representable. By implication it
defines that all other numbers are not representable. If you are told a
bag has only an apple, a banana and an orange in it, then you know
without being told that there is not a mango in it.

I was further supposing, then, that the floating point model did not
actually define any numbers past the representation's precision, which
would leave such things undefined(?). (But the standard does seem to
speak of unrepresentable numbers in terms of conversions.)

They are defined by exclusion.

Ok, I'll go with all that.

Nevertheless, I know there must be a way to close this gap in a mutually
agreeable way.

Not always.

I'm going to latch onto something else CBF said a short time ago which
had to do with what you could "deduce" from a stored value.

Ah, but what he said I disagree with. He claims you can deduce a range
from it, and I disagree because it could represent an exact number, a
value with a possible error of +/-12.7 or any other range.

Are you willing to say that, with no prior knowledge of how a floating
point value was arrived at, basically nothing about previous loss of
precision can be deduced from that stored value?

I would agree with that. A number of people have said that you need to
perform error analysis to know what the value means (and sometimes that
you need to work backwards in your analysis fro the result instead of
forwards from the inputs).

 

[Keith Thompson]

My question is (rephrased), does the standard's floating point model
define the actual mathematical value of 1/3 for a base 10 system, or
does the model merely define the approximation of 1/3? I was supposing
the model could not define the true value of 1/3 in that case (since it
only sums out to the precision, not forever) and therefore the model
must only define the approximation.

If you look at 5.2.4.2.2p1-2, the answer is neither. The FP model
defines a finite subset of the real numbers, where each member of that
subset is (exactly) representable as a floating-point number. The
real number 1/3 is not a member of that subset, so the model has
nothing to say about it. Nor does the model directly say anything
about which representable FP number is closest to the real number 1/3,
though that can be calculated.

The model, at least if you look at just those two paragraphs, defines
a mapping from floating-point values to real values, not vice versa.

Note that you can't even directly express the real number 1/3 in C.
The closest you can come is the expression 1.0/3.0. Each operand is a
floating constant, which is converted to type double (as if) at
translation time. Then the division is performed. The operands of
the division are doubles, not mathematical reals; the accuracy of the
result is implementation-defined.

(This assumes 1/3 is not representable; if FLT_RADIX were a multiple
of 3, it would be.)

Paragraph 5 discusses the accuracy of various operations (which is
implementation-defined and possibly unknown). For example, take two
floating-point numbers, and determine the corresponding real numbers.
Determine the exact mathematical result of performing the operation on
those two real numbers. "Accuracy" is about how close the
floating-point number that actually results from the operation is to
the mathematical real number that theoretically results from the
operation.

(This assumes 1/3 is not representable; if FLT_RADIX were a multiple
of 3, it would be.)

To generalize then, are there *any* numbers which the model defines that
are not exactly representable in an implementation for that
implementation's particular set of parameters to the model? I was
supposing the model did not define any such numbers. (But this is where
I'm fuzzy and will readily accept proof-by-counterexample.)

I believe the answer is no. That's the point of the model, to define
the set of representable floating-point numbers.

[...]

Nevertheless, I know there must be a way to close this gap in a mutually
agreeable way.

I'm going to latch onto something else CBF said a short time ago which
had to do with what you could "deduce" from a stored value.

Are you willing to say that, with no prior knowledge of how a floating
point value was arrived at, basically nothing about previous loss of
precision can be deduced from that stored value?

Yes.

 

[Keith Thompson]

The formula is perfectly accurate. It reflects the value specified
by the FP content. It DOESN'T reflect the meaning of that value.

[...]

What is the difference between "the value" and "the meaning of that
value"? Does the standard define this "meaning"?

 

[Phil Carmody]

Phil Carmody said:


I wondered how long it would take you to work that out.

I earnestly apologise, Richard. He was teetering for quite a while,
too long as you say, but this issue seems to have been a polarising
one.

Now if only the rest of the group can deduce it too and start
exercising their filters a little, we can get a bit more peace and
quiet around here.

Or even better - interesting and on-topic discussion! This could
have been such a thread if it hadn't headed down the loony path.

Phil

 

[Antoninus Twink]

Yes it does.

Holy, holy crap.

You simply couldn't make this up, could you?

Bill Cunningham is an amateur by comparison - I hope he's taking lessons
from this virtuoso display by a true master troll who's managed to play
the ignoramus so well that he's stretched out this ridiculous thread to
435 posts so far - and counting.

(To be fair, he couldn't have done it without Kiki's help - his
Asperger's problem means he can't let /anything/ go, even when it's
obvious to everyone that CBF is ineducable and incorrigible.)

 

[CBFalconer]

The formula is perfectly accurate. It reflects the value specified
by the FP content. It DOESN'T reflect the meaning of that value.

[...]

What is the difference between "the value" and "the meaning of that
value"? Does the standard define this "meaning"?

Not in so many words, but in effect. It specifies how to compute
the uncertainty in the value at some points, using EPSILONs. It
can't be especially precise, because it doesn't specify the FP
specification. All it takes is some understanding of the actual
systems to generalize it. The 'range' word that I have been using
expresses almost all of it.

 

[CBFalconer]

Keith Thompson wrote:

... snip ...

So what? doubles are used to store reals, more or less. ints
are used to store integers. 1.0+DBL_EPSILON/2.0 is a real[1].
1.5 is NOT an integer.

[1] but not a storable real, in a double.

It's that "more or less" that bites you, isn't it? Your model
makes some sense if you ignore those pesky details where it
falls apart.

<snip>

In addition doubles *are* used to store exact integral values
and integer types *are* used to store approximations. Any claim
that you have stored a range when you have in fact stored the
exact value that you intended to store is clearly wrong and would
make error analysis impossible.

So what? You are referring to the programming that is using that
object. I am talking about what you can deduce from the object in
isolation.

That value is. The point is, that taken in isolation you do NOT know
that it is a range. Taken in isolation you only know what the actual
stored value is. is a single exact value specified by a well-defined
model which may represent either an exact quantity or a range which
could be (and ofen is) far larger than the ranges you are talking about.

The standard actually talks about values that "can be represented
exactly" in section 6.3.1.5 (paragraph 2).

Yes, but that is simply a number. You do know what the 'range'
is. The number does not express the range of numbers that will be
stored with that identical value, and simply cannot be told apart.
In isolation what you know is the range of values that can have
been represented thusly.

.... snip ...

So various pieces of wording in the C standard are inconsistent with
your claim that the values stored are always approximations. They are,
however, entirely consistent with the branch of mathematics known as
set theory which allows you do have whatever sets you want.

Go forth and implement a FP system. Then examine when and how it
fails. You will soon appreciate the fundamental truths about such
a system. You will soon see how all values (apart from zero, NaNs,
Infs) are approximations, and how they cannot overlap. One point
is that taking a value and writing it down does NOT convert it from
an approximation to something exact.