Float comparison

[Keith Thompson]

The values of (normalised) floating point variables are defined to
be real numbers.

[...]

Yes. A small quibble: the values of non-normalized floating-point
values are also defined to be real numbers. The standard mentions
subnormal and unnormalized numbers. The distinction you really wanted
was "other than infinities and NaNs".

 

[Keith Thompson]

In the past I have proved to myself that float values can be
represented using "%.8e" format of printf(). float x = FLT_MAX; is
3.40282347e+38. The conversions work both ways exactly and I was
pleased to find the short 'string' representation of any float. I was
concerned with 'close enough' for the conversions to work, not with
the mathematical exactness of anything.

[...]

Note that "%.8e" is insufficient if float is bigger than the common
32-bit representation. (I've actually worked on a system where float
is 64 bits.)

If you want a text representation that's guaranteed to give you a
unique and reversible approximation for each value, you can use the
value of FLT_DIG. For example:
printf("x = %.*e\n", FLT_DIG + 2, x);

And similarly for DBL_DIG and LDBL_DIG.

I'm not sure that "+ 2" is the right fudge factor.

If you have a C99 implementation, you can also use "%a", which uses a
hexadecimal representation and is guaranteed, to use enough digits.
If the radix is a power of 2 (as it usually is), "enough digits" means
that the value is represented exactly; otherwise, enough digits are
shown to guarantee uniqueness. There are a couple of drawbacks: the
format is much less familiar, and it may not be supported in all
implementations.

 

[Keith Thompson]

CBFalconer said:

Is the interval actually defined for anything other than 1.0?

Yes, but it's implementation-defined. The floating-point model in C99
5.2.4.2.2 has a number of parameters (exponent range, precision,
etc.).

"You see this? This is this." CBF means the value that was attempted
to be stored, not the value that was stored.

But what does "attempted" mean? Programs don't attempt to do things.

I think the ultimate answer is in c99 6.3.1.4p2, 6.3.1.5p2, and
probably elsewhere:

# If the value being converted is in the range of values that can be
# represented but cannot be represented exactly, the result is either
# the nearest higher or nearest lower representable value, chosen in an
# implementation-defined manner.

That's talking about conversions. There's similar (but looser)
wording for arithmetic operators (+ - * /) and for the functions in
<math.h>. But all this stuff happens before the resulting value is
stored.

Regardless of how a floating-point value is computed, the result is
some unique value.

The Standard, as such, is defining a relationship between an
unrepresentable subspace and its neighboring representable values:

s t
A|-----|B|-----|C

It says if a conversion ends up in unrepresentable subspace s, the value
stored is A or B. If a conversion ends up in subspace t, the value
stored cannot be A--it must be B or C.

So if I tell you the stored value is EXACTLY B, you really can't tell me
what mathematical value led to that result, but you can tell me with
certainty that it fell somewhere between A and C. This is my read on
what CBF is saying.

Right, but I've been discussing the result itself, not what led to it.
It could just as well have been produced by treating a double object
as an array of unsigned char and using system-specific knowledge to
construct the desired representation.

Yet the value ultimately stored is EXACTLY B. This is my read of what
everyone else is saying. I don't think CBF actually disagrees with
this.

I don't think CBF actually understands this.

Floating-point operations generally yield approximate values; I think
everyone understands this.

*Some* floating-point operations can yield exact values. For example,
1.0 + 1.0 == 2.0 in any sane implementation, though I don't think the
standard quite managers to guarantee this. But I think we can at
least assume that the constant 1.0 yields the value one.

So we can discuss the range of values that might be produced by a
given floating-point expression. But any FP expression will yield
just one value in that range, and when that value is stored in an
object, all information about the range is discarded, leaving only the
unique representable floating-point value, which by definition
(5.2.4.2.2) denotes a single unique real number. And that, I think,
is what CBF has been having trouble with.

 

[Richard Bos]

Agreed. That comes from trying to find an argument to convince
people of the obvious. A very poor argument.

No, it comes from trying to find an argument to convince people of the
obvious _but false_. The poverty is not in the argument, it is inherent
in the purpose.

Richard

 

[Richard Bos]

I've seen this argument many times, and it remains in my experience and
opinion a bogus point often generated by people who grew up on cobol and
don't understand how to use floating point numbers effectively in the
financial sector. Admittedly I've only worked in Bonds & Derivatives
trading, Risk Management and Finance, and only had to concern myself
with yen, lira, lire, sterling, euro, pre-euro, dollars etc so there may
be some esoteric corner where the point holds.

I dunno, but it seems to me that saying that something was commonly used
in Risk Management and Finance is, right now, _not_ the very best
recommendation for best practices. Especially from a Scot...

Richard

 

[Richard Bos]

A wired world, indeed. Correlates to the famous Linus quote "Only wimps
use tape backup: _real_ men just upload their important stuff on ftp,
and let the rest of the world mirror it."

Pah. I challenge Linus to "back up" his important bank account numbers,
PIN codes, and all associated information, somewhere where the world can
mirror it.

Richard

 

[Richard Bos]

Quantum Boggum Sort:

*Thwack* If you quote the Jargon File, do so properly. That's
_bogo_-sort.

Richard

 

[Beej Jorgensen]

But what does "attempted" mean? Programs don't attempt to do things.

In this case, it means, for example, an attempt was made to store a
representable double in a float, where it was not representable.

That's talking about conversions. There's similar (but looser)
wording for arithmetic operators (+ - * /) and for the functions in
<math.h>. But all this stuff happens before the resulting value is
stored.
Agreed.

Right, but I've been discussing the result itself, not what led to it.

Yup. I think Chuck is talking about more than that.

I don't think CBF actually understands this.

Let's get a definitive answer, Chuck! The statement: regardless of what
it "means", how it was arrived at, and what unrepresentable intermediate
values were computed, a value as stored is stored exactly, as provable
with the == operator.

-Beej

 

[Phil Carmody]

The values of (normalised) floating point variables are defined to
be real numbers.

[...]

Yes. A small quibble: the values of non-normalized floating-point
values are also defined to be real numbers. The standard mentions
subnormal and unnormalized numbers. The distinction you really wanted
was "other than infinities and NaNs".

Yup. I didn't want to clutter the sentence too much with additional
concepts that I didn't want to address. Thanks for making sure that
the t was crossed and the i dotted.

Phil

 

[BartC]

The %a format works well for this. To get exact value you also need
to know FLT_RADIX ad FLT_MANT_DIG (or DBL_MANT_DIG).

Thanks, that's the sort of thing I had in mind.

 

[Keith Thompson]

In this case, it means, for example, an attempt was made to store a
representable double in a float, where it was not representable.

The word "attempt" is a bit too anthropomorphic for my tastes. For
example:

double d = 1.0 / 3.0;
float f = d;

You might say that the second initializer "attempts" to store a double
value in a float, but what the declaration actually *means* is that
the float value is (implicitly) converted from double to float, with a
potential loss of information, before being stored. When I wrote the
above declarations, I wasn't "attempting" to store a double value in a
float; I was simply storing the result of the conversion. The loss of
information was any kind of failure.

Yup. I think Chuck is talking about more than that.


Let's get a definitive answer, Chuck! The statement: regardless of what
it "means", how it was arrived at, and what unrepresentable intermediate
values were computed, a value as stored is stored exactly, as provable
with the == operator.

That doesn't quite cover it. C's floating-point model describes the
meaning of a floating-point value as a mathematical real value. The
"==" operator doesn't operate on mathematical real values. Chuck
could consistently answer "yes" to your question while continuing to
maintain that a stored floating-point value denotes a range of real
values; presumably "==" would yield a true result if both operands
denote the same real range.

 

[Phil Carmody]

<ob CBF troll>
You are saying that for this topic HfC is a more reliable source, correct?
</ob CBF troll>

I don't know, as I don't read HfC posts, they're pre-purged so I don't
even know if he's made any. If I see replies to him, I typically just
skip over them, paying them no attention at all.

[snip]

Richard's absolute ('No') is absolutely correct, and not misleading
in any way. If you think you've been misled, then that's because
you're on the wrong path, and attempting to head further down the
wrong path - Richard's trying to pull you onto the correct path.

One is leading you over hill and dale. One is leading you to the dark side.

Truth is that the thing called floating point is a glorified scaled type of
integer arithmetic. Digital computers can only evaluate integers.

Truth is that nowadays they can only evaluate bits.

Truth is:

Real number >stored as> floating point >print out> usually a different real number.

There's no news there. What made you think I wasn't aware of that - are
you a bit thick in the reading-comprehension department?

[SNIP - stuff that superficially looked like it contained nothing of
worth to me]

Phil

 

[Keith Thompson]

The word "attempt" is a bit too anthropomorphic for my tastes. For
example:

double d = 1.0 / 3.0;
float f = d;

You might say that the second initializer "attempts" to store a double
value in a float, but what the declaration actually *means* is that
the float value is (implicitly) converted from double to float, with a

That should be "the double value".

potential loss of information, before being stored. When I wrote the
above declarations, I wasn't "attempting" to store a double value in a
float; I was simply storing the result of the conversion. The loss of
information was any kind of failure.

That should be "wasn't any kind of failure".

Two major meaning-altering typos in one paragraph.

D'oh^H^H^H^H annoyed grunt.

 

[Beej Jorgensen]

The word "attempt" is a bit too anthropomorphic for my tastes.

Well... you know what I meant.

That doesn't quite cover it. C's floating-point model describes the
meaning of a floating-point value as a mathematical real value.

But with limits, most notably p and e.

The "==" operator doesn't operate on mathematical real values.

Sure, but are there numbers the model describes that aren't exactly
representable on a given implementation*? Numbers like one-third (base
10), for instance, the model only approximates out to a given precision,
and that approximation I believe should be exactly storable in a base 10
system of that precision.

* for that implementation's particular parameters to the model

But I'd welcome discussion on this point.

Chuck could consistently answer "yes" to your question while continuing
to maintain that a stored floating-point value denotes a range of real
values

Yes, he could, but I still wanted to hear him say "yes", because it
sounded like you thought he wouldn't say yes. Maybe I misread.

If I were him speaking in defense of a floating-point value denoting a
range, I'd cite the standard's ideas on conversion, on accuracy of
floating point calculations, and the Rationale's discussion of floating
point accuracy.

And you've cited the standard regarding the exactitude of the stored
numbers, which is why I'd think CBF would have a hard time disagreeing
with that particular aspect.

However, I don't think there's any place where the standard actually
says the number does or does not denote the range**, so in lieu of that,
I'd say it's up to human interpretation what numbers in that
unrepresentable space "mean". Maybe they mean "hamsters".

** though a proximity relationship between the number and the
surrounding unrepresentable space is implied by the sections on
conversions and by the model itself

-Beej

 

[Beej Jorgensen]

That should be "the double value".
That should be "wasn't any kind of failure".
Two major meaning-altering typos in one paragraph.

That's ok--my bad reading comprehension actually caused me to misread
these, so they were canceled out.

-Beej

 

[CBFalconer]

.... snip ...


Physics nothing to do with this. Again, I'm not talking about
physical measurements.

I'm talking about the C standard, which you have yet to cite in
any way that directly supports your claims.

Maybe that is the problem. The C standard cannot override
mathematics and physics - it simply has to be subservient. If it
apparently is not, that indicates an omission in the standard.

If the standard states that a value in a FP object represents an
object in the range x*(1-EPSILON) to x*(1+EPSILON), that is fair
enough. If it also says that the FP object value is exact, it is
wrong, or it is reusing words in a different manner, etc.

 

[CBFalconer]

.... snip ...

So what? doubles are used to store reals, more or less. ints
are used to store integers. 1.0+DBL_EPSILON/2.0 is a real[1].
1.5 is NOT an integer.

[1] but not a storable real, in a double.

It's that "more or less" that bites you, isn't it? Your model
makes some sense if you ignore those pesky details where it
falls apart.

<snip>

In addition doubles *are* used to store exact integral values
and integer types *are* used to store approximations. Any claim
that you have stored a range when you have in fact stored the
exact value that you intended to store is clearly wrong and would
make error analysis impossible.

So what? You are referring to the programming that is using that
object. I am talking about what you can deduce from the object in
isolation.

 

[CBFalconer]

Agreed (given certain reason able assumptions).

My point, however, is that the float value stored in x corresponds
to the mathematical real value 0.3333333432674407958984375 . The
mathemetical value 0.333333343 (notation tweaked for consistency)
cannot even be represented as a float value, though of course a
close approximation of it can.

Since the topic of this discussion is exact stored values, an
appoximation, even one that's sufficiently close for most
purposes, would not have been sufficient to make my point.

I disagree. The topic is "What information is contained in a
floating point object", all by itself. This may be the root of our
disagreement. I have repeatedly stated that I was ignoring the
effect of programming on the information in the value.

 

[CBFalconer]

[...]
float x = 0.3333333432674407958984375;
float y = 3.33333343e-01;

.... snip ...

The approximation 0.333333343, though it's perfectly fine in most
contexts, doesn't demonstrate this.

Yes it does. Both the value above are members of the same range,
which is the range specified by the floating object when
initialized with either value. Which is also why (x == y) is
evaluated as true. The only difference is in the programming which
initialized them.

 

[CBFalconer]

.... snip ...

All you did was provide an expression for a different real number
which when converted to floating point type results in the same
value as Keith's (given fair presumptions about the FPU in use).
Your value is no more special than an infinitude of others which
also have that property. Only Keith's had the unique property of
exact equality in the reals.

I consider this ridiculous. You are claiming that two objects,
holding the identical bit patterns, are unequal. The only
difference you can find is the address of those objects, which I
don't think anybody thinks should affect the value. To justify the
statement you would have to show how you can gnaw on the identical
contents (as bit streams) and find a difference.