Float comparison

[Dik T. Winter]

....
> I don't understand your problem. In tinyfloat, EPSILON is
> precisely 1/16. This is the weight of the bit that will be rounded
> in preparing a new fp-object value.

The above quotes clearly show that EPSILON is a single value.

> Think about WHY EPSILON changes at the power of two.

And now suddenly it changes. A bit inconsistent I would say.

 

[gwowen]

(e-mail address removed) said:

You lost me about here.
Thanks for trying, but I think I'll stick to programming.

The only thing worth remembering:

You have a bunch of simple sets whose measure you understand
intuitively.
You measure everything else by comparison with the simple ones.
"Can I cover my bizarre set my cutting up one of my simple sets and
placing the pieces over the top?"

If you can, the measure of the bizarre set is less than or equal to
the measure of the simple one you cut up.

Everything else is technicalities.

 

[Ben Bacarisse]

I don't understand your problem.

Simply that I have tried to find the ranges for 5 consecutive floats
in your 3-bit (or 4-bit normalised) example and you won't tell me
what they are. Since I am too stupid to follow your explanations,
just tell me the actual numbers -- there are only 10 of them.

In tinyfloat, EPSILON is
precisely 1/16. This is the weight of the bit that will be rounded
in preparing a new fp-object value.

OK. You are free to define it like that. That is not C's definition.

...

And these are of the form 1.0+something.

[editorial note: you've edited what I wrote and not changed the
quoting. I gave 5 numbers, each with a possible range. You've
removed most of the ranges (presumably they were wrong) and added
stuff that I don't want anyone to think I wrote. If you delete this
note, please change the quote characters.]

Those ranges are from < f+EPSILON down to > f-EPSILON

Eh? Just tell me the ranges. 10 numbers. That is all I want. At
least I will then have one fixed point in all these changing epsilons
and EPSILONS.

Think about WHY EPSILON changes at the power of two. It has to do
with the rounding in computing the real value x*(1+EPSILON) when
that expression is handed to the fp-system.

C's EPSILON does not change at any power of 2. You have repeatedly
agreed in the past that your EPSILON stands for C XXX_EPSILON
constants but that seems not to be the case any more. If your EPSILON
changes, you have to say how. What is the formula for it?
Alternatively, is there a formula for the range that does not need
your variable EPSILON function?

Somewhere I specified the rounding. If the dropped bit is a 1,
increase the significand. If a 0, don't increase.

So does your model not work with some valid C floating point systems?
If it can, then I wanted to know what happens to the meaning of a
float when the rounding mode is changed. You claim the range is the
meaning; and the range depends on the rounding mode.

 

[Ben Bacarisse]

I would have the same results except for the last entry. It
certainly eases some calculations, but fouls the statistics.

Have you read the Goldberg article so often cited here? Rounding up
causes "drift" that rounding to even does not. There are compelling
reasons to choose round to even.

 

[Keith Thompson]

OK. You are free to define it like that. That is not C's definition.

No, he's not free to use a definition that's inconsistent with the one
given by the standard, at least not if he wants to define a model for
C floating-point numbers.

[...]

 

[Ben Bacarisse]

No, he's not free to use a definition that's inconsistent with the one
given by the standard, at least not if he wants to define a model for
C floating-point numbers.

I don't see why the thing he calls EPSILON can't be defined any way he
likes as long as the distinction is clear (which it has not been).

I am annoyed that he's claimed it be the same as C's XXX_EPSILONs
when it now seems it is not, but that is another matter. Do you see
something inconsistent between his usage and C?

 

[CBFalconer]

And how's that working out?

Well, I understand it!

 

[CBFalconer]

Remember the range is dealing ONLY with the effect of fp-storage.

So, if the values were not known to be exactly 1.0, they had to be
ranges, because that is all the fp-object value means, without
considering the programming. The range for the original x and y
values is going to be from 1.0 + EPSILON down to 1.0 - EPSILON/2
(because of the range shift at 1.0). The largest value possible
for x*y is (1.0 + EPSILON) squared. The smallest is (1.0 -
EPSILON) squared.

call the above CBF's theorum (CBFT)


I don't follow that bit

so by the CBFT
z_min = x_min * y_min;
and
z_max = x_max * y_max;

Now consider
double z1 = 1.0;
double z2 = 1.0 * 1.0;

z1 and z2 represent different ranges. Are they equal?

The range represents only the error introduced by the fp-system
storage. It is a minimum error for the actual value. So the range
of z2 is the same as the range for z1. The possible error is
different - see above.

 

[CBFalconer]

You don't get to change the context to suit your argument.

The expression 1.0/3.0 appeared as a C expression. If you want to
talk about "just arithmetic", please take it to sci.math. We're
talking about the semantics of C floating-point.

If you're not specifying fp division or int division then you are
off-topic.

1.0 and 3.0 are expressions of type double. Do I need to cite the
standard to prove it?

When you write them down they are char strings. If you are
considering them as fp-object-values, they have a range, and don't
represent exactly 1.0 and 3.0. And this is a silly argument, IMO.

 

[CBFalconer]

So you have a value greater than xmax (the upper limit of your range for
the fp-value 1.0) which when converted to an fp-value gets converted to
1.0, i.e. a range which does not include it?

No, look at the values used. foo is smaller than xmax, not
greater. That's why it generates 1.0, which has a range that
includes the value of foo. The exact value of foo CANNOT be stored
in the tinyfloat system.

.... rest based on error

 

[CBFalconer]

.... snip ...


So xmin and xmax, both of which are real numbers, *are* the bounds
of the real range that is represented by the FP number x? Is that
really what you're saying?

They are bounds on numbers that can be inserted into the fp-system
and produce fp-object-values identical to x. The insertion process
takes some real, does whatever is needed to produce the optimum fit
in the fp-system, and stores that value. That xmin value is only
valid for that x and that fp-system.

....

 

[CBFalconer]

We have been trying to establish what CBF's model is by trying to
get him to state it clearly so we can actually find such a point.


I did, and CBF changed the meaning of the terms xmin, xmax, ymin,
ymax I defined in his first reply and then proceeded to not answer
the question I asked using the terms I explicitly defined.

Somewhere back when, yes, I did tighten some definitions. This
happened as it became clear to me what was bothering people. Now
the whole system is a mess with everybody pushing their own
definitions.

I just want to stick with mine, as they have evolved.

 

[CBFalconer]

CBFalconer wrote:
.... snip ...


I will track down the message ID where I first introduced the
terms if you want. I still have the original post somewhere in my
sent box.

Please DON'T bother.

They don't seem to work for that. See the messages with actual numbers
in them.

See my demonstration with the tinyfloat system. They work.

 

[CBFalconer]

You CAN type the above line of C code (or the equivalent for
double) in to a text file. You CAN then pass that C code through
a compiler and produce an executable. And YOU are the person
saying that the real value of one third can exist in a C program.
You cannot have it both ways and say that one third can exist but
a number that can be exactly typed in decimal cannot.

I didn't say it didn't exist. I said it couldn't be used in the
tinyfloat system. There is no place to put those less significant
bits.

We already know that. Remember that I have already stated that I have
implemented some floating point arithmetic in assembler. Just to be
clear, this was on a processor which did not have a floating point
processor (although it did have a couple of assembler instructions to
help in doing floating point arithmetic, basically a normalise and a
denormalise instruction).


So you are saying that a value greater than the upper bound of your
range for 1.0 which gets converted to 1.0

No. I said that a value LESS than xmax (for 1.0) gets converted to
1.0. That xmax was (the binary for the significand) 10001, where
the final 1 was used only for rounding purposes.

....

 

[CBFalconer]

.... snip ...


EPSILON doesn't change; it's a constant. What you call epsilon
does change.

Yes, I used sloppy verbiage. epsilon changes for every x, and is a
real value. The practical EPSILON, which causes a minimum visible
difference in fp-object values, changes when x becomes an exact
power of two. It should have yet another name.

 

[CBFalconer]

No, he's not free to use a definition that's inconsistent with the one
given by the standard, at least not if he wants to define a model for
C floating-point numbers.

The following is the C standard definition. I am interpreting
"least value greater than 1" as the real value that is to be
inserted into the fp-system to form the successor value.

-- the difference between 1 and the least value greater
than 1 that is representable in the given floating
point type, b1-p

FLT_EPSILON 1E-5
DBL_EPSILON 1E-9
LDBL_EPSILON 1E-9

 

[CBFalconer]

[...]
In tinyfloat, EPSILON is precisely 1/16. This is the weight
of the bit that will be rounded in preparing a new fp-object
value.

OK. You are free to define it like that. That is not C's
definition.

Yes it is. See my response a few minutes ago.

He's said several times in response to direct questions that EPSILON
means either FLT_EPSILON, DBL_EPSILON, or LDBL_EPSILON. (More
recently, he's "clarified" that EPSILON and epsilon are two different
things; I don't recall him saying this when he introduced the terms.)

I am only talking about one floating point system. Not the three
defined by the C standard. The results apply to any of those.
Thus the word EPSILON is general to all fp-systems.

 

[Flash Gordon]

No. I can put them in a structure:

typedef struct rational {int numer, int denom} rational;

const rational onethird = {1, 3}; /* <--- this thing */

/* convert a rational to an fp representation */
float formfloat(const rational r) {
return (float)r.numer / r.denom;
}

I thought you were not considering how the floating point objects got
there values? Certainly when I pointed out real uses of double which use
it precisely with no error you said it did not count because it depended
on the code. You can't have it both ways. Well, you can try, but it is
likely that someone will remember and point it out.

Now, since you are allowing analysis of code to impact on value...
double x = 1.0;
x now contains exactly 1.0 with absolutely no error.

 

[Flash Gordon]

And given that the rounding mode can be changed at run-time, there
are even supportable reasons for modern C compilers to not perform
such short-cuts if they think that by not so doing they're doing
you a favour.

OK, I'll accept that in C99 this is a potential issue. Of course, where
all rounding modes produce the same result it should still do the
optimisation!

 

[CBFalconer]

Have you read the Goldberg article so often cited here? Rounding
up causes "drift" that rounding to even does not. There are
compelling reasons to choose round to even.

I was thinking about that last night, and I think I have a strong
argument for using the simpler (is the bit 1) criterion for
rounding. Consider the value:

1110.1000

Now we say it is exactly halfway between the two possible rounded
values, because there are no more 1 bits in the value. However,
that value represents a real, and the reals outnumber the rationals
by an order of infinities. Therefore there almost MUST be another
1 bit somewhere to the right, and therefore we should round up.

That article, when I read it, is discussing forward and backward
error analysis. It is applicable to the maths involved. What I
have been discussing here has to do with the meaning of the stored
float.