The machine epsilon

[jacob navia]

Hi.

Continuing with my tutorial, here is an entry I have added recently. I
hope it is not controversial. If you see any errors/ambiguities/etc please
just answer in this thread.

Thanks in advance for your help.
----------------------------------------------------------------------
The machine epsilon

The machine epsilon is the smallest number that changes the result of an
addition operation at the point where the representation of the numbers
is the densest. In IEEE754 representation this number has an exponent
value of the bias, and a fraction of 1. If you add a number smaller than
this to 1.0, the result will be 1.0. For the different representations
we have in the standard header <float.h>:

#define FLT_EPSILON 1.19209290e-07F // float
#define DBL_EPSILON 2.2204460492503131e-16 // double
#define LDBL_EPSILON 1.084202172485504434007452e-19L //long double
// qfloat epsilon truncated so that it fits in this page...
#define QFLT_EPSILON 1.09003771904865842969737513593110651 ... E-106

This defines are part of the C99 ANSI standard. For the standard types
(float, double and long double) this defines should always exist in
other compilers.

Here is a program that will find out the machine epsilon for a given
floating point representation.

#include <stdio.h>
int main(void)
{
double float_radix=2.0;
double inverse_radix = 1.0/float_radix;
double machine_precision = 1.0;
double temp = 1.0 + machine_precision;

while (temp != 1.0) {
machine_precision *= inverse_radix;
temp = 1.0 + machine_precision ;
printf("%.17g\n",machine_precision);
}
return 0;
}

 

[Richard Heathfield]

jacob navia said:

For the different
representations we have in the standard header <float.h>:

#define FLT_EPSILON 1.19209290e-07F // float
#define DBL_EPSILON 2.2204460492503131e-16 // double
#define LDBL_EPSILON 1.084202172485504434007452e-19L //long double
// qfloat epsilon truncated so that it fits in this page...
#define QFLT_EPSILON 1.09003771904865842969737513593110651 ... E-106

This defines are part of the C99 ANSI standard.

Well, three of them are.

Your text suffers from your usual confusion between lcc-win32 and C.

<snip>

 

[jacob navia]

jacob navia said:



Conforming implementations must not define QFLT_EPSILON in <float.h>

The C standard paragraph J.5.6: Common extensions:
J.5.6 Other arithmetic types
Additional arithmetic types, such as _ _int128 or double double, and
their appropriate conversions are defined (6.2.5, 6.3.1). Additional
floating types may have more range or precision than long double, may be
used for evaluating expressions of other floating types, and may be
used to define float_t or double_t.

Well, three of them are.

You cut the next sentence!

"For the standard types (float, double and long double) this defines
should always exist in other compilers. "

This is a good example of BIAS when quoting.

 

[Richard Tobin]

[/QUOTE]

Presumably you meant "these", not "this". And it would be less jargonish
to say "definitions" rather than "defines".

You cut the next sentence!

"For the standard types (float, double and long double) this defines
should always exist in other compilers. "

This is a good example of BIAS when quoting.

I don't think so. There's no need to say something that's wrong here,
even if you clarify it in the next sentence.

-- Richard

 

[jacob navia]

Presumably you meant "these", not "this". And it would be less jargonish
to say "definitions" rather than "defines".

You cut the next sentence!

"For the standard types (float, double and long double) this defines
should always exist in other compilers. "

This is a good example of BIAS when quoting.

I don't think so. There's no need to say something that's wrong here,
even if you clarify it in the next sentence.

-- Richard
[/QUOTE]

OK. Now it is:

These definitions (except the qfloat part) are part of the C99 ANSI
standard. For the standard types (float, double and long double) they
should always exist in other compilers. The type qfloat is an extension
of lcc-win32.

 

[Richard Heathfield]

jacob navia said:

The C standard paragraph J.5.6: Common extensions:
J.5.6 Other arithmetic types
Additional arithmetic types, such as _ _int128 or double double, and
their appropriate conversions are defined (6.2.5, 6.3.1). Additional
floating types may have more range or precision than long double, may
be
used for evaluating expressions of other floating types, and may be
used to define float_t or double_t.

What has that to do with what I said? I didn't say that implementations
can't provide extra types. I said a conforming implementation must not

You cut the next sentence!

Yes. It was not relevant to my point, which is that QFLT_EPSILON is not
part of the C Standard.

"For the standard types (float, double and long double) this defines
should always exist in other compilers. "

This is a good example of BIAS when quoting.

No, it isn't. Your statement incorrectly claimed that QFLT_EPSILON was
part of the C99 Standard. The fact that it was followed by another
statement which didn't reiterate the claim does not make your original
claim correct.

 

[Richard Heathfield]

jacob navia said:

OK. Now it is:

These definitions (except the qfloat part) are part of the C99 ANSI
standard. For the standard types (float, double and long double) they
should always exist in other compilers. The type qfloat is an
extension of lcc-win32.

Why mention lcc-win32 extensions in a C tutorial?

And if it's an lcc-win32 tutorial, why ask for comment in a C newsgroup?
Is there not an lcc-win32 newsgroup where you can discuss lcc-win32
extensions?

 

[Richard Tobin]

Why mention lcc-win32 extensions in a C tutorial?

And if it's an lcc-win32 tutorial, why ask for comment in a C newsgroup?
Is there not an lcc-win32 newsgroup where you can discuss lcc-win32
extensions?

You're not being reasonable. He's writing a C tutorial intended for
lcc-win32 users. Since most of it is standard C, I see no reason why
he shouldn't post about it here.

-- Richard

 

[Richard Heathfield]

Richard Tobin said:

You're not being reasonable.

Shurely shome mishtake?

He's writing a C tutorial

The internal evidence of his articles suggests otherwise.

<snip>

 

[Eric Sosman]

Hi.

Continuing with my tutorial, here is an entry I have added recently. I
hope it is not controversial. If you see any errors/ambiguities/etc please
just answer in this thread.

Thanks in advance for your help.
----------------------------------------------------------------------
The machine epsilon

The machine epsilon is the smallest number that changes the result of an
addition operation at the point where the representation of the numbers
is the densest.

"The densest" grates, first because you don't define what
"dense" means, and second because under the only reasonable
definition I can think of the statement is false. (Distinct
FP values are "denser" at smaller magnitudes, and FP numbers
support magnitudes considerably smaller than unity.)

In IEEE754 representation this number has an exponent
value of the bias, and a fraction of 1.

If by "the bias" you mean the traditional offset in the
encoding of the exponent, then I think this statement is wrong.
An FP epsilon has to do with the precision of the fraction,
not with the span of possible exponents.

If you add a number smaller than
this to 1.0, the result will be 1.0. For the different representations
we have in the standard header <float.h>:

Finally, we get to an understandable definition: x is the
FP epsilon if 1+x is the smallest representable number greater
than 1 (when evaluated in the appropriate type). Why not just
start with this "practitioner's definition" instead of with the
ambiguities and misstatements? In addition to the virtue of
being a correct definition, it also captures the notion of the
"graininess" of FP numbers, and hence the reason why someone
would be interested in the value of epsilon.

 

[Richard Tobin]

In IEEE754 representation this number has an exponent
value of the bias, and a fraction of 1.

[/QUOTE]

If by "the bias" you mean the traditional offset in the
encoding of the exponent, then I think this statement is wrong.
An FP epsilon has to do with the precision of the fraction,
not with the span of possible exponents.

The epsilon is such that if it were de-normalised to have the same
exponent as 1.0, only the lowest bit of the fraction would be set. To
normalise it, you need to shift that lowest bit up into the hidden
bit, and adjust the exponent accordingly. For 1.0, the exponent is
equal to the bias. So the epsilon has an exponent equal to the bias
minus the number of fraction bits, and the fraction part is zero
(because of the hidden bit).

-- Richard

 

[Eric Sosman]

If by "the bias" you mean the traditional offset in the
encoding of the exponent, then I think this statement is wrong.
An FP epsilon has to do with the precision of the fraction,
not with the span of possible exponents.

The epsilon is such that if it were de-normalised to have the same
exponent as 1.0, only the lowest bit of the fraction would be set. To
normalise it, you need to shift that lowest bit up into the hidden
bit, and adjust the exponent accordingly. For 1.0, the exponent is
equal to the bias. So the epsilon has an exponent equal to the bias
minus the number of fraction bits, and the fraction part is zero
(because of the hidden bit).[/QUOTE]

Right: Not equal to the bias, as I said. (And in any case
it's wrong to speak of the exponent *value* being equal to the
bias or to the bias minus delta: the exponent is 1-delta and
the bias crops up in the *encoding* of that value.)

 

[Richard Tobin]

(And in any case
it's wrong to speak of the exponent *value* being equal to the
bias or to the bias minus delta: the exponent is 1-delta and
the bias crops up in the *encoding* of that value.)

Surely you mean 0-delta?

It does seem to be common to call the encoded value the exponent; I
suppose the context usually makes it clear. But I agree that it's
inaccurate - I should have said the "exponent bits" or the "encoded
exponent" or something like that.

-- Richard

 

[Army1987]

The C standard paragraph J.5.6: Common extensions:
J.5.6 Other arithmetic types
Additional arithmetic types, such as _ _int128 or double double, and
their appropriate conversions are defined (6.2.5, 6.3.1). Additional
floating types may have more range or precision than long double, may be
used for evaluating expressions of other floating types, and may be
used to define float_t or double_t.

Where does it allow you to use an identifier which doesn't begin
with an underscore and the standard never reserves?
What happens if i write a program with the lines
#include <float.h>
int main(int QFLT_EPSILON, char *argv[])
(yes, I'm allowed to do that) and try to compile it with your
compiler?

 

[jacob navia]

If by "the bias" you mean the traditional offset in the
encoding of the exponent, then I think this statement is wrong.
An FP epsilon has to do with the precision of the fraction,
not with the span of possible exponents.

The epsilon is such that if it were de-normalised to have the same
exponent as 1.0, only the lowest bit of the fraction would be set. To
normalise it, you need to shift that lowest bit up into the hidden
bit, and adjust the exponent accordingly. For 1.0, the exponent is
equal to the bias. So the epsilon has an exponent equal to the bias
minus the number of fraction bits, and the fraction part is zero
(because of the hidden bit).

-- Richard[/QUOTE]

That is what I am trying to say. The epsilon is
sign: 0
exponent: zero, i.e. the bias.
mantissa: 0000000000 0000000000 0000000000 0000000000 0000000000 01
<--10-->
Using printf:
printf("%a",DBL_EPSILON); 0x1.0000000000000p-052

 

[Richard Tobin]

That is what I am trying to say. The epsilon is
sign: 0
exponent: zero, i.e. the bias.
mantissa: 0000000000 0000000000 0000000000 0000000000 0000000000 01

If you put those bits into a double, you don't get the epsilon (try
it). You are writing it as a de-normalised representation, but if you
put those bits in a double they would be interpreted as a normalized
value, equal to 1+epsilon.

The IEEE representation of DBL_EPSILON is

sign bit: 0
exponent bits: 01111001011 (representing -52)
mantissa bits: 000.... (representing 1.0, because of the hidden bit)

-- Richard

 

[jacob navia]

mantissa: 0000000000 0000000000 0000000000 0000000000 0000000000 01

This is wrong, excuse me. The mantissa is normalized, and the bits are:
100000000 etc

The value is 2^-52--> 2.2204460492503130808472633361816e-16

 

[Eric Sosman]

Eric Sosman wrote On 06/29/07 07:31,:

[...]
If you add a number smaller than
this to 1.0, the result will be 1.0. For the different representations
we have in the standard header <float.h>:

Finally, we get to an understandable definition: x is the
FP epsilon if 1+x is the smallest representable number greater
than 1 (when evaluated in the appropriate type). [...]

Now that I think of it, the two descriptions are
not the same. Mine is correct, as far as I know, but
Jacob's is subtly wrong. (Hint: Rounding modes.)

 

[Eric Sosman]

Richard Tobin wrote On 06/29/07 08:53,:

Surely you mean 0-delta?

I don't think so. The fraction of a normalized, non-
zero, finite IEEE number has a value 0.5 <= f < 1, so
unity is represented as two to the first times one-half:
2^1 * .100...000(2). The unbiased exponent value in the
representation of unity is therefore one, not zero.

Of course, omitting any definition of "delta" leaves
quite a bit of wiggle room. ;-)

 

[Richard Tobin]

I don't think so. The fraction of a normalized, non-
zero, finite IEEE number has a value 0.5 <= f < 1, so
unity is represented as two to the first times one-half:
2^1 * .100...000(2). The unbiased exponent value in the
representation of unity is therefore one, not zero.

I was interpreting the hidden bit as representing 1 so that with the
fraction bits the mantissa is 1.f, i.e. in the range [1,2), and
interpreting an exponent field with bits 0111... as representing 0.
This seems to be the way it's usually described in the web pages I
just searched. But it is equivalent to say that the mantissa is 0.1f
and the exponent is one greater.

-- Richard