Floating Point Accuracy

[Jon Rea]

Hi,

I have just tried compiling some scientific calculation code on Redhad
Linux using gcc-3.4.4 that I normally compile on MS Visual C++ 03 and
05. I was surprised to get different floating point answers on the two
systems (identical hardware spec AMD Athlon). Both are being compiled in
'Debug' mode with things like Fast_Math disabled.

After a little debuging and lots of printf() statements, all my errors
are found to be cumulative floating point rounding errors.

Most data is read in as floats and then casted to doubles for internal
matrix calculations, before being casted back to floats for storage and
output.

Questions:

1) What is the best way to deal with floating point behavior differences
on different systems?
2) In a printf("%N.Nf\n",myVal); style statement, what is the highest
number of decimal places that one can be accurate to? %8.3f / %9.4f /
%10.5f ???
3) There is a PRAGMA on VC to allow function level setting of
FloatingPoint compile mode so that you can set Fast/Precise for
individual functions. Is there a cross-platform way to do this?

Many thanks for any help,
There is quite a lot of hair loss over here ;-)
Jon Rea

 

[Victor Bazarov]

I have just tried compiling some scientific calculation code on Redhad
Linux using gcc-3.4.4 that I normally compile on MS Visual C++ 03 and
05. I was surprised to get different floating point answers on the two
systems [...]

Questions:

1) What is the best way to deal with floating point behavior
differences on different systems? [...]

http://www.google.com/search?hl=en&q=what+every+computer+scientist+should+know+about+floating+point

V

 

[osmium]

1) What is the best way to deal with floating point behavior differences
on different systems?

As far as I know you just live with it. I don't think there is any way to
get each bit to agree on two different platforms. You write your code such
that you can live with the answers in the least precise form. You have
already been given a link on that.

 

[Jon Rea]

As far as I know you just live with it. I don't think there is any way to
get each bit to agree on two different platforms. You write your code such
that you can live with the answers in the least precise form. You have
already been given a link on that.

Many thanks for your rapid replies, very interesting! Ive had a skim
read of that article, but I'll go back and have a full read later.

I originally thought that differences in floating point were derived
from differences in the way a given hardware architecture dealt with the
number, i.e. the special floating point parts of the processor core.
Still not sure I totally understand why there should therefore be any
differences between Linux and Windows on the same hardware... Ill have
to read more ;-)

Cheers,
Jon

 

[Ben Pope]

I originally thought that differences in floating point were derived
from differences in the way a given hardware architecture dealt with the
number, i.e. the special floating point parts of the processor core.
Still not sure I totally understand why there should therefore be any
differences between Linux and Windows on the same hardware... Ill have
to read more ;-)

If you add a small number to a big number, many times, they might
disappear. If you add together several small numbers and then add that
to the big number, then it might make a difference. Hence, the order of
addition of numbers becomes relevant if they are of significantly
differing magnitudes. By the time you do a few multiplications and
additions, you can be quite a way off target before you know it.

Compilers can decide to reorder the additions (or not) as they see fit,
so it's quite possible that a different compiler, or even compiler
version will produce differing results.

Performing calculations on floating point numbers is like moving piles
of sand, every time you do it, you lose a bit of sand and pick up a bit
of dirt.

Ben Pope

 

[red floyd]

Hi,

I have just tried compiling some scientific calculation code on Redhad
Linux using gcc-3.4.4 that I normally compile on MS Visual C++ 03 and
05. I was surprised to get different floating point answers on the two
systems (identical hardware spec AMD Athlon). Both are being compiled in
'Debug' mode with things like Fast_Math disabled.

Depends. Are you using float, double, or long double?

AFAIK, on MS and G++ float is IEEE 32 bit, and double is IEEE 64 bits.
You run into differences with long double, though. MS makes it 64 bits,
while g++ uses a (denormalized) 80 bit value.

 

[Jon Rea]

Depends. Are you using float, double, or long double?

AFAIK, on MS and G++ float is IEEE 32 bit, and double is IEEE 64 bits.
You run into differences with long double, though. MS makes it 64 bits,
while g++ uses a (denormalized) 80 bit value.

I'm just using floats and doubles. I'm getting differences between MS
and g++ even though they are both supposed to use the same 'IEEE 32 bit'
standard.

 

[osmium]

I'm just using floats and doubles. I'm getting differences between MS and
g++ even though they are both supposed to use the same 'IEEE 32 bit'
standard.

AFAIK there is no requirement that the output routines have to provide the
actual value represented in the variable. The days of printing 2.99999999
instead of 3 are over and my guess is that this was accomplished by a little
judicious juggling in the conversion to characters. If you are determined
to resolve this, you might look at the data in hex to see what's really
there.

 

[Walter Bright]

I'm just using floats and doubles. I'm getting differences between MS
and g++ even though they are both supposed to use the same 'IEEE 32 bit'
standard.

You can still get differences because some compilers generate code that
retains full 80 bit precision on intermediate values, whereas others
truncate intermediate values to 64 bits. Both are allowed by the
standard, but the latter implementation is a bad idea as far as I'm
concerned.

The Digital Mars C++ compiler fully supports 80 bit long doubles, and
also does all temporaries to full 80 bit precision. The CPU supports 80
bit precision, why not use it?

-Walter Bright
www.digitalmars.com C, C++, D programming language compilers