# number accuracy

Discussion in 'Perl Misc' started by kencavagnolo@gmail.com, Dec 13, 2006.

1. ### Guest

I was digging through some code this morning after encountering a
program error and I came across the following...

For this bit of code:
#-------------------------------------------------------#
\$eta = 0.5;
\$etacrit = 0.97;
while (\$eta <= \$etacrit) {
(\$eta < 0.80) ? (\$etastep = 0.1) :
(\$eta < 0.95) ? (\$etastep = 0.05) :
(\$etastep = 0.01);
\$eta += \$etastep;
}
#-------------------------------------------------------#

Which *should* output:
0.6
0.7
0.8
0.85
0.9
0.95
0.96
0.97

It instead was outputing:
0.6
0.7
0.8
0.9
0.95
0.96
0.97

So I cut the code out of the main program and had it print to the
floating point accuracy at the beginning of the loop and found this:
0.100000000000000005551115123126
0.200000000000000011102230246252
0.300000000000000044408920985006
0.400000000000000022204460492503
0.500000000000000000000000000000
0.599999999999999977795539507497
0.699999999999999955591079014994
0.799999999999999933386618522491
0.899999999999999911182158029987
0.949999999999999955591079014994
0.959999999999999964472863211995
0.969999999999999973354647408996

So I now see the reason why the < 0.8 comparison failed because the
value of \$eta is actually 0.7999999 and not 0.8. I know there are fixes
for this such as making the comparions with 0.79, using a sprintf,
rounding, truncating, et cetera; but my officemate checked this code on
his machine and we got the same result.

My question is... WHY? Why is the addition below 0.5 of a number which
is slightly above 0.1, but then above 0.5 the addition becomes
something which is slightly less than 0.1? And, how does one fix (if
that's possible) this problem?

Thanks.

, Dec 13, 2006

2. ### Paul LalliGuest

wrote:
> My question is... WHY? Why is the addition below 0.5 of a number which
> is slightly above 0.1, but then above 0.5 the addition becomes
> something which is slightly less than 0.1? And, how does one fix (if
> that's possible) this problem?

This is not specific to Perl, but is a problem with computers in
perldoc -q 999
perldoc perlnumber

Paul Lalli

Paul Lalli, Dec 13, 2006

3. ### Guest

wrote:

> So I now see the reason why the < 0.8 comparison failed because the
> value of \$eta is actually 0.7999999 and not 0.8.

perldoc -q 9999

--
The best way to get a good answer is to ask a good question.
David Filmer (http://DavidFilmer.com)

, Dec 13, 2006
4. ### Charlton WilburGuest

>>>>> "k" == kencavagnolo <> writes:

k> My question is... WHY? Why is the addition below 0.5 of a
k> number which is slightly above 0.1, but then above 0.5 the
k> addition becomes something which is slightly less than 0.1?

perldoc -q 'long decimal'
http://docs.sun.com/source/806-3568/ncg_goldberg.html

k> And, how does one fix (if that's possible) this problem?

Start by understanding why and how it happens.

Charlton

--
Charlton Wilbur

Charlton Wilbur, Dec 13, 2006
5. ### KenGuest

Thank you all very much.

The fix is in and everything works fine.

Ken, Dec 13, 2006
6. ### Tad McClellanGuest

<> wrote:
> I was digging through some code this morning after encountering a
> program error and I came across the following...
>
> For this bit of code:
> #-------------------------------------------------------#
> \$eta = 0.5;
> \$etacrit = 0.97;
> while (\$eta <= \$etacrit) {
> (\$eta < 0.80) ? (\$etastep = 0.1) :
> (\$eta < 0.95) ? (\$etastep = 0.05) :
> (\$etastep = 0.01);
> \$eta += \$etastep;
> }
> #-------------------------------------------------------#
>
> Which *should* output:

No it shouldn't.

The code contains no statements that should make output,
so it should output nothing.

--
Tad McClellan SGML consulting
Perl programming
Fort Worth, Texas

Tad McClellan, Dec 14, 2006