number accuracy

Discussion in 'Perl Misc' started by kencavagnolo@gmail.com, Dec 13, 2006.

  1. Guest

    I was digging through some code this morning after encountering a
    program error and I came across the following...

    For this bit of code:
    #-------------------------------------------------------#
    $eta = 0.5;
    $etacrit = 0.97;
    while ($eta <= $etacrit) {
    ($eta < 0.80) ? ($etastep = 0.1) :
    ($eta < 0.95) ? ($etastep = 0.05) :
    ($etastep = 0.01);
    $eta += $etastep;
    }
    #-------------------------------------------------------#

    Which *should* output:
    0.6
    0.7
    0.8
    0.85
    0.9
    0.95
    0.96
    0.97

    It instead was outputing:
    0.6
    0.7
    0.8
    0.9
    0.95
    0.96
    0.97

    So I cut the code out of the main program and had it print to the
    floating point accuracy at the beginning of the loop and found this:
    0.100000000000000005551115123126
    0.200000000000000011102230246252
    0.300000000000000044408920985006
    0.400000000000000022204460492503
    0.500000000000000000000000000000
    0.599999999999999977795539507497
    0.699999999999999955591079014994
    0.799999999999999933386618522491
    0.899999999999999911182158029987
    0.949999999999999955591079014994
    0.959999999999999964472863211995
    0.969999999999999973354647408996

    So I now see the reason why the < 0.8 comparison failed because the
    value of $eta is actually 0.7999999 and not 0.8. I know there are fixes
    for this such as making the comparions with 0.79, using a sprintf,
    rounding, truncating, et cetera; but my officemate checked this code on
    his machine and we got the same result.

    My question is... WHY? Why is the addition below 0.5 of a number which
    is slightly above 0.1, but then above 0.5 the addition becomes
    something which is slightly less than 0.1? And, how does one fix (if
    that's possible) this problem?

    Thanks.
    , Dec 13, 2006
    #1
    1. Advertising

  2. Paul Lalli Guest

    wrote:
    > My question is... WHY? Why is the addition below 0.5 of a number which
    > is slightly above 0.1, but then above 0.5 the addition becomes
    > something which is slightly less than 0.1? And, how does one fix (if
    > that's possible) this problem?


    This is not specific to Perl, but is a problem with computers in
    general. Please read:
    perldoc -q 999
    perldoc perlnumber

    Paul Lalli
    Paul Lalli, Dec 13, 2006
    #2
    1. Advertising

  3. Guest

    wrote:

    > So I now see the reason why the < 0.8 comparison failed because the
    > value of $eta is actually 0.7999999 and not 0.8.


    perldoc -q 9999

    --
    The best way to get a good answer is to ask a good question.
    David Filmer (http://DavidFilmer.com)
    , Dec 13, 2006
    #3
  4. >>>>> "k" == kencavagnolo <> writes:

    k> My question is... WHY? Why is the addition below 0.5 of a
    k> number which is slightly above 0.1, but then above 0.5 the
    k> addition becomes something which is slightly less than 0.1?

    perldoc -q 'long decimal'
    http://docs.sun.com/source/806-3568/ncg_goldberg.html

    k> And, how does one fix (if that's possible) this problem?

    Start by understanding why and how it happens.

    Charlton


    --
    Charlton Wilbur
    Charlton Wilbur, Dec 13, 2006
    #4
  5. Ken Guest

    Thank you all very much.

    The fix is in and everything works fine.
    Ken, Dec 13, 2006
    #5
  6. <> wrote:
    > I was digging through some code this morning after encountering a
    > program error and I came across the following...
    >
    > For this bit of code:
    > #-------------------------------------------------------#
    > $eta = 0.5;
    > $etacrit = 0.97;
    > while ($eta <= $etacrit) {
    > ($eta < 0.80) ? ($etastep = 0.1) :
    > ($eta < 0.95) ? ($etastep = 0.05) :
    > ($etastep = 0.01);
    > $eta += $etastep;
    > }
    > #-------------------------------------------------------#
    >
    > Which *should* output:



    No it shouldn't.

    The code contains no statements that should make output,
    so it should output nothing.

    :)


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Dec 14, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?Q2hhcmxlc0E=?=

    Accuracy and CSS

    =?Utf-8?B?Q2hhcmxlc0E=?=, Jan 16, 2006, in forum: ASP .Net
    Replies:
    5
    Views:
    665
  2. PercyP
    Replies:
    5
    Views:
    553
    PercyP
    Oct 20, 2004
  3. Replies:
    15
    Views:
    629
    David Harmon
    May 5, 2005
  4. Replies:
    1
    Views:
    413
    Jonathan Mcdougall
    May 11, 2005
  5. aurora
    Replies:
    3
    Views:
    392
    aurora
    Feb 2, 2005
Loading...

Share This Page