Double data type subtraction changes precision

Discussion in 'ASP .Net' started by neerajb@noida.nospamhcltech.com, Feb 13, 2009.

  1. Guest

    I have 3 variables, all Double data type. When I subtract dblA - dblB =
    dblC, what should be a simple number changes to a lot of decimals. For
    example: 2.04 - 1.02 becomes 1.0199999999999999.... Why is this? It seems
    that if dblA and dblB are only 2 decimals, the result should be 2 decimals or
    less... Very critical in the application that I'm working on that this
    number comes out to the expected 2 decimals, but it can vary to any number of
    decimals depending on what values are passed into dblA and dblB.

    This is ridiculous. THis is happening in both .NET 1.1 and .NET 2.0
    , Feb 13, 2009
    #1
    1. Advertising

  2. Guest

    Thanks Mark for the quick response.

    "Mark Rae [MVP]" wrote:

    > ""
    > <> wrote in message
    > news:...
    >
    > > I have 3 variables, all Double data type. When I subtract dblA - dblB =
    > > dblC, what should be a simple number changes to a lot of decimals. For
    > > example: 2.04 - 1.02 becomes 1.0199999999999999.... Why is this? It
    > > seems
    > > that if dblA and dblB are only 2 decimals, the result should be 2 decimals
    > > or
    > > less... Very critical in the application that I'm working on that this
    > > number comes out to the expected 2 decimals, but it can vary to any number
    > > of
    > > decimals depending on what values are passed into dblA and dblB.
    > >
    > > This is ridiculous. THis is happening in both .NET 1.1 and .NET 2.0

    >
    > Completely standard behaviour, not just in .NET but in all computer
    > languages:
    > http://www.yoda.arachsys.com/csharp/floatingpoint.html
    >
    > Use Decimal instead of double...
    >
    >
    > --
    > Mark Rae
    > ASP.NET MVP
    > http://www.markrae.net
    >
    >
    , Feb 14, 2009
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Sydex
    Replies:
    12
    Views:
    6,483
    Victor Bazarov
    Feb 17, 2005
  2. haley
    Replies:
    7
    Views:
    2,993
    Chris Uppal
    Jan 4, 2007
  3. Replies:
    9
    Views:
    701
    Christopher Bazley
    Jan 30, 2010
  4. katrinaVictim@.

    Look at this data issue: Double Precision

    katrinaVictim@., Nov 17, 2005, in forum: ASP General
    Replies:
    3
    Views:
    230
    functoid@.
    Nov 27, 2005
  5. anilgiri2010a

    Set Precision to Variable of type double in ASP

    anilgiri2010a, Jan 25, 2012, in forum: ASP General
    Replies:
    0
    Views:
    879
    anilgiri2010a
    Jan 25, 2012
Loading...

Share This Page