Decimal precision ?

Discussion in 'ASP .Net' started by Piotr, Apr 2, 2004.

  1. Piotr

    Piotr Guest

    Hi

    I'm working with Datagrid containg decimal (editable) numbers. The
    user-entered numbers are read by decimal.Parse() method. However, Datagrid
    formats some decimals as "1", but some as (eg) "1.00" - depends on what the
    user had entered. The page has DISABLED viewstate.

    So, let someone explain me why the number 1 is not the same as 1.0 or 1.000?
    I cannot find any property to set the precision.

    The number of decimal digits can be different in different rows, so no
    static format option can be used. Currently the only option to get correct
    formatting is (a bit stupid) construction like this:
    ((decimal)DataBinder.Eval(Container.DataItem, "Quantity")).ToString("F" +
    ((int)DataBinder.Eval(Container.DataItem, "DecimalPlaces")).ToString()

    Regards.
    Piotr, Apr 2, 2004
    #1
    1. Advertising

  2. Hi, Piotr,

    "Piotr" <> wrote in message
    news:c4j6g1$ni8$...
    > Hi
    >
    > I'm working with Datagrid containg decimal (editable) numbers. The
    > user-entered numbers are read by decimal.Parse() method. However, Datagrid
    > formats some decimals as "1", but some as (eg) "1.00" - depends on what

    the
    > user had entered. The page has DISABLED viewstate.
    >
    > So, let someone explain me why the number 1 is not the same as 1.0 or

    1.000?

    There is no difference *between the values* indeed. The difference is in the
    way these are stored.

    1 is stored as 1 / 10^0
    1.0 is stored as 10 / 10^1
    1.000 is stored as 1000 / 10^3

    See:
    http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassTopic.asp

    This code illustrates it:

    decimal d = decimal.One;
    for(int i = 0; i < 20;i++)
    {
    d = -decimal.Parse(d.ToString("F" + i));
    int[] strip = decimal.GetBits(d);
    ulong value = Convert.ToUInt64(
    ((uint)strip[1]).ToString("x") +
    ((uint)strip[0]).ToString("x"), 16);
    Console.WriteLine(
    "{0,-22} = {1,-22} / 10^{2,-3} {3}",
    d,
    value,
    (strip[3] & 0xFF0000) >> 16,
    (((strip[3] >> 31) & 1) > 0 ? "Negative" : "Positive"));
    }

    See the topic on GetBits method:
    http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassGetBitsTopic.asp

    > I cannot find any property to set the precision.
    >
    > The number of decimal digits can be different in different rows, so no
    > static format option can be used. Currently the only option to get correct
    > formatting is (a bit stupid) construction like this:
    > ((decimal)DataBinder.Eval(Container.DataItem, "Quantity")).ToString("F" +
    > ((int)DataBinder.Eval(Container.DataItem, "DecimalPlaces")).ToString()


    You can try setting the DataFormatString property alternatively:

    "F" + DataBinder.Eval(Container.DataItem, "DecimalPlaces").ToString()

    should be fine.

    Hope this helps
    Martin

    >
    > Regards.
    >
    Martin Dechev, Apr 2, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?ISO-8859-1?Q?Mattias_Br=E4ndstr=F6m?=

    Arbitrary precision decimal numbers

    =?ISO-8859-1?Q?Mattias_Br=E4ndstr=F6m?=, Apr 21, 2005, in forum: C++
    Replies:
    5
    Views:
    522
    Alex Vinokur
    Apr 26, 2005
  2. Sam
    Replies:
    2
    Views:
    2,149
  3. Pp

    Precision upto n decimal points

    Pp, Dec 4, 2011, in forum: C Programming
    Replies:
    10
    Views:
    730
    gwowen
    Dec 5, 2011
  4. Replies:
    3
    Views:
    295
  5. Kay Y. Jheallee

    Setting longer default decimal precision

    Kay Y. Jheallee, Nov 18, 2013, in forum: Python
    Replies:
    7
    Views:
    106
    Tim Golden
    Nov 20, 2013
Loading...

Share This Page