Decimal precision ?

P

Piotr

Hi

I'm working with Datagrid containg decimal (editable) numbers. The
user-entered numbers are read by decimal.Parse() method. However, Datagrid
formats some decimals as "1", but some as (eg) "1.00" - depends on what the
user had entered. The page has DISABLED viewstate.

So, let someone explain me why the number 1 is not the same as 1.0 or 1.000?
I cannot find any property to set the precision.

The number of decimal digits can be different in different rows, so no
static format option can be used. Currently the only option to get correct
formatting is (a bit stupid) construction like this:
((decimal)DataBinder.Eval(Container.DataItem, "Quantity")).ToString("F" +
((int)DataBinder.Eval(Container.DataItem, "DecimalPlaces")).ToString()

Regards.
 
M

Martin Dechev

Hi, Piotr,

Piotr said:
Hi

I'm working with Datagrid containg decimal (editable) numbers. The
user-entered numbers are read by decimal.Parse() method. However, Datagrid
formats some decimals as "1", but some as (eg) "1.00" - depends on what the
user had entered. The page has DISABLED viewstate.

So, let someone explain me why the number 1 is not the same as 1.0 or
1.000?

There is no difference *between the values* indeed. The difference is in the
way these are stored.

1 is stored as 1 / 10^0
1.0 is stored as 10 / 10^1
1.000 is stored as 1000 / 10^3

See:
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassTopic.asp

This code illustrates it:

decimal d = decimal.One;
for(int i = 0; i < 20;i++)
{
d = -decimal.Parse(d.ToString("F" + i));
int[] strip = decimal.GetBits(d);
ulong value = Convert.ToUInt64(
((uint)strip[1]).ToString("x") +
((uint)strip[0]).ToString("x"), 16);
Console.WriteLine(
"{0,-22} = {1,-22} / 10^{2,-3} {3}",
d,
value,
(strip[3] & 0xFF0000) >> 16,
(((strip[3] >> 31) & 1) > 0 ? "Negative" : "Positive"));
}

See the topic on GetBits method:
http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassGetBitsTopic.asp
I cannot find any property to set the precision.

The number of decimal digits can be different in different rows, so no
static format option can be used. Currently the only option to get correct
formatting is (a bit stupid) construction like this:
((decimal)DataBinder.Eval(Container.DataItem, "Quantity")).ToString("F" +
((int)DataBinder.Eval(Container.DataItem, "DecimalPlaces")).ToString()

You can try setting the DataFormatString property alternatively:

"F" + DataBinder.Eval(Container.DataItem, "DecimalPlaces").ToString()

should be fine.

Hope this helps
Martin
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,733
Messages
2,569,439
Members
44,829
Latest member
PIXThurman

Latest Threads

Top