Math.round question

R

Richard Cranium

Using the number 1.015 [many numbers having a 5 at the third decimal point
could be used but not all]

*****************************
the following script returns 1.02, which is correct

var T;
var S=new String(Math.round(parseFloat(1.015).toFixed(2)*100));
while (S.length<2) S='0'+S
var temp2 = S.substr(0, T=(S.length-2)) + '.' + S.substr(T, 2);
temp2 = 1.02

but it causes .005 - .009 to return .00, which is wrong
*****************************



******************************
the following script returns 1.01, which is wrong

var T;
var S=new String(Math.round(parseFloat(1.015)*100));
while (S.length<2) S='0'+S
var temp2 = S.substr(0, T=(S.length-2)) + '.' + S.substr(T, 2);
temp2 = 1.01

but it causes .001 - .004 to return .00, which is correct
and .005 - .009 to return .01, which is also correct
******************************


Do you have any ideas as to why it is this way, and if there is a better
[more exact] way to round numbers, these methods seem a little inconsistent.
I'll use method 1 because anything past .009 rounds correctly but what's up
with .005 - .009 rounding to .00 and not .01.
Method 2 just has way to many numbers that round incorrectly.

--
 
H

Hans Kesting

Using the number 1.015 [many numbers having a 5 at the third decimal point
could be used but not all]

*****************************
the following script returns 1.02, which is correct

var T;
var S=new String(Math.round(parseFloat(1.015).toFixed(2)*100));
while (S.length<2) S='0'+S
var temp2 = S.substr(0, T=(S.length-2)) + '.' + S.substr(T, 2);
temp2 = 1.02

but it causes .005 - .009 to return .00, which is wrong
*****************************

******************************
the following script returns 1.01, which is wrong

var T;
var S=new String(Math.round(parseFloat(1.015)*100));
while (S.length<2) S='0'+S
var temp2 = S.substr(0, T=(S.length-2)) + '.' + S.substr(T, 2);
temp2 = 1.01

but it causes .001 - .004 to return .00, which is correct
and .005 - .009 to return .01, which is also correct
******************************

Do you have any ideas as to why it is this way, and if there is a better
[more exact] way to round numbers, these methods seem a little inconsistent.
I'll use method 1 because anything past .009 rounds correctly but what's up
with .005 - .009 rounding to .00 and not .01.
Method 2 just has way to many numbers that round incorrectly.

Two guesses:

1) "bankers rounding" is used, where a ".5" is rounded to the nearest
EVEN integer
2) 1.015 isn't 1.015 but 1.01499 (or something like that)

Is this jscript.net? You might want to ask in a jscript.net specific
group. You question has more to do with the exact workings of the
language than with asp.net.

Hans Kesting
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,774
Messages
2,569,598
Members
45,147
Latest member
CarenSchni
Top