Strange Conversion Error from double to int

[Barry]

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??

Thanks,

Barry.

 

[JKop]

Barry posted:

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

double di = 0.0;

do
{
string.Format( "%li", int(di) );

pDC->TextOut( left +int(di), 100, string );

di += 51.2;

} while ( di <= 512.0 )


Now isn't that pretty.


-JKop

 

[Peter van Merkerk]

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??

Some numbers cannot be exactly represented in floating point format.
Therefore adding 51.2 ten times does necessarily not yield exactly 512.
It may in fact result in 511.999999999999999 being stored in di. Since
string.Format() is not a standard C++ function you will have to lookup
the documentation of that function yourself. Apparently this function
does do some rounding. Considering it displays only five digits behind
the decimal point (the accuracy a double is much higher than that)
512.00000 is probably a closer representation of what is in the di
variable than 511.99999.

If you need exact results don't use floating point numbers.

 

[John Harrison]

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??

Thanks,

Barry.

Because floating point arithmetic is not exact, and unless you really,
really know what you are doing you shouldn't assume that it is going to be.

In your case there is no exact floating point representation of the number
51.2 and so errors accumulate.

john

 

[Karl Heinz Buchegger]

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

replace that with

string.Format("%.14lf",di);

and see for yourself

You might also want to study
http://www.petebecker.com/js200006.html

 

[Peter van Merkerk]

Some numbers cannot be exactly represented in floating point format.
Therefore adding 51.2 ten times does necessarily not yield exactly 512.

Correction: does *not* necessarily yield exactly 512.

 

[SaltPeter]

The following code compiled using MSVC++6.0 -

double di=0.0;
for(;di<=512.0
{
string.Format("%li",int(di));
pDC-> TextOut( left +int(di), 100,string);

di += 51.2;
}

gives the following unexpected output -

0 51 102 153 204 256 307 358 409 460 511

Why 511 and not 512??

Replacing with -

string.Format("%lf",di);

gives 512.00000

so why is the casting of 512.0 {int(di)} giving 511??

That should be expected. since one can argue that any double where d >=
511.0 and d < 512.0 is converted to an integer with the value of 511. Since
floating numbers are designed to express such values as 2/3 or
0.666666666667, its perfectly legal for 512.0 to be represented by
511.999999999999.

The problem here is not the representation or conversion of the double. The
problem lies in the code. For starters, a floating number should never be
accompanied with a comparison operator of any kind (d <= 512.0) in the
context of a loop condition. In this case, if d were an int, your processor
would be saved from a tremendous amount of useless computations.

The comparison in following code is guarenteed (and expected) to fail:

int main()
{
double d = 0.0;

for (int i = 0; i < 10; i++)
{
d += 51.2;
}

double dx = 512.0;

if (dx != d)
{
std::cout << "comparison failure, d = " << d << std::endl;
}

return 0;
}