Inconsistent results from int(floatNumber)

[gershar]

I had some problems with some Python projects that gave variable
results that I could not track down. Eventually and reluctantly I
converted them to Java. Later, when I had more time I tried to analyze
what the Python code was doing and found something strange. The
following snippet illustrates the problem.
i += 0.1
z = i * 10.0
print
print z
print int(z)

-499.0
-499

-498.0
-498

-497.0
-496

-496.0
-495

-495.0
-494


The first two iterations look OK but after that the int(z) function
returns the wrong value. It looks like the value was rounded down. If
a just do this:-497
I get the value I expect.
So what is the problem?

It looks like a rounding problem but on the surface there is nothing
to round. I am aware that there are rounding limitations with floating
point arithmetic but the value passed to int() is always correct. What
would cause it to be off by 1 full digit in only some cases? Perhaps
something behind the scenes in the bowels of the interpreter ?.

I could not get the thing to fail without being inside the for loop;
does that have something to do with it?

To fix the problem I could use round() or math.floor(). Like this.
i += 0.1
z = i * 10.0
print
print z
print(round(z))

-499.0
-499.0

-498.0
-498.0

-497.0
-497.0

-496.0
-496.0

-495.0
-495.0

Why should I have to do this?

Is there a general rule of thumb to know when this could be a problem?

Should any float-to-int conversion be suspect?

The above code was run in Python 2.5.4 on WinXP and Python 2.6.2 on
Linux(Fedora12)
Can anyone verify if this would be the same on 3.x?

 

[Ian]

It looks like a rounding problem but on the surface there is nothing
to round. I am aware that there are rounding limitations with floating
point arithmetic but the value passed to int() is always correct.

No, it isn't:
.... i += 0.1
.... z = i * 10.0
.... print
.... print z
.... print repr(z)
.... print int(z)
....

-499.0
-499.0
-499

-498.0
-498.0
-498

-497.0
-496.99999999999994
-496

-496.0
-495.99999999999994
-495

-495.0
-494.99999999999994
-494

Cheers,
Ian

 

[Dave Angel]

I had some problems with some Python projects that gave variable
results that I could not track down. Eventually and reluctantly I
converted them to Java. Later, when I had more time I tried to analyze
what the Python code was doing and found something strange. The
following snippet illustrates the problem.

i += 0.1
z = i * 10.0
print
print z
print int(z)

-499.0
-499

-498.0
-498

-497.0
-496

-496.0
-495

-495.0
-494


The first two iterations look OK but after that the int(z) function
returns the wrong value. It looks like the value was rounded down. If

No, the value is truncated upward, towards zero. Down would be towards
negative infinity. And rounding would work, that's not what int() does.

a just do this:
-497
I get the value I expect.
So what is the problem?

It looks like a rounding problem but on the surface there is nothing
to round. I am aware that there are rounding limitations with floating
point arithmetic but the value passed to int() is always correct.

Define "correct." If you mean that it's exactly an integer, that's
false. It's off a little, because 0.1 isn't exact, and multiplying by
10 doesn't fix it.

What
would cause it to be off by 1 full digit in only some cases? Perhaps
something behind the scenes in the bowels of the interpreter ?.

I could not get the thing to fail without being inside the for loop;
does that have something to do with it?

To fix the problem I could use round() or math.floor(). Like this.

round() is entirely different than truncating or flooring.

i += 0.1
z = i * 10.0
print
print z
print(round(z))

-499.0
-499.0

-498.0
-498.0

-497.0
-497.0

-496.0
-496.0

-495.0
-495.0

Why should I have to do this?

Is there a general rule of thumb to know when this could be a problem?

Should any float-to-int conversion be suspect?

The above code was run in Python 2.5.4 on WinXP and Python 2.6.2 on
Linux(Fedora12)
Can anyone verify if this would be the same on 3.x?

This doesn't have anything to do with Python, but everything to do with
binary floating point. The thing that's confusing you most is that you
think that if a value prints as 497.0, that it's actually equal to
497.0. False.

When you have an approximate value like 0.1, and you do arithmetic with
it, sometimes the values aren't exact.

In ython 3, they avoid this particular symptom usually by printing the
values differently. But the problem still exists.


DaveA

 

[Terry Reedy]

I had some problems with some Python projects that gave variable
results that I could not track down. Eventually and reluctantly I
converted them to Java. Later, when I had more time I tried to analyze
what the Python code was doing and found something strange. The
following snippet illustrates the problem.

i += 0.1

The binary float resulting from the conversion of .1 is slightly greater
than .1, so this increases i by slightly more than .1

z = i * 10.0

so z is increased be lightly more that 1

print
print z
print int(z)

float.__int__ truncates toward 0.

-499.0
-499

-498.0
-498

-497.0
-496

And here the extra increase shows up.

-496.0
-495

-495.0
-494

It looks like a rounding problem but on the surface there is nothing
to round. I am aware that there are rounding limitations with floating
point arithmetic but the value passed to int() is always correct.

No it is not. To see this, print more digits (which themselves are
approximations of the actual binary value). With 3.1.2 on x86 system:

i = -50.0
form = '{:24.18f}'.format
print(form(.1))
for x in range(15):
i += 0.1
z = i * 10.0
print()
print(form(z))
print(int(z))
0.100000000000000006

-499.000000000000000000
-499

-498.000000000000000000
-498

-496.999999999999943157
-496

-495.999999999999943157
-495

-494.999999999999943157
-494

To completely understand, you would have to look at the binary bit
pattern and know the exact behavior of floating point arithmetic on a
system and the exact decimal to binary to decimal conversion algorithm.

 

[Ian]

The binary float resulting from the conversion of .1 is slightly greater
than .1, so this increases i by slightly more than .1


so z is increased be lightly more that 1

It should also be pointed out that Java in fact does the same thing:

$ cat Test.java
class Test
{
public static void main(String[] argv)
{
double i = -50.0;
for (int x = 0; x < 5; x++)
{
i += 0.1;
double z = i * 10.0;
System.out.println("");
System.out.println(z);
System.out.println((int) z);
}
}
}
$ javac Test.java
$ java Test

-499.0
-499

-498.0
-498

-496.99999999999994
-496

-495.99999999999994
-495

-494.99999999999994
-494

Cheers,
Ian

 

[gershar]

I had some problems with some Python projects that gave variable
results that I could not track down. Eventually and reluctantly I
converted them to Java. Later, when I had more time I tried to analyze
what the Python code was doing and found something strange. The
following snippet illustrates the problem.


        i += 0.1
        z = i * 10.0
        print
        print z
        print int(z)

-499.0
-499

-498.0
-498

-497.0
-496

-496.0
-495

-495.0
-494

The first two iterations look OK but after that the int(z) function
returns the wrong value. It looks like the value was rounded down.  If
a just do this:>>> int(-497.0)

-497
I get the value I expect.
So what is the problem?

It looks like a rounding problem but on the surface there is nothing
to round. I am aware that there are rounding limitations with floating
point arithmetic but the value passed to int() is always correct. What
would cause it to be off by 1 full digit in only some cases? Perhaps
something behind the scenes in the bowels of the interpreter ?.

I could not get the thing to fail without being inside the for loop;
does that have something to do with it?

To fix the problem I could use round() or math.floor().  Like this.


        i += 0.1
        z = i * 10.0
        print
        print z
        print(round(z))

-499.0
-499.0

-498.0
-498.0

-497.0
-497.0

-496.0
-496.0

-495.0
-495.0

Why should I have to do this?

Is there a general rule of thumb to know when this could be a problem?

Should any float-to-int conversion be suspect?

The above code was run in Python 2.5.4 on WinXP and Python 2.6.2 on
Linux(Fedora12)
Can anyone verify if this would be the same on 3.x?

Good responses from this group!
Thanks for the insight

Regards