1.090516455488E9 / 1000000.000 ???

M

Math

Hello,

I got a simple and probably stupid newby question..
When I compute:
1.090516455488E9 / 1000000
the result is 1090516455.49
Should be 1090516455.488

I know something with float and //...

Anybody?
How do I get correct number?
 
G

Georg Brandl

Math said:
Hello,

I got a simple and probably stupid newby question..
When I compute:
1.090516455488E9 / 1000000
the result is 1090516455.49
Should be 1090516455.488

I know something with float and //...

Anybody?
How do I get correct number?

Python 2.4.2 (#1, Mar 12 2006, 00:14:41)
[GCC 3.4.5 (Gentoo 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.1090516455.49

print is using str() which formats the float number with 12 digits
precision and therefore rounds the result.
repr() however, which is used by the interpreter when printing
out expression results, uses 17 digits precision which is why
you can see how the result is stored in the computer's memory
(1090516455.487999999999999....).

Georg
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top