Floating point calculation problem

[Hyun chul Park]

hi.
i'm confused by float point calculation.
like this!

 

[Zoran Regvart]

Hi,

hi.
i'm confused by float point calculation.
like this!

try using BigDecimal, like:

irb(main):001:0> require 'bigdecimal'
=> true
irb(main):002:0> x = BigDecimal("123.6") - BigDecimal("123")
=> #<BigDecimal:8b13c,'0.6E0',4(16)>
irb(main):003:0> x.to_f
=> 0.6

zoran

 

[Hyun chul Park]

Hi,


try using BigDecimal, like:

irb(main):001:0> require 'bigdecimal'
=> true
irb(main):002:0> x = BigDecimal("123.6") - BigDecimal("123")
=> #<BigDecimal:8b13c,'0.6E0',4(16)>
irb(main):003:0> x.to_f
=> 0.6

zoran

thank you, you're reply.

 

[Axel Etzold]

-------- Original-Nachricht --------

Datum: Tue, 8 Jul 2008 17:39:50 +0900
Von: Hyun chul Park <[email protected]>
An: (e-mail address removed)
Betreff: Floating point calculation problem

Hi --

hi.
i'm confused by float point calculation.
like this!

you may want to take a look at this:

http://docs.sun.com/source/806-3568/ncg_goldberg.html

If you want to do exact calculations, you can use fractions

http://www.ruby-doc.org/stdlib/libdoc/rational/rdoc/classes/Rational.html

To convert between the two, for each real number (Float), you can use its continued fraction
approximation.

http://www.math.mtu.edu/mathlab/COURSES/holt/dnt/pell1.html

Here's a link to a Ruby program that calculates digits of Pi from its continued fraction representation
(and also a Java one, which is much longer):

http://www.cs.utsa.edu/~wagner/pi/pi_cont.html

Best regards,

Axel