Precision and ==

Discussion in 'Ruby' started by LAMBEAU Bernard, Apr 7, 2010.

  1. [Note: parts of this message were removed to make it a legal post.]

    We all know that finite representation comes with some difficulties...
    but...

    x = 45.0*(Math::pI / 180.0)
    y = 0.785398163397448
    puts "#{x.class} : #{x}"
    puts "#{y.class} : #{y}"
    puts 45.0*(Math::pI / 180.0) == 0.785398163397448

    prints

    Float : 0.785398163397448
    Float : 0.785398163397448
    false

    Does to_s show something less precise than the internal representation of
    Floats?

    thx
    Bernard
    LAMBEAU Bernard, Apr 7, 2010
    #1
    1. Advertising

  2. On Wed, Apr 7, 2010 at 1:02 PM, LAMBEAU Bernard <> wrote:
    > We all know that finite representation comes with some difficulties...
    > but...
    >
    > x = 45.0*(Math::pI / 180.0)
    > y = 0.785398163397448
    > puts "#{x.class} : #{x}"
    > puts "#{y.class} : #{y}"
    > puts 45.0*(Math::pI / 180.0) == 0.785398163397448
    >
    > prints
    >
    > Float : 0.785398163397448
    > Float : 0.785398163397448
    > false
    >
    > Does to_s show something less precise than the internal representation of
    > Floats?


    Yes, it rounds to 15 decimal positions:

    irb(main):001:0> x = 45.0*(Math::pI / 180.0)
    => 0.785398163397448
    irb(main):002:0> y = 0.785398163397448
    => 0.785398163397448
    irb(main):003:0> puts "#{x.class} : #{"%.20f" % x}"
    Float : 0.78539816339744827900
    => nil
    irb(main):004:0> puts "#{y.class} : #{"%.20f" % y}"
    Float : 0.78539816339744794593

    Jesus.
    Jesús Gabriel y Galán, Apr 7, 2010
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Vincent Lefevre

    XPath/XSLT and extended precision

    Vincent Lefevre, Aug 21, 2003, in forum: XML
    Replies:
    4
    Views:
    888
    Vincent Lefevre
    Aug 21, 2003
  2. Erik2000
    Replies:
    3
    Views:
    30,855
    Karl Heinz Buchegger
    Jun 27, 2003
  3. john smith
    Replies:
    1
    Views:
    526
    tom_usenet
    Aug 5, 2003
  4. Knackeback

    double and precision

    Knackeback, Aug 29, 2003, in forum: C++
    Replies:
    0
    Views:
    446
    Knackeback
    Aug 29, 2003
  5. Roger Leigh
    Replies:
    4
    Views:
    458
    Roger Leigh
    Nov 12, 2003
Loading...

Share This Page