Arithmetic operation bug

Discussion in 'Ruby' started by Manoj Chourasia, Jul 24, 2009.

  1. v1="128.015"
    puts((v1.to_f*1000).to_i).to_s // giving 128014 instead of 128015

    Why the the result is rounding up to 128014 but the correct result
    should be 128015.
    --
    Posted via http://www.ruby-forum.com/.
    Manoj Chourasia, Jul 24, 2009
    #1
    1. Advertising

  2. the problem seems to be in to_i

    2009/7/24 Manoj Chourasia <>

    > v1=3D"128.015"
    > puts((v1.to_f*1000).to_i).to_s // giving 128014 instead of 128015
    >
    > Why the the result is rounding up to 128014 but the correct result
    > should be 128015.
    > --
    > Posted via http://www.ruby-forum.com/.
    >
    >



    --=20
    -- Mirko Viviani --
    GPG-PGP Public Key: 0xE4E8FAB1
    Fingerprint: 14D3 A373 E926 7737 DF32 502B A4C4 1CE2 E4E8 FAB1
    ***********************************************
    "=93Machines take me by surprise with great frequency.=94 A. Turing
    Mirko Viviani, Jul 24, 2009
    #2
    1. Advertising

  3. Manoj Chourasia

    Chris Hulan Guest

    On Jul 24, 9:11 am, Manoj Chourasia <> wrote:
    > v1="128.015"
    > puts((v1.to_f*1000).to_i).to_s // giving 128014 instead of 128015
    >
    > Why the the result is rounding up to 128014 but the correct result
    > should be 128015.
    > --
    > Posted viahttp://www.ruby-forum.com/.


    Floats are tricky:
    (120..130).each{|x| puts x,((x+0.015)*1000).to_i}

    Have a look at "What Every Computer Scientist Should Know About
    Floating-Point Arithmetic" (http://docs.sun.com/source/806-3568/
    ncg_goldberg.html)
    Chris Hulan, Jul 24, 2009
    #3
  4. On Friday 24 July 2009, Manoj Chourasia wrote:
    > |v1=3D"128.015"
    > |puts((v1.to_f*1000).to_i).to_s // giving 128014 instead of 128015
    > |
    > |Why the the result is rounding up to 128014 but the correct result
    > |should be 128015.


    I'm not an expert, but I think because when you write 128.015, the number y=
    ou=20
    actually get is 128.0149999999999863575... (because 128.015 can't be=20
    represented as a number with a finite number of decimal digits in binary, j=
    ust=20
    like, for example, 1/17 can't be repsented with a finite decimal number in=
    =20
    decimal). You usually don't see this because by default only a small number=
    of=20
    decimal digits are shown, taking rounding into account. You can obtain a=20
    greater number of digits using, for example, format:

    puts format("%.20f", 128.015)
    =3D> 128.01499999999998635758

    Now, when you multiply the number by 1000, what you actually get is:=20
    128014.99999999998544808477. And, since Float#to_i truncates the number (it=
    =20
    doesn't round it), you get the number 128014.

    I hope this helps

    Stefano
    Stefano Crocco, Jul 24, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. joshc
    Replies:
    5
    Views:
    533
    Keith Thompson
    Mar 31, 2005
  2. Mehta Shailendrakumar

    Arithmetic shift operation in C

    Mehta Shailendrakumar, Jun 15, 2005, in forum: C Programming
    Replies:
    43
    Views:
    2,415
    Richard Bos
    Jun 21, 2005
  3. david ullua
    Replies:
    13
    Views:
    636
  4. Thomas Kreuzer

    Read string for arithmetic operation

    Thomas Kreuzer, Aug 3, 2006, in forum: C Programming
    Replies:
    4
    Views:
    428
    CBFalconer
    Aug 4, 2006
  5. Buzz Lightyear
    Replies:
    10
    Views:
    1,094
    Alexander Bartolich
    Aug 12, 2009
Loading...

Share This Page