Rounding to X digits

G

GOTO Kentaro

In message Re: Rounding to X digits
Both are (IIRC) ISO 31-0 conformant, but one uses ISO 31-0 B.3 rule A
and the other uses ISO 31-0 B.3 rule B.

That said, one of the rules (the one sprintf uses, to round down
following even digits is an abomination and should be eliminated from the
face of the earth. nnn.nn5 should ALWAYS round up and rounded values
should not be re-rounded (which is what the contrary rule assumes is
happening).

Thanks Markus. I had misunderstood long time.

I found a citation (probably) from ISO 31-0 Annex B
http://www.dataaction.co.jp/31-e.htm


Gotoken
 
R

Robert Klemme

GOTO Kentaro said:
In message Re: Rounding to X digits


And there is an algorithmic difference: many implementation of
sprintf(3) conform to ISO 31-0 but Float#round doesn't.

puts 0.5.round #=> 1
puts sprintf("%.0f", 0.5) #=> 0

Which version of Ruby are you using?
=> "1.8.1"

Regards

robert
 
R

Robert Klemme

Markus said:
Yes, that's exactly why it's there. But there are several reasons
(e.g. Benford's Law http://mathworld.wolfram.com/BenfordsLaw.html) why
it doesn't even work for its intended purpose. The real answer is to
not re-round the data.

As for developing strong feelings about it, all you need to do is
work downstream from someone who swears by it and try to do some valid
numerical analysis. The only person I ever knew with a worse lament was
a friend who worked downstream of an astronomy professor (who should
have been emeritusized years before) that routinely converted his images
to jpeg "to save disk space."

A real "star destroyer"... :))

*chuckle*

robert
 
G

Gavin Kistner

Which version of Ruby are you using?
=> "1.8.1"

[Slim:~] gavinkis% ruby -e "p Array.new(10000).collect{
sprintf('%.0f',0.5) }.uniq"
["0"]
[Slim:~] gavinkis% ruby --version
ruby 1.8.2 (2004-07-29) [powerpc-darwin7.5.0]
 
R

Robert Klemme

Gavin Kistner said:
Which version of Ruby are you using?
=> "1.8.1"

[Slim:~] gavinkis% ruby -e "p Array.new(10000).collect{

Do you think the 10000 makes it more convincing? :)
sprintf('%.0f',0.5) }.uniq"
["0"]
[Slim:~] gavinkis% ruby --version
ruby 1.8.2 (2004-07-29) [powerpc-darwin7.5.0]

Now, is this due to the Ruby version or rather the libC version (I tested
with cygwin)?

Kind regards

robert
 
G

Gavin Kistner

Do you think the 10000 makes it more convincing? :)

Obviously! :)

No, I wasn't fully paying attention to this thread, but did see the
talk about alternating rounding. I didn't know if it applied to the
sprintf case, so, just in case it was a pseudo-random result I was
checking if perhaps it varied between 0/1.
Now, is this due to the Ruby version or rather the libC version (I
tested
with cygwin)?

Dunno; I simply offer my data point for observation, and will let
others figger out the discrepancy.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top