python/ruby benchmark.

[\]

I took a look at
http://shootout.alioth.debian.org/benchmark.php?test=3Dall&lang=3Druby&lang=
2=3Dpython&sort=3Dfullcpu
and found the tests on ruby disapointing .
So what I'm asking for is a link to some ruby-specific optimisation tips.

--=20
Regards, Groleo!

# touch universe
# chmod +x universe
# ./universe

 

[gabriele renzi]

I took a look at
http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu
and found the tests on ruby disapointing .
So what I'm asking for is a link to some ruby-specific optimisation tips.

a simple thing: do the most normal thing instead of writing code in ruby
that is innatural, i.e. instead of rewritine an heapsort by hand, use
the builtin sort/sort!/sort_by routines. Instead of writing a random
function by hand, use the builtin rand().

If you really feel ruby (or python) is slow you can act later optimizing
it, not doing it before you ever need to be faster. "premature
optimization is the root of all evil", people say.

 

[Glenn Parker]

So what I'm asking for is a link to some ruby-specific optimisation tips.

Avoid creating new objects, e.g. re-use strings and arrays, or modify
them in place when possible, especially large ones.

Use built-in iterators as much as possible.

Pray for YARV!

 

[Vincent Foley]

As Gabriele mentionned, they implement a lot of stuff that is done in C
in Ruby. Also, the tests must use the exact same algorithm, so this
disadvantages Ruby is some situations. I say the test should compare
same algorithms, but also the one best suited to the language.

 

[Lothar Scholz]

Hello gabriele,


gr> a simple thing: do the most normal thing instead of writing code in ruby
gr> that is innatural, i.e. instead of rewritine an heapsort by hand, use
gr> the builtin sort/sort!/sort_by routines. Instead of writing a random
gr> function by hand, use the builtin rand().

Sorry you don't understand this benchmark. It is there to do this
algorithms to compare the basic data operations.

The benchmark is not useable to compare compiled languages with
interpretered, but comparing python with ruby is a valid comparision.

And that ruby is so worse then python is a real problem. If you still
try to deny this, then you have a real mental problem.

If ruby is 9 times slower then python in for example function calling intensive
code that means that ruby will slow down your development time a lot
as it forces you to use optimized code or even C code much more often
then python.

 

[Kent Sibilev]

Java is an order of magnitude faster than Ruby. The development of a
web app using Rails is an order of magnitude faster than the
development of a similar web app using Struts/Hibernate frameworks.
There are other factors that can slow down your development time
besides the speed of your language of choice.

Cheers,
Kent
Just another guy with huge mental problems.

 

[Dibya Prakash]

Hi,
I definitely agree.Only feature of a particular programming language
cannot determine the speed of software.Other factors are also
crucially important.
I didnot know about YARV,but gr8 that I came to know it thru u
people.I'll surely check it out.

Bye
Dibya Prakash
Happy coding

 

[Lothar Scholz]

Hello Kent,

KS> Java is an order of magnitude faster than Ruby. The development of a
KS> web app using Rails is an order of magnitude faster than the
KS> development of a similar web app using Struts/Hibernate frameworks.
KS> There are other factors that can slow down your development time
KS> besides the speed of your language of choice.

If i can't use the language because it is to slow to solve the problem
then the development time doesn't matter. There are just too many
problem domains left where you can't ignore it.

But we all hope and pray for YARV.

 

[gabriele renzi]

Lothar Scholz ha scritto:

Hello gabriele,




gr> a simple thing: do the most normal thing instead of writing code in ruby
gr> that is innatural, i.e. instead of rewritine an heapsort by hand, use
gr> the builtin sort/sort!/sort_by routines. Instead of writing a random
gr> function by hand, use the builtin rand().

Sorry you don't understand this benchmark. It is there to do this
algorithms to compare the basic data operations.

I understand the benchmark.
I'm just saying that it shows "you should not implement X in ruby" not
that "X takes a lot of time in ruby" since you may not need to implement
X (i.e. because it is already there or because you would not actually
ever need a recursive floating point function).

The benchmark is not useable to compare compiled languages with
interpretered, but comparing python with ruby is a valid comparision.

And that ruby is so worse then python is a real problem. If you still
try to deny this, then you have a real mental problem.

I don't think I did

If ruby is 9 times slower then python in for example function calling intensive
code that means that ruby will slow down your development time a lot
as it forces you to use optimized code or even C code much more often
then python.

I agree, It would be nice if ruby was faster. Luckily,
people are working on this.

 

[Jim Freeze]

I agree, It would be nice if ruby was faster. Luckily,
people are working on this.

Don't go away thinking that X is faster than Ruby. Do
your own tests. We tested Perl against Ruby in a real
world problem (netlist parsing) and Ruby was faster
than Perl by 30%. When we did a C implementation
(which could now be done automatically with optimize)
it was 100% faster (ie, took half the time.)

These statements that X language is faster than
Ruby is like saying that Einstein is smarter than
Mozart. In what?

 

[Phil Tomson]

Don't go away thinking that X is faster than Ruby. Do
your own tests. We tested Perl against Ruby in a real
world problem (netlist parsing) and Ruby was faster
than Perl by 30%. When we did a C implementation
(which could now be done automatically with optimize)
it was 100% faster (ie, took half the time.)

Are you saying that the Ruby netlist parser was 100% faster than the C
implementation or the other way around?

BTW: I recall someone at a PDX.rb meeting who said he had benchmarked
Ruby vs. OO-style Perl and Ruby was quite a bit faster (don't recall the
percentage). As I recall the explanation was that doing OO in perl
requires more levels of indirection and thus more function calls.

Phil

 

[Jim Freeze]

Are you saying that the Ruby netlist parser was 100% faster than the C
implementation or the other way around?

The C implementation was Ruby C (my bad). So, the Ruby C
implementation ended up twice as fast as the Perl
implementation. Bad comparison you say. Well, in this
environment, we found over four netlist parsers written
in Perl. These parsers were intermingled with the netlist
munging code, so it was difficult to do abstraction and
pull out the netlist parser in Perl and optimize it.

But you say, well all that could and should have been done in
Perl. My response is, well, you had 10 years, what were you
waiting for?

The pragmatic nature of Ruby lent itself to writing better code,
more modular code, and thus the ability to optimize a particular
module muuuchh more than Perl.

 

[Isaac Gouy]

As Gabriele mentionned, they implement a lot of stuff that is done in C
in Ruby.
Specifically?

Also, the tests must use the exact same algorithm, so this
disadvantages Ruby is some situations. I say the test should compare
same algorithms, but also the one best suited to the language.

Do this disadvantage Ruby more than PHP?
http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=php&sort=fullcpu

Or more than Tcl, or more than GNU Smalltalk?
http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=gst&sort=fullcpu

 

[Austin Ziegler]

Do this disadvantage Ruby more than PHP?
Or more than Tcl, or more than GNU Smalltalk?

When the algorithm, as entered, won't complete, yes. When compared
against a language that has tail recursion optimisation, or a possible
situation where optimisation of the compiler deals with recursion
intelligently? Yes.

The definition of Ackermann numbers is, I presume, well known. Because
some platforms can't modify their stack level (Windows), and because
the stack level isn't modified in the runs for the benchmark, there's
going to be an issue with the deep recursion. In this case, it would
be appropriate to reduce the level of recursion. But given the rules
mentioned, this isn't acceptable.

What's the *purpose* of something like the Ackermann test? Is it to
see how quickly the environment winds and unwinds the stack? How much
state it carries around?

I personally think that most of the shootout items are bogus in just
about every way. Don't get me wrong -- I'm not at all suggesting that
Ruby couldn't be faster. I'm just saying that benchmarks, like
statistics, are lies.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Isaac Gouy]

When the algorithm, as entered, won't complete, yes. When compared
against a language that has tail recursion optimisation, or a possible
situation where optimisation of the compiler deals with recursion
intelligently? Yes.

Ruby completes ackermann for N=7
http://shootout.alioth.debian.org/g...ean-0&p3=gcc-3&p4=gcc-0&sort=fullcpu#cputable

Which is how there can be a comparison between Ruby and Tcl on
ackermann
http://shootout.alioth.debian.org/g...st=all&lang=ruby&lang2=tcl&sort=fullcpu#ratio

The definition of Ackermann numbers is, I presume, well known. Because
some platforms can't modify their stack level (Windows), and because
the stack level isn't modified in the runs for the benchmark, there's
going to be an issue with the deep recursion.

We use Debian Linux, so tell us how to modify the stack level for Ruby.


-snip-

I personally think that most of the shootout items are bogus in just
about every way. Don't get me wrong -- I'm not at all suggesting that
Ruby couldn't be faster. I'm just saying that benchmarks, like
statistics, are lies.

Sounds like me in editorial mode.
http://shootout.alioth.debian.org/g...cpu&file=benchmarking&title=Flawed Benchmarks

 

[Trey Campbell]

I personally think that most of the shootout items are bogus in just
about every way. Don't get me wrong -- I'm not at all suggesting that
Ruby couldn't be faster. I'm just saying that benchmarks, like
statistics, are lies.

-austin

The most succinct way that I've heard it put is that there are "lies,
damn lies, and benchmarks".

Trey Campbell
(e-mail address removed)

 

[Austin Ziegler]

We use Debian Linux, so tell us how to modify the stack level for Ruby.

I'm sorry you do. However, yours should be one of the ones that should
be able to modify the depth of the stack with ulimit.

If I cared enough to get more than annoyed at this damned shootout
with its inane implementation rules, I'd do more. But as it is, the
best I can tell people is that the Alioth shootout is bullshit.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Isaac Gouy]

The most succinct way that I've heard it put is that there are "lies,
damn lies, and benchmarks".

Trey Campbell
(e-mail address removed)

There isn't with benchmarks per-se; there can be a problem with how one
chooses to interpret benchmarks.

 

[Austin Ziegler]

There isn't with benchmarks per-se; there can be a problem with
how one chooses to interpret benchmarks.

No, it's the benchmarks themselves that are the problem. Remember
that NVidia got caught detecting a benchmark in their drivers and
optimizing output for the benchmark.

Similarly, this thread was started because someone came across your
language shootout benchmark page and assumed that Ruby is
dramatically slower in Real Life Examples because it's slower in the
benchmarks. Which, I can assure you, it most assuredly NOT.

You get it correct when you indicate that "the best benchmark is
your program." That's the only one that matters. It also lets you
measure benchmarks that can't be categorised in CPU seconds or
memory use or FLOPS or other nonsense like that.

It's hype, but the idea that Rails makes one 10x more productive is
part of the value of Ruby. Unless your program is of a special case
-- and a lot of these special cases are handled already -- then Ruby
programs will be developed faster and more easily maintained than
their non-Ruby counterparts. Enough so that execution speed is
something of a wash.

At any rate, I consider the Alioth shootout to be harmful to all
languages involved. There is no useful value provided by it,
especially as it does not permit language-appropriate modifications
to the algorithms in use.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Isaac Gouy]

No, it's the benchmarks themselves that are the problem. Remember
that NVidia got caught detecting a benchmark in their drivers and
optimizing output for the benchmark.

Similarly, this thread was started because someone came across your
language shootout benchmark page and assumed that Ruby is
dramatically slower in Real Life Examples because it's slower in the
benchmarks.

As I said - There isn't a problem with benchmarks per-se; there can be
a problem with how one chooses to interpret benchmarks.

Which, I can assure you, it most assuredly NOT.

Not "dramatically slower" than what other language, for what specific
"Real Life Examples"?

It's hype, but the idea that Rails makes one 10x more productive is
part of the value of Ruby. Unless your program is of a special case
-- and a lot of these special cases are handled already -- then Ruby
programs will be developed faster and more easily maintained than
their non-Ruby counterparts. Enough so that execution speed is
something of a wash.

Faster to develop-in than what other languages - Smalltalk? Lisp?
Mathematica?

At any rate, I consider the Alioth shootout to be harmful to all
languages involved. There is no useful value provided by it,
especially as it does not permit language-appropriate modifications
to the algorithms in use.

What's a language-appropriate modification?
Have you ever asked for one to be accepted in the Alioth shootout?