Ruby Performance

[Bill Kelly]

When I posted my original question, I had no idea that the later
conversation would contain such passion, humor, sarcasim and
personal insult.

It's Usenet. There's one in every crowd.

Based on my experience of Ruby (which no doubt has been partly
determined by my experiences with other languages) I can honestly say
that it offers me little that is new/unachievable in other languages.

<smile> Well for any Turing-complete language, there's nothing
technically achievable in one that isn't also achievable in all
the others.

The one thing Turing-completeness doesn't cover is the number
and/or degree of headaches the programmer is subjected to before
the desired achievement is accomplished.

From what I've read from other Rubyists, and from my own
experience, many of us use Ruby because we find it more fun and
more productive than other languages we've learned so far.

(For me, that's Forth, C, C++, Java, Perl, Python, and Smalltalk.)

Best wishes to you in whatever language YOU find most fun and
most productive. It's not going to be Ruby for everyone, it just
is for a lot of us.


Regards,

Bill

 

[Matthew Desmarais]

Hi Isaac,

Before we go any further, I want to be clear about a couple of things.
I have no problems with what goes on around the shootout site. I don't
think that you'll find many people here that do, since a lot of them
probably found their own path to ruby. There are quite a few
adventurous and curious souls on the list and they spend a lot of time
pulling ruby apart trying to understand it more clearly and find ways to
make it better. (Putting Matz in the unenviable position of having to
reconsider some of the basic features of the language on a regular
basis. If I were he, I'd have an ulcer by now...)

By all means, carry on with experimentation with all of the languages
that you can get your hands on. But let's not pretend that we're
measuring anything.

I was trying to answer "I don't think that I've learned much from the
shootout".


Let's try again - "What is it that we are learning about programming
languages?"

I don't know what you learn about programming languages from shootout,
the website states "Our goals are to learn about programming
languages..." - the goals of the folk administering the shootout are to
learn about programming languages...

I understand you here, but I don't think that you are very clear about
this. More later...

Let's try again - "What do we gain from comparing performance in
"various (possibly meaningless) ways?"

Some perspective on how performance varies between programming language
implementations and tasks.

This is a part of what I take issue with. The site offers a great deal
of disclaimers regarding the worthlessness of the "benchmarks" there.
But most of the space on the main page is devoted to reporting and
comparing these benchmarks.

History - Doug Bagley's Shootout begat Aldo Calpini's Win32 Shootout
begat Brent Fulgham's Shootout.

With regard to your justification of the name, I'm not sure that history
is a good reason to keep it.

The site provides multiple comparison programs, which show various
different language implementations "winning".

The site shows different ways to "win" - by CPU time, by memory use, by
LOCs.

The site provides a synthetic overall score and invites you to
"manipulate the multipliers and weights to make your favourite language
the fastest programming language in the Shootout" for "a solution that
is simple, neat, and wrong".

Does the site proclaim A is faster than B, or subvert that simplistic
notion?

Isaac, you entry into this discussion thread was the following post:

That's even more simplistic than the benchmark programs on the Computer
Language Shootout

Here's Ruby vs Perl

http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&sort=fullcpu

and here are the old Doug Bagley programs

http://shootout.alioth.debian.org/old/benchmark.php?test=all&lang=ruby&sort=fullcpu
<http://shootout.alioth.debian.org/old/benchmark.php?test=all&lang=ruby&sort=fullcpu>

When you follow the Ruby vs Perl link, you are treated to a nice picture
and some numbers, all of which (you would assume from reading the page)
offer some insight into the comparative merits of the languages.

In this case, the site _does_ proclaim a winner in any number of categories.

The site talks out of both sides of its mouth. Even with all of the
language condemning the "simplistic notion" of winners and losers, the
site still offers results. It is these results that are dishonest and
misleading.

I think that the site would work much better _without_ the results
page. Why not let the curious discover their own results? Hands on
experimentation is the best way to explore the concepts that you are
interested in. Maybe you can find a way to offer free shell accounts
that give access to the tools required to experiment with these
"benchmarks".

Or at the very least, take the focus off of those confounded numbers.

 

[Isaac Gouy]

Hi Matthew

-snip-

By all means, carry on with experimentation with all of the languages
that you can get your hands on. But let's not pretend that we're
measuring anything.

On the contrary, we are measuring 'something'.

imo The issue is interpretation - what relevance (if any) do those
measurements have, what worth (if any) do those measurements have, what
(if anything) do they mean?


-snip-

This is a part of what I take issue with. The site offers a great deal
of disclaimers regarding the worthlessness of the "benchmarks" there.
But most of the space on the main page is devoted to reporting and
comparing these benchmarks.

Flawed is not a synonym for worthless ;-)


-snip-

With regard to your justification of the name, I'm not sure that history
is a good reason to keep it.

You may be right about that.


-snip-

When you follow the Ruby vs Perl link, you are treated to a nice picture
and some numbers, all of which (you would assume from reading the page)
offer some insight into the comparative merits of the languages.

In this case, the site _does_ proclaim a winner in any number of categories.

Yes - under said conditions, said program X was measured to be A,B,C
and said program Y was measured to be D,E,F.

imo that's not The issue.

The site talks out of both sides of its mouth. Even with all of the
language condemning the "simplistic notion" of winners and losers, the
site still offers results. It is these results that are dishonest and
misleading.

Let me clarify what I meant by "simplistic notion".

I meant it's simplistic to measure one program using language
implementation A, and one program using language implementation B, and
then *generalize* to say that language A is faster than language B.

I meant it's simplistic to measure cpu time without measuring memory
use, and then claim that program A is faster than program B.

It's simply true that under said conditions, said program X was
measured to be A,B,C and said program Y was measured to be D,E,F -
there's nothing dishonest about those results.

I think that the site would work much better _without_ the results
page.

Which page is "the results page"?

 

[ogilthorpe]

When you follow the Ruby vs Perl link, you are treated to a nice pictu=
re

Yes - under said conditions, said program X was measured to be A,B,C
and said program Y was measured to be D,E,F.

imo that's not The issue.

Aha! It appears that this is where we'll have to agree to disagree. For
me this is The issue. There were no conditions mentioned on the page
that you linked to.

There are so many factors that contribute to any measure of performance
for a "program" that any comparison must be qualified to the point of
irrelevance. Some such factors might be:

- Compiled vs Interpreted
-- Implementation of the Compiler/Interpreter
--- Version of the implementation
-- Compilation flags

- Implementation of the algorithm
-- Suitability of the implementation to the algorithm

- System Environment and its affects on the execution environment.

I could spend all day listing considerations like these. To attempt to
boil all of this down to one number for any measure of performance is
naive. These numbers aren't useful to the casual observer who will
misunderstand them, nor are they useful to the person who spends the time
to investigate the execution environment and draw their own conclusions.

Good luck with the site Isaac. I hope that it continues to go well for
you and that you continue to learn from it.

 

[Isaac Gouy]

[in reply to no single post in particular]

Well I really had no idea of how much emotional ettatchment people had
to their languages of choice.

When I posted my original question, I had no idea that the later
conversation would
contain such passion, humor, sarcasim and personal insult.

Any way, my orginal post stemmed from my curiosity with regards
to what Ruby was trying to achieve:

The highly non-real-world and simplistic benchmarks were not meant to
provoke the zeolots, but rather well displayed my lack of understanding
for what Ruby was trying to achieve. Of which I still dont quite understand

"a genuine object-oriented, easy-to-use scripting language"
http://www.rubygarden.org/faq/entry/show/5

 

[Julian Leviston]

thank god someone did.

I feel a lot of love on this thread.

Should I invoke Godwin's Law and put this argument to rest?

http://en.wikipedia.org/wiki/Godwin's_law

Okay.

All benchmarks are the brainchild of Hitler and his nazis.

 

[Isaac Gouy]

Aha! It appears that this is where we'll have to agree to disagree. For
me this is The issue. There were no conditions mentioned on the page
that you linked to.

There are so many factors that contribute to any measure of performance
for a "program" that any comparison must be qualified to the point of
irrelevance. Some such factors might be:

- Compiled vs Interpreted
-- Implementation of the Compiler/Interpreter
--- Version of the implementation

The specific implementation and version is given at the bottom of the
page I linked to
http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&sort=fullcpu#about

-- Compilation flags

Are shown for each program, they may vary program to program, for
example
http://shootout.alioth.debian.org/benchmark.php?test=ackermann&lang=gcc&id=3&sort=fullcpu#log

- Implementation of the algorithm
-- Suitability of the implementation to the algorithm

Not sure exactly what you mean, perhaps this
http://shootout.alioth.debian.org/benchmark.php?test=ackermann&lang=all&sort=fullcpu#about

- System Environment and its affects on the execution environment.

Is given in the FAQ
http://shootout.alioth.debian.org/faq.php?sort=fullcpu#measure

I could spend all day listing considerations like these. To attempt to
boil all of this down to one number for any measure of performance is
naive. These numbers aren't useful to the casual observer who will
misunderstand them, nor are they useful to the person who spends the time
to investigate the execution environment and draw their own conclusions.

Good luck with the site Isaac. I hope that it continues to go well for
you and that ou continue to learn from it.

Thank you.