Speed sprint

[Benedikt Müller]

Hi
We all know: Ruby is not the fastest interpreted language out there. I
think, this isn't the goal, but I also think, Ruby is too slow.
I would suggest, that a time, say two monthes or three, only speed
patches and bugfixes are accepted. So the development could be
concentrated on speed.
I'd like to hear what over users out there and the developers say to this idea.

 

[Alexander Jesner]

Ruby is not the fastest interpreted language out there.

If you have not already done so, switch to Ruby 1.9.

 

[Benedikt Müller]

If you have not already done so, switch to Ruby 1.9.

Done, and I know that there are improvements. But it's not enough

--=20
Gru=C3=9F, Benedikt

 

[Tony Arcieri]

wrote:

Done, and I know that there are improvements. But it's not enough

Have you taken a look at some of the other Ruby implementations available,
such as JRuby, Rubinius, and MacRuby?

JRuby is able to leverage the performance features of the Java Virtual
Machine. Rubinius and MacRuby are both using LLVM to produce native code
from Ruby sources. Ruby 1.9, on the other hand, uses a stack machine
interpreter and does not compile to native code.

Depending on the specific nature of your performance problems, one of these
alternative implementations may be addressing your issues already.

--=20
Tony Arcieri
Medioh! A Kudelski Brand

 

[Randall Alexander]

Done, and I know that there are improvements. But it's not enough

understand they are usually a version behind regular Ruby but you have the
power of the JVM. I agree with speeding Ruby up to the credit of the core
team they did make an effort on the last release. I do think there are a
few other options. (Don't get me wrong I am not opposed to the speed sprint=
s
I just wanted to present other options) As far as I know the Ruby runtime
is missing lookahead ability. If that is correct adding lookahead could
give the runtime a huge performance boost. Another alternative to asking
the Ruby core team to do these sprints we could profile, make changes, and
submit the commits (and test cases) ourselves. I have never worked on open
source project before, but I am speaking with someone about how to get
involved and be a productive active member. It takes more than I originall=
y
thought.


--=20
Randy

 

[Robert Klemme]

Done, and I know that there are improvements. But it's not enough

You know that greed is one of the seven deadly sins, do you? ;-)

Kind regards

robert

 

[Jörg W Mittag]

If you have not already done so, switch to Ruby 1.9.

I have seen this claim that Ruby 1.9 is somehow faster than Ruby 1.8
repeated over and over again, but I have *never* seen any credible
evidence for that, neither in my own benchmarks nor in Antonio
Cangiano's (or any other, for that matter). Does anyone have any
evidence that this is actually the case? I would be very interested in
that.

jwm

 

[Chuck Remes]

=20
I have seen this claim that Ruby 1.9 is somehow faster than Ruby 1.8
repeated over and over again, but I have *never* seen any credible
evidence for that, neither in my own benchmarks nor in Antonio
Cangiano's (or any other, for that matter). Does anyone have any
evidence that this is actually the case? I would be very interested in
that.

Are you looking at the same benchmarks and charts that I am?

http://antoniocangiano.com/category/ruby-benchmark-suite/

This clearly shows that 1.9.1 is consistently and measurably faster than =
1.8 in nearly every benchmark. My own experience proves this out too.=20

I am a recent convert to 1.9.1 after I discovered that it ran *all* of =
my scripts at least 50% faster than 1.8.7. Between 1.9.1 and JRuby I am =
loving my ruby performance lately. I am also tracking Rubinius closely; =
it is very fast but its performance is less consistent than MRI and =
JRuby so far.

cr

 

[Benoit Daloze]

2010/2/23 J=F6rg W Mittag

I have seen this claim that Ruby 1.9 is somehow faster than Ruby 1.8
repeated over and over again, but I have *never* seen any credible
evidence for that, neither in my own benchmarks nor in Antonio
Cangiano's (or any other, for that matter). Does anyone have any
evidence that this is actually the case? I would be very interested in
that.

jwm

Hi,

"I have *never* seen" Maybe you didn't look enough. Anyway, I think there
are plenty of benchmarks ...

Test yourself and you'll see.
I think it's most of the time faster or equal in Ruby 1.9. There is a reaso=
n
for this speed improvement, I let you search it.

You mention usual Antonio Cangiano Benchmarks:
http://antoniocangiano.com/2007/02/19/ruby-implementations-shootout-ruby-vs=
-yarv-vs-jruby-vs-gardens-point-ruby-net-vs-rubinius-vs-cardinal/

Well, there is a long time I keep traces of these benchmarks:

MBP =3D> MacBookPro, 2x2.26GHz, 2Go

P =3D> patchlevel
R =3D> revision
T =3D> trunk
Time Computer OS 32/64bit RubyVersion
0.684 MBP Mac 64 1.9.2 2010-01-14 T 26319
0.836 MBP Lin 32 1.9.2 2009-07-18 T 24186
0.899 MBP Mac 32 1.9.2 2009-11-04 T 25635
1.719 MBP Win 32 1.9.2 2009-07-18
1.850 MBP Mac 32 1.8.6 2008-08-11 P 287
2.000 MBP Mac 32 1.8.7 2009-12-24 P 248
2.406 MBP Win 32 1.9.1 2009-01-30 R 21907
2.937 MBP Win 32 1.8.6 2007-09-24 P 111

If that is not clear ...
All 1.9.2 are faster(more than 2 times here). The only exception is probabl=
y
due to better implementation of 1.8 on Mac than Windows (the 1.9.1 is
probably an early version too).

Regards,

B.D.

 

[Jörg W Mittag]

Are you looking at the same benchmarks and charts that I am?

http://antoniocangiano.com/category/ruby-benchmark-suite/

Yes, I am.

This clearly shows that 1.9.1 is consistently and measurably faster
than 1.8 in nearly every benchmark.

No, it doesn't. It shows that 1.9.1 running on one VM is faster than
1.8 on a *totally different* VM. That's shoddy benchmarking. If you
want to benchmark 1.8 vs. 1.9, you benchmark 1.8 vs. 1.9 and not 1.8
and some other things vs. 1.9 and some totally unrelated, completely
different other things.

This is statistics 101, and is usually called "controlling your
variables", although the great Zed Shaw in his inimitable style said
it much better: "if you want to measure shit, don't measure other
shit."

You claim that the performance increase is due to Ruby 1.9. But how do
you know that? How do you know it is not due to YARV? Or different
compiler options? Or different ./configure options? After all, it
wouldn't be the first time that Antonio had to withdraw or correct his
benchmark results.

My own experience proves this out too.

Then show your results! As I wrote before: I would love to see them!

jwm

 

[Nick Brown]

http://shootout.alioth.debian.org/

These benchmarks show that Ruby 1.9 (MRI) is much faster than 1.8. In
fact, it is even faster than Python (the closest competing* language).
But keep in mind that dynamically-typed, everything-is-an-object
languages will always come with performance penalties.

*Not that we're actually "competing"--we're all friends, helping
eachother make great very-high-level languages with our respective
preferred syntaxes and feature-sets. But Ruby is still way better. ;-)

 

[Jörg W Mittag]

2010/2/23 Jörg W Mittag

"I have *never* seen" Maybe you didn't look enough. Anyway, I think there
are plenty of benchmarks ...

Test yourself and you'll see.

Did you even read the article you replied to? I already wrote that I
*did* my own tests and they I *did not* see any statistically
significant difference in performance.

I think it's most of the time faster or equal in Ruby 1.9.

It's interesting that you use the word "think" here. So, apparently,
you have your (subconcious) doubts, too.

There is a reason for this speed improvement, I let you search it.

Again, I would like to know. The *only* source I could find so far, is
a video from a BoF session with Jon Lam from maybe two years ago,
which goes something like this:

| Audience: What about 1.9 support?
| JL: Microsoft has commited to making IronRuby a *real* Ruby
| Implementation that runs *real* Ruby code. Right now, *real* Ruby
| code is written in Ruby 1.8. We would *love* to start with Ruby
| 1.9, because it has some semantic changes that make it much easier
| to implement and much easier to implement *fast*. But we have to
| stay with 1.8 for now.

[Don't hold me to the exact wording. This is all from memory, and it
was just a minor sidenote in a video I watched two years ago.]

This is pretty much the only source I could find which actually names
a *reason* for the claimed speed improvements. However, "some semantic
changes" isn't actually incredibly precise, after all, in a
programming language, pretty much *everything* is a "semantic change".

So, if you know the reason, please share it. Saying "I let you search
it" isn't exactly helpful. After all, I wouldn't have asked if I
hadn't searched first, that's Netiquette 101.

You mention usual Antonio Cangiano Benchmarks:
http://antoniocangiano.com/2007/02/...rdens-point-ruby-net-vs-rubinius-vs-cardinal/

Well, there is a long time I keep traces of these benchmarks:

MBP => MacBookPro, 2x2.26GHz, 2Go

P => patchlevel
R => revision
T => trunk
Time Computer OS 32/64bit RubyVersion
0.684 MBP Mac 64 1.9.2 2010-01-14 T 26319
0.836 MBP Lin 32 1.9.2 2009-07-18 T 24186
0.899 MBP Mac 32 1.9.2 2009-11-04 T 25635
1.719 MBP Win 32 1.9.2 2009-07-18
1.850 MBP Mac 32 1.8.6 2008-08-11 P 287
2.000 MBP Mac 32 1.8.7 2009-12-24 P 248
2.406 MBP Win 32 1.9.1 2009-01-30 R 21907
2.937 MBP Win 32 1.8.6 2007-09-24 P 111

Unfortunately, this table doesn't show the execution environment of
the individual benchmark runs. The only way in which this table is
useful, is if the benchmark runs were tightly controlled. Otherwise
you get uncontrolled variables, and all your benchmark results mean
squat.

In statistics, this is called "controlling your variables", but Zed
Shaw said it much better: "If you want to measure shit, don't measure
other shit."

In other words, if you want to measure Ruby 1.9 vs. Ruby 1.8, measure
Ruby 1.9 vs. Ruby 1.8 and not Ruby 1.9 and YARV vs. Ruby 1.8 and MRI,
because that way you will never know whether the performance
difference came from Ruby 1.9, YARV or a combination of the two.

If that is not clear ...
All 1.9.2 are faster(more than 2 times here). The only exception is probably
due to better implementation of 1.8 on Mac than Windows (the 1.9.1 is
probably an early version too).

Like I wrote above: *if* you actually *did* properly control your
variables for these benchmarks, then, yes, they show indeed a
performance increase for Ruby 1.9. And in that case, you really should
publish these results, because, like I wrote earlier, there is nothing
like this out there, at the moment.

jwm

 

[Roger Pack]

Then show your results! As I wrote before: I would love to see them!

http://blog.pluron.com/2009/05/ruby-19-performance.html

Their rails runs 2x faster with 1.9

-r

 

[Chuck Remes]

=20
Yes, I am.

Oh, boy, here we go. By the tone of your response, I'm not sure if =
you're a jerk or I'm in a bad mood. I'll assume you are *not* and that I =
*am*. But just in case you are this guy...

http://xkcd.com/386/

=20
No, it doesn't. It shows that 1.9.1 running on one VM is faster than
1.8 on a *totally different* VM. That's shoddy benchmarking. If you
want to benchmark 1.8 vs. 1.9, you benchmark 1.8 vs. 1.9 and not 1.8
and some other things vs. 1.9 and some totally unrelated, completely
different other things.

I don't understand your point here. How can I benchmark 1.9.1 against =
1.8.7 without the VMs being different? The whole point of the move to =
1.9.1 was YARV and its new interpreter. The VMs are different and unless =
someone undertakes the effort to move the 1.8 syntax on top of YARV, =
that won't change. You mentioned this in two separate replies to this =
thread so I know it is an important point for you. You *cannot* bench =
1.8 versus 1.9 without YARV entering the equation.

I suppose you are saying that there is no valid comparison between 1.8.x =
and 1.9.x because of this difference? If not, explain yourself better.

If you *do* believe that there is no valid comparison because the VMs =
are different, then please explain how 1.8.x and 1.9.x could be =
rigorously compared.

I don't know about you, but I run my ruby code on MRI, YARV, Java6 and =
LLVM (Rubinius and MacRuby). The VM doesn't matter to me as long as the =
results are the same and each platform gives me the performance I =
require.=20

I don't know how you run your code without a VM; sounds neat.

This is statistics 101, and is usually called "controlling your
variables", although the great Zed Shaw in his inimitable style said
it much better: "if you want to measure shit, don't measure other
shit."
=20
You claim that the performance increase is due to Ruby 1.9. But how do
you know that? How do you know it is not due to YARV? Or different
compiler options? Or different ./configure options? After all, it
wouldn't be the first time that Antonio had to withdraw or correct his
benchmark results.

You're looking for a level of rigor that just doesn't exist in software =
benchmarking and particularly for ruby open source. I ran these =
benchmarks on my system. I can give you the basic details but my machine =
likely performs differently from the *exact* same machine with the same =
software load on it. There are dozens of Linux distros all with =
differing patch levels; same goes for BSD, Windows and OSX. Maybe my =
machine runs a little hotter (therefore slower) than an exact clone; I'm =
not about to get a temperature controlled room to bench this language. =
All of these things are likely to effect the bench results.

I say, "So what." Unless there is a bug in a major system component =
(e.g. libc, pthreads, etc), the compiler toolchain, or a hardware =
defect, the results across comparable systems are pretty close. That is =
good enough.

Results from lots of independent sources show that 1.9 is faster than =
1.8. Why? It's worthy of investigation by someone other than me.

=20
Then show your results! As I wrote before: I would love to see them!

I emailed my results to you using Cogniano's benchmark suite available =
from github. I used the source available via the "rvm" utility and =
benched both 1.8.7-p248 and 1.9.1-p376 built on my system (OSX Intel, =
10.6.2, dual quad-core, blah blah blah).

Ruby 1.9.1 was faster in 60 (of 100+) tests, the same in 2 or 3, and =
slower in about 20. These are against the "micro" benchmarks in the test =
suite. In my world, the usual scripts I run are all noticeably faster =
with 1.9.1 versus any 1.8.x MRI branch.

cr

 

[Walton Hoops]

So what your saying is that the VM is part of Ruby? I think your analogy would be better stated that a speed comparison between a Nissan Cube and a Nissan GT-R is invalid because they were driven on different tracks for the time trials.

In that case, IT would be valid.

Yes, the YARV VM _is_ an intrinsic part of Ruby 1.9, just as the JVM
_is_ an intrinsic part of Java.

Thus the original analogy stands, and it makes perfect sense to compare
MRI Ruby 1.8 results with
Ruby 1.9 results using YARV because one of the key improvements in 1.9
is the usage of YARV.

 

[Bill Kelly]

In other words, if you want to measure Ruby 1.9 vs. Ruby 1.8, measure
Ruby 1.9 vs. Ruby 1.8 and not Ruby 1.9 and YARV vs. Ruby 1.8 and MRI,
because that way you will never know whether the performance
difference came from Ruby 1.9, YARV or a combination of the two.

The confusion is understandable.

Historically, on the Matz lineage of interpreters, ruby 1.9 (formerly
ruby 2.0) had long been associated with several key feature bullet
points, particularly:

* new VM for increased speed
* multinationalization support

I joined ruby-talk in 2001, and it was being discussed even then.
Originally the new VM was to be called Rite instead of YARV, and it
was expected to debut in ruby 2.0 instead of 1.9:

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/23267


These days, when we look at other ruby interpreters, such as JRuby,
it's easy to see that 1.8 vs. 1.9 language semantics are orthogonal
to the VM. So, indeed, on JRuby (for example), one can benchmark
1.8 semantics vs. 1.9 semantics without changing the VM.

But in the Matz lineage, 1.8 implies MRI, and 1.9 implies YARV.

Further, the new VM was always understood to be the primary *reason*
to expect speed improvements from 1.8 -> 1.9 in the Matz lineage.


Regards,

Bill

 

[Michael Brooks]

I have seen this claim that Ruby 1.9 is somehow faster than Ruby 1.8
repeated over and over again, but I have *never* seen any credible
evidence for that, neither in my own benchmarks nor in Antonio
Cangiano's (or any other, for that matter). Does anyone have any
evidence that this is actually the case? I would be very interested in
that.

Jörg W Mittag

Hello Jörg:

I found this interesting:
http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=all

It shows averaged scores of Ruby 1.8.7 (the Ruby MRI) vs Ruby 1.9.1 and vs a
whole bunch of other languages.

Michael

 

[Douglas Seifert]

My own "real world" benchmark comparing 1.9 to 1.8:

http://www.dseifert.net/archive/2009-11/

-Doug Seifert

 

[Luis Lavena]

My own "real world" benchmark comparing 1.9 to 1.8:

http://www.dseifert.net/archive/2009-11/

By real world definition you should include IO operations (either
network or file based).

Ruby 1.9 is slower on IO on any platform (and worse on Windows) due
encoding functionality built in.

My real world problem is solve sudoku with a recursive method... very
real world