Benchmark Mono - Ruby

[Michael Gebhart]

Hi,

because of my interest in mono and ruby I have done a small benchmark.
These are the results:

Mono Code:

using System;

class Bench {
public static void Main() {
double d = 0;

for (int i = 0; i < 1000000000; i++) {
d = d + i;
}

Console.WriteLine(d);
}

Needs 10.8 Seconds.


In Ruby:

d=0
1000000000.times {
d = d + 1
}

puts d


Needs: 8 minutes and 20 seconds.


Does not look very good Is there a possibility to tune my ruby-program,
to be as fast as mono?

Greetings

Mike

 

[Jamis Buck]

Hi,

because of my interest in mono and ruby I have done a small benchmark.
These are the results:

Mono Code:

using System;

class Bench {
public static void Main() {
double d = 0;

for (int i = 0; i < 1000000000; i++) {
d = d + i;
}

Console.WriteLine(d);
}

Needs 10.8 Seconds.


In Ruby:

d=0
1000000000.times {
d = d + 1
}

puts d


Needs: 8 minutes and 20 seconds.


Does not look very good Is there a possibility to tune my ruby-program,
to be as fast as mono?

At the risk of sounding defensive...

My first question when seeing benchmarks like this: how many
real-world programs have, as their performance bottleneck, a massive
loop that just increments numbers? I'm sure there are some--I don't
deny it--but is your application one of them?

If you want to compare performance between two programming languages
and execution environments, the first rule is to find a benchmark that
properly measures the performance of those languages in the correct
domain. Figure out what you want those languages to do, for real, and
write a simple test that does that thing. Then run your test.

Second rule: figure out what to measure. Is execution time the only
important factor? Or is memory consumption an issue as well? Network
traffic? Development time? Maintenence effort? Portability? Etc.

If execution time and memory consumption are both the most important,
you'd do well to abandon both Ruby and Mono and code in C. Or assembly
code.

You can *always* find a situation in which language X will
"outperform" language Y (for some definition of "outperform"). There
are things that Ruby does better than Mono, I'm sure (but not being
familiar with Mono, I'm not qualified to say what they might be).
Although, at a glance, it looks like the Mono sample took twice as
many lines of code as the Ruby sample...

Just some things to think about. Certainly continue your
investigations--please don't think I'm discouraging that--but make
sure you are testing the right things, and comparing the right
metrics.

- Jamis

 

[Navindra Umanee]

d=0
1000000000.times {
d = d + 1
}

puts d

1000000000.times --> 1000000000 method calls to block
d = d + 1 --> 1000000000 method calls to d
puts d --> 2 method calls
-----------------------------------------------------
2000000002 method calls
=====================================================

Well, it sucks, but I guess you might want to realise that you are
performing *at least* 2000000001 method calls in Ruby, probably *a
factor* more at the C-level implementation, using the above code.

I don't think you can really get around that, since even using a for
loop probably requires that you use a range which might involve more
method calls.

I guess that's the price of pervasive and flexible OO throughout the
language. I wonder if ruby2c might be able to optimise this sort of
thing.

How does Mono fare when performing 2000000002 method calls?

Cheers,
Navin.

 

[Kent Sibilev]

Consider this 'tuning':

$ cat t.rb
require 'ruby_to_c'

module Inline
class Ruby < Inline::C
def initialize(mod)
super
end

def optimize(meth)
src = RubyToC.translate(@mod, meth)
@mod.class_eval "alias :#{meth}_slow :#{meth}"
@mod.class_eval "remove_method :#{meth}"
c src
end
end
end

class Test
def run
d=0
1000000000.downto(1) {
d = d + 1
}
return d
end

inlineRuby) do |builder|
builder.optimize :run
end
end

puts Test.new.run

$ time ruby t.rb
1000000000

real 0m3.118s
user 0m2.670s
sys 0m0.050s

$ cat t.cs
using System;

class Bench {
public static void Main() {
double d = 0;

for (int i = 0; i < 1000000000; i++) {
d = d + i;
}

Console.WriteLine(d);
}
}

$ mcs t.cs
Compilation succeeded

$ time mono t.exe
4.99999999067109E+17

real 0m37.692s
user 0m34.540s
sys 0m0.040s


Cheers,
Kent.

 

[Alexander Staubo]

Consider this 'tuning': [snip]

class Test
def run
d=0
1000000000.downto(1) {
d = d + 1
} [snip]
double d = 0;

for (int i = 0; i < 1000000000; i++) {
d = d + i;
}

To be fair, the type of the variable in the Mono example is "double",
but the Ruby example uses an int. A more appropriate Ruby version:

def run
d = 0.0
1000000000.downto(1) {
d = d + 1.0
}
return d
end

Alexander.

 

[Kent Sibilev]

That's true, but this example is meaningless anyway. The point is that
if you find more than a couple of places in your application where Ruby
does not provide an acceptable performance, you probably should use
another language for the task. I just wanted to demonstrate that with
the help of RubyInline or even Ruby2C you can translate these
bottleneck spots without much of the hassle.

Cheers,
Kent.

Consider this 'tuning': [snip]
class Test
def run
d=0
1000000000.downto(1) {
d = d + 1
} [snip]
double d = 0;
for (int i = 0; i < 1000000000; i++) {
d = d + i;
}

To be fair, the type of the variable in the Mono example is "double",
but the Ruby example uses an int. A more appropriate Ruby version:

def run
d = 0.0
1000000000.downto(1) {
d = d + 1.0
}
return d
end

Alexander.

 

[Zach Dennis]

Kent,

this was pretty hella cool. I'm going to check out Inline::C now that I
see an easy example and your benchmark. =) Gratzi,

Zach

 

[George Moschovitis]

WOW,

I am impressed!

George

 

[Sam Roberts]

Quoteing (e-mail address removed), on Sun, Feb 06, 2005 at 12:05:12PM +0900:

Hi,

because of my interest in mono and ruby I have done a small benchmark.

Ruby is an interpreted language. Its strengths are speed of development
and flexibility of implementation. C# is a compiled language.

Java and C# exist in a kind of half-world - more flexible than C++ in
some ways (GC, at least), but not as fast as the lower-level languages.

If you want to do fast math, you should be working in C or C++. If you
want both speed and a beautiful OO language, you write the math in C,
and bind it into ruby.

People doing cryptography in .Net do the same -> all the crypto code is
C/assembler, and it is bound in.

Cheers,
Sam

 

[Alexander Kellett]

I've made this benchmarking in some more languages (using d=d+1):

In my Pentium 4 2Ghz 256Mb SDRAM 7200rpm:

Java: 00m14,45s (d as long)
Java: 00m12,70s (d as double)
C: 00m03,27s (d as long int)
C: 00m15,90s (d as double)
Squeak Smalltalk: 01m48,60s
Ruby: 18m12,21s
Python: 14m04,83s

with c+llvm it is instantaneous.
Alex

 

[Kaspar Schiess]

(In response to by
Michael Gebhart)

In Ruby:

d=0
1000000000.times {
d = d + 1
}

As far as I can see, no one tried YARV. I didn't run these tests myself,
they are the current benchmarks from the YARV-devel list. And yes, the
numbers different, but you get the picture:

whileloop:
i = 0
while i<10000000
i+=1
end
--
user system total real
ruby 18.780000 0.050000 18.830000 ( 18.944069)
yarv 1.210000 0.000000 1.210000 ( 1.215278)

simpleiter:
1000000.times{|simpleiter|
simpleiter
}

 

[Michael Walter]

Is that with optimizations on (for C)?

Alexander Staubo, 6/2/2005 03:14:


I've made this benchmarking in some more languages (using d=d+1):

In my Pentium 4 2Ghz 256Mb SDRAM 7200rpm:

Java: 00m14,45s (d as long)
Java: 00m12,70s (d as double)
C: 00m03,27s (d as long int)
C: 00m15,90s (d as double)
Squeak Smalltalk: 01m48,60s
Ruby: 18m12,21s
Python: 14m04,83s

I couldn't understand why the C code could be slower than the Java one
(is the same code, as java and C have similar sintaxes, the java one has
the boilerpart and is it).

SmallTalk is 100% OOo, with a lot of message calls and is much faster
for this task than Ruby or Python.

 

[Caio Tiago Oliveira]

Michael Walter, 6/2/2005 19:40:

Is that with optimizations on (for C)?

No optimizations on. I never used it and the code someone posted here
sounds a quite complicated for a little task. It may be interesting, but
the better would be something transparent to the developer.

 

[Michael Walter]

Michael Walter, 6/2/2005 19:40:

No optimizations on.

OK. It makes no sense comparing times, then.

I never used it and the code someone posted here
sounds a quite complicated for a little task. It may be interesting, but
the better would be something transparent to the developer.

I don't understand what you mean.

Greetings,
Michael

 

[Caio Tiago Oliveira]

Michael Walter, 6/2/2005 22:40:

OK. It makes no sense comparing times, then.

I don't understand what you mean.

The optimizations are hard to do.

 

[Michael Walter]

I'm talking about compiler switches.

Michael Walter, 6/2/2005 22:40:

The optimizations are hard to do.

 

[Charles Miller]

Mono Code:

using System;

class Bench {
public static void Main() {
double d = 0;

for (int i = 0; i < 1000000000; i++) {
d = d + i;
}

Console.WriteLine(d);
}

Needs 10.8 Seconds.

Ruby code:

time ruby -e 'puts (999999999 ** 2 + 999999999) / 2'
499999999500000000

real 0m0.034s
user 0m0.010s
sys 0m0.010s

Proving once again that algorithm choice is far more important than
language choice.

C

 

[martinus]

And proves once again that the best optimizer sits behind the ears

martinus

 

[backdoc]

Ruby code:

time ruby -e 'puts (999999999 ** 2 + 999999999) / 2'
499999999500000000

real 0m0.034s
user 0m0.010s
sys 0m0.010s

Proving once again that algorithm choice is far more important than
language choice.

C

Wow! I didn't realize that ruby could be executed from the command line
like that. That's awesome.

darren

 

[Michael Gebhart]

Hi,

great! Really great!!

Thanks

Mike