Benchmark Mono - Ruby

Discussion in 'Ruby' started by Michael Gebhart, Feb 6, 2005.

  1. Hi,

    because of my interest in mono and ruby I have done a small benchmark.
    These are the results:

    Mono Code:

    using System;

    class Bench {
    public static void Main() {
    double d = 0;

    for (int i = 0; i < 1000000000; i++) {
    d = d + i;
    }

    Console.WriteLine(d);
    }

    Needs 10.8 Seconds.


    In Ruby:

    d=0
    1000000000.times {
    d = d + 1
    }

    puts d


    Needs: 8 minutes and 20 seconds.


    Does not look very good :( Is there a possibility to tune my ruby-program,
    to be as fast as mono?

    Greetings

    Mike
    Michael Gebhart, Feb 6, 2005
    #1
    1. Advertising

  2. Michael Gebhart

    Jamis Buck Guest

    On 12:05 Sun 06 Feb , Michael Gebhart wrote:
    > Hi,
    >
    > because of my interest in mono and ruby I have done a small benchmark.
    > These are the results:
    >
    > Mono Code:
    >
    > using System;
    >
    > class Bench {
    > public static void Main() {
    > double d = 0;
    >
    > for (int i = 0; i < 1000000000; i++) {
    > d = d + i;
    > }
    >
    > Console.WriteLine(d);
    > }
    >
    > Needs 10.8 Seconds.
    >
    >
    > In Ruby:
    >
    > d=0
    > 1000000000.times {
    > d = d + 1
    > }
    >
    > puts d
    >
    >
    > Needs: 8 minutes and 20 seconds.
    >
    >
    > Does not look very good :( Is there a possibility to tune my ruby-program,
    > to be as fast as mono?


    At the risk of sounding defensive...

    My first question when seeing benchmarks like this: how many
    real-world programs have, as their performance bottleneck, a massive
    loop that just increments numbers? I'm sure there are some--I don't
    deny it--but is your application one of them?

    If you want to compare performance between two programming languages
    and execution environments, the first rule is to find a benchmark that
    properly measures the performance of those languages in the correct
    domain. Figure out what you want those languages to do, for real, and
    write a simple test that does that thing. Then run your test.

    Second rule: figure out what to measure. Is execution time the only
    important factor? Or is memory consumption an issue as well? Network
    traffic? Development time? Maintenence effort? Portability? Etc.

    If execution time and memory consumption are both the most important,
    you'd do well to abandon both Ruby and Mono and code in C. Or assembly
    code.

    You can *always* find a situation in which language X will
    "outperform" language Y (for some definition of "outperform"). There
    are things that Ruby does better than Mono, I'm sure (but not being
    familiar with Mono, I'm not qualified to say what they might be).
    Although, at a glance, it looks like the Mono sample took twice as
    many lines of code as the Ruby sample...

    Just some things to think about. Certainly continue your
    investigations--please don't think I'm discouraging that--but make
    sure you are testing the right things, and comparing the right
    metrics.

    - Jamis

    --
    Jamis Buck

    http://jamis.jamisbuck.org
    ------------------------------
    "I am Victor of Borge. You will be assimil-nine-ed."
    Jamis Buck, Feb 6, 2005
    #2
    1. Advertising

  3. Michael Gebhart <> wrote:
    > d=0
    > 1000000000.times {
    > d = d + 1
    > }
    >
    > puts d


    1000000000.times --> 1000000000 method calls to block
    d = d + 1 --> 1000000000 method calls to d
    puts d --> 2 method calls
    -----------------------------------------------------
    2000000002 method calls
    =====================================================

    Well, it sucks, but I guess you might want to realise that you are
    performing *at least* 2000000001 method calls in Ruby, probably *a
    factor* more at the C-level implementation, using the above code.

    I don't think you can really get around that, since even using a for
    loop probably requires that you use a range which might involve more
    method calls.

    I guess that's the price of pervasive and flexible OO throughout the
    language. I wonder if ruby2c might be able to optimise this sort of
    thing.

    How does Mono fare when performing 2000000002 method calls?

    Cheers,
    Navin.
    Navindra Umanee, Feb 6, 2005
    #3
  4. Michael Gebhart

    Kent Sibilev Guest

    Consider this 'tuning':

    $ cat t.rb
    require 'ruby_to_c'

    module Inline
    class Ruby < Inline::C
    def initialize(mod)
    super
    end

    def optimize(meth)
    src = RubyToC.translate(@mod, meth)
    @mod.class_eval "alias :#{meth}_slow :#{meth}"
    @mod.class_eval "remove_method :#{meth}"
    c src
    end
    end
    end

    class Test
    def run
    d=0
    1000000000.downto(1) {
    d = d + 1
    }
    return d
    end

    inline:)Ruby) do |builder|
    builder.optimize :run
    end
    end

    puts Test.new.run

    $ time ruby t.rb
    1000000000

    real 0m3.118s
    user 0m2.670s
    sys 0m0.050s

    $ cat t.cs
    using System;

    class Bench {
    public static void Main() {
    double d = 0;

    for (int i = 0; i < 1000000000; i++) {
    d = d + i;
    }

    Console.WriteLine(d);
    }
    }

    $ mcs t.cs
    Compilation succeeded

    $ time mono t.exe
    4.99999999067109E+17

    real 0m37.692s
    user 0m34.540s
    sys 0m0.040s


    Cheers,
    Kent.

    On Feb 5, 2005, at 10:05 PM, Michael Gebhart wrote:

    > Hi,
    >
    > because of my interest in mono and ruby I have done a small benchmark.
    > These are the results:
    >
    > Mono Code:
    >
    > using System;
    >
    > class Bench {
    > public static void Main() {
    > double d = 0;
    >
    > for (int i = 0; i < 1000000000; i++) {
    > d = d + i;
    > }
    >
    > Console.WriteLine(d);
    > }
    >
    > Needs 10.8 Seconds.
    >
    >
    > In Ruby:
    >
    > d=0
    > 1000000000.times {
    > d = d + 1
    > }
    >
    > puts d
    >
    >
    > Needs: 8 minutes and 20 seconds.
    >
    >
    > Does not look very good :( Is there a possibility to tune my
    > ruby-program,
    > to be as fast as mono?
    >
    > Greetings
    >
    > Mike
    >
    Kent Sibilev, Feb 6, 2005
    #4
  5. Kent Sibilev wrote:
    > Consider this 'tuning':

    [snip]
    >
    > class Test
    > def run
    > d=0
    > 1000000000.downto(1) {
    > d = d + 1
    > }

    [snip]
    > double d = 0;
    >
    > for (int i = 0; i < 1000000000; i++) {
    > d = d + i;
    > }


    To be fair, the type of the variable in the Mono example is "double",
    but the Ruby example uses an int. A more appropriate Ruby version:

    def run
    d = 0.0
    1000000000.downto(1) {
    d = d + 1.0
    }
    return d
    end

    Alexander.
    Alexander Staubo, Feb 6, 2005
    #5
  6. Michael Gebhart

    Kent Sibilev Guest

    That's true, but this example is meaningless anyway. The point is that
    if you find more than a couple of places in your application where Ruby
    does not provide an acceptable performance, you probably should use
    another language for the task. I just wanted to demonstrate that with
    the help of RubyInline or even Ruby2C you can translate these
    bottleneck spots without much of the hassle.

    Cheers,
    Kent.

    On Feb 6, 2005, at 1:14 AM, Alexander Staubo wrote:

    > Kent Sibilev wrote:
    >> Consider this 'tuning':

    > [snip]
    >> class Test
    >> def run
    >> d=0
    >> 1000000000.downto(1) {
    >> d = d + 1
    >> }

    > [snip]
    >> double d = 0;
    >> for (int i = 0; i < 1000000000; i++) {
    >> d = d + i;
    >> }

    >
    > To be fair, the type of the variable in the Mono example is "double",
    > but the Ruby example uses an int. A more appropriate Ruby version:
    >
    > def run
    > d = 0.0
    > 1000000000.downto(1) {
    > d = d + 1.0
    > }
    > return d
    > end
    >
    > Alexander.
    >
    Kent Sibilev, Feb 6, 2005
    #6
  7. Michael Gebhart

    Zach Dennis Guest

    Kent,

    this was pretty hella cool. I'm going to check out Inline::C now that I
    see an easy example and your benchmark. =) Gratzi,

    Zach
    Zach Dennis, Feb 6, 2005
    #7
  8. WOW,

    I am impressed!

    George
    George Moschovitis, Feb 6, 2005
    #8
  9. Michael Gebhart

    Sam Roberts Guest

    Quoteing , on Sun, Feb 06, 2005 at 12:05:12PM +0900:
    > Hi,
    >
    > because of my interest in mono and ruby I have done a small benchmark.


    Ruby is an interpreted language. Its strengths are speed of development
    and flexibility of implementation. C# is a compiled language.

    Java and C# exist in a kind of half-world - more flexible than C++ in
    some ways (GC, at least), but not as fast as the lower-level languages.

    If you want to do fast math, you should be working in C or C++. If you
    want both speed and a beautiful OO language, you write the math in C,
    and bind it into ruby.

    People doing cryptography in .Net do the same -> all the crypto code is
    C/assembler, and it is bound in.

    Cheers,
    Sam
    Sam Roberts, Feb 6, 2005
    #9
  10. On Feb 6, 2005, at 10:07 PM, Caio Tiago Oliveira wrote:
    > I've made this benchmarking in some more languages (using d=d+1):
    >
    > In my Pentium 4 2Ghz 256Mb SDRAM 7200rpm:
    >
    > Java: 00m14,45s (d as long)
    > Java: 00m12,70s (d as double)
    > C: 00m03,27s (d as long int)
    > C: 00m15,90s (d as double)
    > Squeak Smalltalk: 01m48,60s
    > Ruby: 18m12,21s
    > Python: 14m04,83s


    with c+llvm it is instantaneous.
    Alex
    Alexander Kellett, Feb 6, 2005
    #10
  11. (In response to news:p by
    Michael Gebhart)

    > In Ruby:
    >
    > d=0
    > 1000000000.times {
    > d = d + 1
    > }


    As far as I can see, no one tried YARV. I didn't run these tests myself,
    they are the current benchmarks from the YARV-devel list. And yes, the
    numbers different, but you get the picture:

    whileloop:
    i = 0
    while i<10000000
    i+=1
    end
    --
    user system total real
    ruby 18.780000 0.050000 18.830000 ( 18.944069)
    yarv 1.210000 0.000000 1.210000 ( 1.215278)

    simpleiter:
    1000000.times{|simpleiter|
    simpleiter
    }
    --
    user system total real
    ruby 1.130000 0.000000 1.130000 ( 1.122475)
    yarv 0.470000 0.000000 0.470000 ( 0.473542)

    --
    kaspar

    code manufacture - ruby lab
    www.tua.ch/ruby
    Kaspar Schiess, Feb 6, 2005
    #11
  12. Is that with optimizations on (for C)?

    On Mon, 7 Feb 2005 06:07:40 +0900, Caio Tiago Oliveira
    <> wrote:
    > Alexander Staubo, 6/2/2005 03:14:
    >
    > > To be fair, the type of the variable in the Mono example is "double",
    > > but the Ruby example uses an int. A more appropriate Ruby version:

    >
    > I've made this benchmarking in some more languages (using d=d+1):
    >
    > In my Pentium 4 2Ghz 256Mb SDRAM 7200rpm:
    >
    > Java: 00m14,45s (d as long)
    > Java: 00m12,70s (d as double)
    > C: 00m03,27s (d as long int)
    > C: 00m15,90s (d as double)
    > Squeak Smalltalk: 01m48,60s
    > Ruby: 18m12,21s
    > Python: 14m04,83s
    >
    > I couldn't understand why the C code could be slower than the Java one
    > (is the same code, as java and C have similar sintaxes, the java one has
    > the boilerpart and is it).
    >
    > SmallTalk is 100% OOo, with a lot of message calls and is much faster
    > for this task than Ruby or Python.
    >
    >
    Michael Walter, Feb 6, 2005
    #12
  13. Michael Walter, 6/2/2005 19:40:
    > Is that with optimizations on (for C)?
    >
    > On Mon, 7 Feb 2005 06:07:40 +0900, Caio Tiago Oliveira wrote:
    >> I've made this benchmarking in some more languages (using d=d+1):
    >>
    >> In my Pentium 4 2Ghz 256Mb SDRAM 7200rpm:
    >>
    >> Java: 00m14,45s (d as long)
    >> Java: 00m12,70s (d as double)
    >> C: 00m03,27s (d as long int)
    >> C: 00m15,90s (d as double)
    >> Squeak Smalltalk: 01m48,60s
    >> Ruby: 18m12,21s
    >> Python: 14m04,83s



    No optimizations on. I never used it and the code someone posted here
    sounds a quite complicated for a little task. It may be interesting, but
    the better would be something transparent to the developer.
    Caio Tiago Oliveira, Feb 7, 2005
    #13
  14. On Mon, 7 Feb 2005 09:40:18 +0900, Caio Tiago Oliveira
    <> wrote:
    > Michael Walter, 6/2/2005 19:40:
    > > Is that with optimizations on (for C)?
    > >
    > > On Mon, 7 Feb 2005 06:07:40 +0900, Caio Tiago Oliveira wrote:
    > >> I've made this benchmarking in some more languages (using d=d+1):
    > >>
    > >> In my Pentium 4 2Ghz 256Mb SDRAM 7200rpm:
    > >>
    > >> Java: 00m14,45s (d as long)
    > >> Java: 00m12,70s (d as double)
    > >> C: 00m03,27s (d as long int)
    > >> C: 00m15,90s (d as double)
    > >> Squeak Smalltalk: 01m48,60s
    > >> Ruby: 18m12,21s
    > >> Python: 14m04,83s

    >
    > No optimizations on.

    OK. It makes no sense comparing times, then.

    > I never used it and the code someone posted here
    > sounds a quite complicated for a little task. It may be interesting, but
    > the better would be something transparent to the developer.

    I don't understand what you mean.

    Greetings,
    Michael
    Michael Walter, Feb 7, 2005
    #14
  15. Michael Walter, 6/2/2005 22:40:
    > On Mon, 7 Feb 2005 09:40:18 +0900, Caio Tiago Oliveira wrote:
    >> No optimizations on.

    > OK. It makes no sense comparing times, then.
    >
    >> I never used it and the code someone posted here sounds a quite
    >> complicated for a little task. It may be interesting, but the
    >> better would be something transparent to the developer.

    > I don't understand what you mean.


    The optimizations are hard to do.
    Caio Tiago Oliveira, Feb 7, 2005
    #15
  16. I'm talking about compiler switches. :)


    On Mon, 7 Feb 2005 10:45:27 +0900, Caio Tiago Oliveira
    <> wrote:
    > Michael Walter, 6/2/2005 22:40:
    > > On Mon, 7 Feb 2005 09:40:18 +0900, Caio Tiago Oliveira wrote:
    > >> No optimizations on.

    > > OK. It makes no sense comparing times, then.
    > >
    > >> I never used it and the code someone posted here sounds a quite
    > >> complicated for a little task. It may be interesting, but the
    > >> better would be something transparent to the developer.

    > > I don't understand what you mean.

    >
    > The optimizations are hard to do.
    >
    >
    Michael Walter, Feb 7, 2005
    #16
  17. On 12:05 Sun 06 Feb , Michael Gebhart wrote:
    > Mono Code:
    >
    > using System;
    >
    > class Bench {
    > public static void Main() {
    > double d = 0;
    >
    > for (int i = 0; i < 1000000000; i++) {
    > d = d + i;
    > }
    >
    > Console.WriteLine(d);
    > }
    >
    > Needs 10.8 Seconds.

    Ruby code:

    time ruby -e 'puts (999999999 ** 2 + 999999999) / 2'
    499999999500000000

    real 0m0.034s
    user 0m0.010s
    sys 0m0.010s

    Proving once again that algorithm choice is far more important than
    language choice. :)

    C
    Charles Miller, Feb 7, 2005
    #17
  18. Michael Gebhart

    martinus Guest

    And proves once again that the best optimizer sits behind the ears :)

    martinus
    martinus, Feb 7, 2005
    #18
  19. Michael Gebhart

    backdoc Guest

    Charles Miller wrote:
    > On 12:05 Sun 06 Feb , Michael Gebhart wrote:
    >
    >> Mono Code:
    >>
    >> using System;
    >>
    >> class Bench {
    >> public static void Main() {
    >> double d = 0;
    >>
    >> for (int i = 0; i < 1000000000; i++) {
    >> d = d + i;
    >> }
    >>
    >> Console.WriteLine(d);
    >> }
    >>
    >> Needs 10.8 Seconds.

    >
    > Ruby code:
    >
    > time ruby -e 'puts (999999999 ** 2 + 999999999) / 2'
    > 499999999500000000
    >
    > real 0m0.034s
    > user 0m0.010s
    > sys 0m0.010s
    >
    > Proving once again that algorithm choice is far more important than
    > language choice. :)
    >
    > C
    >
    >
    >


    Wow! I didn't realize that ruby could be executed from the command line
    like that. That's awesome.

    darren
    backdoc, Feb 8, 2005
    #19
  20. Hi,

    great! Really great!!

    Thanks

    Mike
    Michael Gebhart, Feb 8, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Manat K.

    Mono

    Manat K., Aug 9, 2003, in forum: ASP .Net
    Replies:
    3
    Views:
    789
    Natty Gur
    Aug 11, 2003
  2. Replies:
    1
    Views:
    2,735
    Scott Allen
    May 30, 2005
  3. Hal Fulton

    Ruby and .NET/Mono

    Hal Fulton, Aug 10, 2004, in forum: Ruby
    Replies:
    1
    Views:
    86
    Tim Sutherland
    Aug 10, 2004
  4. Hal Fulton

    Ruby and Mono

    Hal Fulton, Aug 11, 2004, in forum: Ruby
    Replies:
    11
    Views:
    230
    Robert Feldt
    Aug 12, 2004
  5. Michael Gebhart

    Gtk/Gnome-Applications with Ruby or Mono

    Michael Gebhart, Feb 5, 2005, in forum: Ruby
    Replies:
    7
    Views:
    96
    Nikolai Weibull
    Feb 5, 2005
Loading...

Share This Page