python/ruby benchmark.

Discussion in 'Ruby' started by \, Jun 9, 2005.

  1. \

    \ Guest

    \, Jun 9, 2005
    #1
    1. Advertising

  2. "</script> ha scritto:
    > I took a look at
    > http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu
    > and found the tests on ruby disapointing :(.
    > So what I'm asking for is a link to some ruby-specific optimisation tips.


    a simple thing: do the most normal thing instead of writing code in ruby
    that is innatural, i.e. instead of rewritine an heapsort by hand, use
    the builtin sort/sort!/sort_by routines. Instead of writing a random
    function by hand, use the builtin rand().

    If you really feel ruby (or python) is slow you can act later optimizing
    it, not doing it before you ever need to be faster. "premature
    optimization is the root of all evil", people say.
     
    gabriele renzi, Jun 9, 2005
    #2
    1. Advertising

  3. \

    Glenn Parker Guest

    > So what I'm asking for is a link to some ruby-specific optimisation tips.

    Avoid creating new objects, e.g. re-use strings and arrays, or modify
    them in place when possible, especially large ones.

    Use built-in iterators as much as possible.

    Pray for YARV! :)

    --
    Glenn Parker | glenn.parker-AT-comcast.net | <http://www.tetrafoil.com/>
     
    Glenn Parker, Jun 9, 2005
    #3
  4. As Gabriele mentionned, they implement a lot of stuff that is done in C
    in Ruby. Also, the tests must use the exact same algorithm, so this
    disadvantages Ruby is some situations. I say the test should compare
    same algorithms, but also the one best suited to the language.
     
    Vincent Foley, Jun 9, 2005
    #4
  5. Hello gabriele,

    gr> "</script> ha scritto:
    >> I took a look at
    >> http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu
    >> and found the tests on ruby disapointing :(.
    >> So what I'm asking for is a link to some ruby-specific optimisation tips.


    gr> a simple thing: do the most normal thing instead of writing code in ruby
    gr> that is innatural, i.e. instead of rewritine an heapsort by hand, use
    gr> the builtin sort/sort!/sort_by routines. Instead of writing a random
    gr> function by hand, use the builtin rand().

    Sorry you don't understand this benchmark. It is there to do this
    algorithms to compare the basic data operations.

    The benchmark is not useable to compare compiled languages with
    interpretered, but comparing python with ruby is a valid comparision.

    And that ruby is so worse then python is a real problem. If you still
    try to deny this, then you have a real mental problem.

    If ruby is 9 times slower then python in for example function calling intensive
    code that means that ruby will slow down your development time a lot
    as it forces you to use optimized code or even C code much more often
    then python.


    --
    Best regards, emailto: scholz at scriptolutions dot com
    Lothar Scholz http://www.ruby-ide.com
    CTO Scriptolutions Ruby, PHP, Python IDE 's
     
    Lothar Scholz, Jun 9, 2005
    #5
  6. \

    Kent Sibilev Guest

    Java is an order of magnitude faster than Ruby. The development of a
    web app using Rails is an order of magnitude faster than the
    development of a similar web app using Struts/Hibernate frameworks.
    There are other factors that can slow down your development time
    besides the speed of your language of choice.

    Cheers,
    Kent
    Just another guy with huge mental problems.

    On 6/9/05, Lothar Scholz <> wrote:
    > Hello gabriele,
    >=20
    > gr> "</script> ha scritto:
    > >> I took a look at
    > >> http://shootout.alioth.debian.org/benchmark.php?test=3Dall&lang=3Druby=

    &lang2=3Dpython&sort=3Dfullcpu
    > >> and found the tests on ruby disapointing :(.
    > >> So what I'm asking for is a link to some ruby-specific optimisation ti=

    ps.
    >=20
    > gr> a simple thing: do the most normal thing instead of writing code in r=

    uby
    > gr> that is innatural, i.e. instead of rewritine an heapsort by hand, use
    > gr> the builtin sort/sort!/sort_by routines. Instead of writing a random
    > gr> function by hand, use the builtin rand().
    >=20
    > Sorry you don't understand this benchmark. It is there to do this
    > algorithms to compare the basic data operations.
    >=20
    > The benchmark is not useable to compare compiled languages with
    > interpretered, but comparing python with ruby is a valid comparision.
    >=20
    > And that ruby is so worse then python is a real problem. If you still
    > try to deny this, then you have a real mental problem.
    >=20
    > If ruby is 9 times slower then python in for example function calling int=

    ensive
    > code that means that ruby will slow down your development time a lot
    > as it forces you to use optimized code or even C code much more often
    > then python.
    >=20
    >=20
    > --
    > Best regards, emailto: scholz at scriptolutions d=

    ot com
    > Lothar Scholz http://www.ruby-ide.com
    > CTO Scriptolutions Ruby, PHP, Python IDE 's
    >=20
    >=20
    >=20
    >
     
    Kent Sibilev, Jun 9, 2005
    #6
  7. Hi,
    I definitely agree.Only feature of a particular programming language
    cannot determine the speed of software.Other factors are also
    crucially important.
    I didnot know about YARV,but gr8 that I came to know it thru u
    people.I'll surely check it out.

    Bye
    Dibya Prakash
    Happy coding

    On 6/9/05, Kent Sibilev <> wrote:
    > Java is an order of magnitude faster than Ruby. The development of a
    > web app using Rails is an order of magnitude faster than the
    > development of a similar web app using Struts/Hibernate frameworks.
    > There are other factors that can slow down your development time
    > besides the speed of your language of choice.
    >=20
    > Cheers,
    > Kent
    > Just another guy with huge mental problems.
    >=20
    > On 6/9/05, Lothar Scholz <> wrote:
    > > Hello gabriele,
    > >
    > > gr> "</script> ha scritto:
    > > >> I took a look at
    > > >> http://shootout.alioth.debian.org/benchmark.php?test=3Dall&lang=3Dru=

    by&lang2=3Dpython&sort=3Dfullcpu
    > > >> and found the tests on ruby disapointing :(.
    > > >> So what I'm asking for is a link to some ruby-specific optimisation =

    tips.
    > >
    > > gr> a simple thing: do the most normal thing instead of writing code in=

    ruby
    > > gr> that is innatural, i.e. instead of rewritine an heapsort by hand, u=

    se
    > > gr> the builtin sort/sort!/sort_by routines. Instead of writing a rando=

    m
    > > gr> function by hand, use the builtin rand().
    > >
    > > Sorry you don't understand this benchmark. It is there to do this
    > > algorithms to compare the basic data operations.
    > >
    > > The benchmark is not useable to compare compiled languages with
    > > interpretered, but comparing python with ruby is a valid comparision.
    > >
    > > And that ruby is so worse then python is a real problem. If you still
    > > try to deny this, then you have a real mental problem.
    > >
    > > If ruby is 9 times slower then python in for example function calling i=

    ntensive
    > > code that means that ruby will slow down your development time a lot
    > > as it forces you to use optimized code or even C code much more often
    > > then python.
    > >
    > >
    > > --
    > > Best regards, emailto: scholz at scriptolutions=

    dot com
    > > Lothar Scholz http://www.ruby-ide.com
    > > CTO Scriptolutions Ruby, PHP, Python IDE 's
    > >
    > >
    > >
    > >

    >=20
    >
     
    Dibya Prakash, Jun 9, 2005
    #7
  8. Hello Kent,

    KS> Java is an order of magnitude faster than Ruby. The development of a
    KS> web app using Rails is an order of magnitude faster than the
    KS> development of a similar web app using Struts/Hibernate frameworks.
    KS> There are other factors that can slow down your development time
    KS> besides the speed of your language of choice.

    If i can't use the language because it is to slow to solve the problem
    then the development time doesn't matter. There are just too many
    problem domains left where you can't ignore it.

    But we all hope and pray for YARV.


    --
    Best regards, emailto: scholz at scriptolutions dot com
    Lothar Scholz http://www.ruby-ide.com
    CTO Scriptolutions Ruby, PHP, Python IDE 's
     
    Lothar Scholz, Jun 9, 2005
    #8
  9. Lothar Scholz ha scritto:
    > Hello gabriele,
    >
    > gr> "</script> ha scritto:
    >
    >>>I took a look at
    >>>http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ruby&lang2=python&sort=fullcpu
    >>>and found the tests on ruby disapointing :(.
    >>>So what I'm asking for is a link to some ruby-specific optimisation tips.

    >
    >
    > gr> a simple thing: do the most normal thing instead of writing code in ruby
    > gr> that is innatural, i.e. instead of rewritine an heapsort by hand, use
    > gr> the builtin sort/sort!/sort_by routines. Instead of writing a random
    > gr> function by hand, use the builtin rand().
    >
    > Sorry you don't understand this benchmark. It is there to do this
    > algorithms to compare the basic data operations.


    I understand the benchmark.
    I'm just saying that it shows "you should not implement X in ruby" not
    that "X takes a lot of time in ruby" since you may not need to implement
    X (i.e. because it is already there or because you would not actually
    ever need a recursive floating point function).

    > The benchmark is not useable to compare compiled languages with
    > interpretered, but comparing python with ruby is a valid comparision.
    >
    > And that ruby is so worse then python is a real problem. If you still
    > try to deny this, then you have a real mental problem.


    I don't think I did

    > If ruby is 9 times slower then python in for example function calling intensive
    > code that means that ruby will slow down your development time a lot
    > as it forces you to use optimized code or even C code much more often
    > then python.


    I agree, It would be nice if ruby was faster. Luckily,
    people are working on this.
     
    gabriele renzi, Jun 10, 2005
    #9
  10. \

    Jim Freeze Guest

    * gabriele renzi <> [2005-06-10 09:35:28 +0900]:

    > I agree, It would be nice if ruby was faster. Luckily,
    > people are working on this.


    Don't go away thinking that X is faster than Ruby. Do
    your own tests. We tested Perl against Ruby in a real
    world problem (netlist parsing) and Ruby was faster
    than Perl by 30%. When we did a C implementation
    (which could now be done automatically with optimize)
    it was 100% faster (ie, took half the time.)

    These statements that X language is faster than
    Ruby is like saying that Einstein is smarter than
    Mozart. In what?

    --
    Jim Freeze
    Theory and practice are the same, in theory. -- Ryan Davis
     
    Jim Freeze, Jun 10, 2005
    #10
  11. \

    Phil Tomson Guest

    In article <>,
    Jim Freeze <> wrote:
    >* gabriele renzi <> [2005-06-10 09:35:28 +0900]:
    >
    >> I agree, It would be nice if ruby was faster. Luckily,
    >> people are working on this.

    >
    >Don't go away thinking that X is faster than Ruby. Do
    >your own tests. We tested Perl against Ruby in a real
    >world problem (netlist parsing) and Ruby was faster
    >than Perl by 30%. When we did a C implementation
    >(which could now be done automatically with optimize)
    >it was 100% faster (ie, took half the time.)


    Are you saying that the Ruby netlist parser was 100% faster than the C
    implementation or the other way around?

    BTW: I recall someone at a PDX.rb meeting who said he had benchmarked
    Ruby vs. OO-style Perl and Ruby was quite a bit faster (don't recall the
    percentage). As I recall the explanation was that doing OO in perl
    requires more levels of indirection and thus more function calls.

    Phil
     
    Phil Tomson, Jun 10, 2005
    #11
  12. \

    Jim Freeze Guest

    * Phil Tomson <> [2005-06-10 13:55:28 +0900]:

    > In article <>,
    > Jim Freeze <> wrote:
    > >* gabriele renzi <> [2005-06-10 09:35:28 +0900]:
    > >
    > >> I agree, It would be nice if ruby was faster. Luckily,
    > >> people are working on this.

    > >
    > >Don't go away thinking that X is faster than Ruby. Do
    > >your own tests. We tested Perl against Ruby in a real
    > >world problem (netlist parsing) and Ruby was faster
    > >than Perl by 30%. When we did a C implementation
    > >(which could now be done automatically with optimize)
    > >it was 100% faster (ie, took half the time.)

    >
    > Are you saying that the Ruby netlist parser was 100% faster than the C
    > implementation or the other way around?


    The C implementation was Ruby C (my bad). So, the Ruby C
    implementation ended up twice as fast as the Perl
    implementation. Bad comparison you say. Well, in this
    environment, we found over four netlist parsers written
    in Perl. These parsers were intermingled with the netlist
    munging code, so it was difficult to do abstraction and
    pull out the netlist parser in Perl and optimize it.

    But you say, well all that could and should have been done in
    Perl. My response is, well, you had 10 years, what were you
    waiting for?

    The pragmatic nature of Ruby lent itself to writing better code,
    more modular code, and thus the ability to optimize a particular
    module muuuchh more than Perl. ;)

    --
    Jim Freeze
    Theory and practice are the same, in theory. -- Ryan Davis
     
    Jim Freeze, Jun 10, 2005
    #12
  13. \

    Isaac Gouy Guest

    Vincent Foley wrote:
    > As Gabriele mentionned, they implement a lot of stuff that is done in C
    > in Ruby.


    Specifically?

    > Also, the tests must use the exact same algorithm, so this
    > disadvantages Ruby is some situations. I say the test should compare
    > same algorithms, but also the one best suited to the language.


    Do this disadvantage Ruby more than PHP?
    http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=php&sort=fullcpu

    Or more than Tcl, or more than GNU Smalltalk?
    http://shootout.alioth.debian.org/great/benchmark.php?test=all&lang=ruby&lang2=gst&sort=fullcpu
     
    Isaac Gouy, Jun 10, 2005
    #13
  14. On 6/10/05, Isaac Gouy <> wrote:
    > > Also, the tests must use the exact same algorithm, so this
    > > disadvantages Ruby is some situations. I say the test should compare
    > > same algorithms, but also the one best suited to the language.

    > Do this disadvantage Ruby more than PHP?
    > Or more than Tcl, or more than GNU Smalltalk?


    When the algorithm, as entered, won't complete, yes. When compared
    against a language that has tail recursion optimisation, or a possible
    situation where optimisation of the compiler deals with recursion
    intelligently? Yes.

    The definition of Ackermann numbers is, I presume, well known. Because
    some platforms can't modify their stack level (Windows), and because
    the stack level isn't modified in the runs for the benchmark, there's
    going to be an issue with the deep recursion. In this case, it would
    be appropriate to reduce the level of recursion. But given the rules
    mentioned, this isn't acceptable.

    What's the *purpose* of something like the Ackermann test? Is it to
    see how quickly the environment winds and unwinds the stack? How much
    state it carries around?

    I personally think that most of the shootout items are bogus in just
    about every way. Don't get me wrong -- I'm not at all suggesting that
    Ruby couldn't be faster. I'm just saying that benchmarks, like
    statistics, are lies.

    -austin
    --=20
    Austin Ziegler *
    * Alternate:
     
    Austin Ziegler, Jun 10, 2005
    #14
  15. \

    Isaac Gouy Guest

    Austin Ziegler wrote:
    > On 6/10/05, Isaac Gouy <> wrote:
    > > > Also, the tests must use the exact same algorithm, so this
    > > > disadvantages Ruby is some situations. I say the test should compare
    > > > same algorithms, but also the one best suited to the language.

    > > Do this disadvantage Ruby more than PHP?
    > > Or more than Tcl, or more than GNU Smalltalk?

    >
    > When the algorithm, as entered, won't complete, yes. When compared
    > against a language that has tail recursion optimisation, or a possible
    > situation where optimisation of the compiler deals with recursion
    > intelligently? Yes.


    Ruby completes ackermann for N=7
    http://shootout.alioth.debian.org/g...ean-0&p3=gcc-3&p4=gcc-0&sort=fullcpu#cputable

    Which is how there can be a comparison between Ruby and Tcl on
    ackermann
    http://shootout.alioth.debian.org/g...st=all&lang=ruby&lang2=tcl&sort=fullcpu#ratio


    > The definition of Ackermann numbers is, I presume, well known. Because
    > some platforms can't modify their stack level (Windows), and because
    > the stack level isn't modified in the runs for the benchmark, there's
    > going to be an issue with the deep recursion.


    We use Debian Linux, so tell us how to modify the stack level for Ruby.


    -snip-
    > I personally think that most of the shootout items are bogus in just
    > about every way. Don't get me wrong -- I'm not at all suggesting that
    > Ruby couldn't be faster. I'm just saying that benchmarks, like
    > statistics, are lies.


    Sounds like me in editorial mode.
    http://shootout.alioth.debian.org/g...cpu&file=benchmarking&title=Flawed Benchmarks
     
    Isaac Gouy, Jun 10, 2005
    #15
  16. On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
    >
    > I personally think that most of the shootout items are bogus in just
    > about every way. Don't get me wrong -- I'm not at all suggesting that
    > Ruby couldn't be faster. I'm just saying that benchmarks, like
    > statistics, are lies.
    >
    > -austin
    > --
    > Austin Ziegler *
    > * Alternate:
    >
    >


    The most succinct way that I've heard it put is that there are "lies,
    damn lies, and benchmarks".

    Trey Campbell
     
    Trey Campbell, Jun 10, 2005
    #16
  17. On 6/10/05, Isaac Gouy <> wrote:
    > Austin Ziegler wrote:
    > > The definition of Ackermann numbers is, I presume, well known. Because
    > > some platforms can't modify their stack level (Windows), and because
    > > the stack level isn't modified in the runs for the benchmark, there's
    > > going to be an issue with the deep recursion.

    > We use Debian Linux, so tell us how to modify the stack level for Ruby.


    I'm sorry you do. However, yours should be one of the ones that should
    be able to modify the depth of the stack with ulimit.

    If I cared enough to get more than annoyed at this damned shootout
    with its inane implementation rules, I'd do more. But as it is, the
    best I can tell people is that the Alioth shootout is bullshit.

    -austin
    --=20
    Austin Ziegler *
    * Alternate:
     
    Austin Ziegler, Jun 10, 2005
    #17
  18. \

    Isaac Gouy Guest

    Trey Campbell wrote:
    > On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
    > >
    > > I personally think that most of the shootout items are bogus in just
    > > about every way. Don't get me wrong -- I'm not at all suggesting that
    > > Ruby couldn't be faster. I'm just saying that benchmarks, like
    > > statistics, are lies.
    > >
    > > -austin
    > > --
    > > Austin Ziegler *
    > > * Alternate:
    > >
    > >

    >
    > The most succinct way that I've heard it put is that there are "lies,
    > damn lies, and benchmarks".
    >
    > Trey Campbell
    >


    There isn't with benchmarks per-se; there can be a problem with how one
    chooses to interpret benchmarks.
     
    Isaac Gouy, Jun 10, 2005
    #18
  19. On 6/10/05, Isaac Gouy <> wrote:
    > Trey Campbell wrote:
    >> On Jun 10, 2005, at 1:02 PM, Austin Ziegler wrote:
    >>> I personally think that most of the shootout items are bogus in
    >>> just about every way. Don't get me wrong -- I'm not at all
    >>> suggesting that Ruby couldn't be faster. I'm just saying that
    >>> benchmarks, like statistics, are lies.

    >> The most succinct way that I've heard it put is that there are
    >> "lies, damn lies, and benchmarks".

    > There isn't with benchmarks per-se; there can be a problem with
    > how one chooses to interpret benchmarks.


    No, it's the benchmarks themselves that are the problem. Remember
    that NVidia got caught detecting a benchmark in their drivers and
    optimizing output for the benchmark.

    Similarly, this thread was started because someone came across your
    language shootout benchmark page and assumed that Ruby is
    dramatically slower in Real Life Examples because it's slower in the
    benchmarks. Which, I can assure you, it most assuredly NOT.

    You get it correct when you indicate that "the best benchmark is
    your program." That's the only one that matters. It also lets you
    measure benchmarks that can't be categorised in CPU seconds or
    memory use or FLOPS or other nonsense like that.

    It's hype, but the idea that Rails makes one 10x more productive is
    part of the value of Ruby. Unless your program is of a special case
    -- and a lot of these special cases are handled already -- then Ruby
    programs will be developed faster and more easily maintained than
    their non-Ruby counterparts. Enough so that execution speed is
    something of a wash.

    At any rate, I consider the Alioth shootout to be harmful to all
    languages involved. There is no useful value provided by it,
    especially as it does not permit language-appropriate modifications
    to the algorithms in use.

    -austin
    --=20
    Austin Ziegler *
    * Alternate:
     
    Austin Ziegler, Jun 10, 2005
    #19
  20. \

    Isaac Gouy Guest

    Austin Ziegler wrote:
    > On 6/10/05, Isaac Gouy <> wrote:
    > > There isn't with benchmarks per-se; there can be a problem with
    > > how one chooses to interpret benchmarks.

    >
    > No, it's the benchmarks themselves that are the problem. Remember
    > that NVidia got caught detecting a benchmark in their drivers and
    > optimizing output for the benchmark.
    >
    > Similarly, this thread was started because someone came across your
    > language shootout benchmark page and assumed that Ruby is
    > dramatically slower in Real Life Examples because it's slower in the
    > benchmarks.


    As I said - There isn't a problem with benchmarks per-se; there can be
    a problem with how one chooses to interpret benchmarks.


    > Which, I can assure you, it most assuredly NOT.


    Not "dramatically slower" than what other language, for what specific
    "Real Life Examples"?


    > It's hype, but the idea that Rails makes one 10x more productive is
    > part of the value of Ruby. Unless your program is of a special case
    > -- and a lot of these special cases are handled already -- then Ruby
    > programs will be developed faster and more easily maintained than
    > their non-Ruby counterparts. Enough so that execution speed is
    > something of a wash.


    Faster to develop-in than what other languages - Smalltalk? Lisp?
    Mathematica?


    > At any rate, I consider the Alioth shootout to be harmful to all
    > languages involved. There is no useful value provided by it,
    > especially as it does not permit language-appropriate modifications
    > to the algorithms in use.


    What's a language-appropriate modification?
    Have you ever asked for one to be accepted in the Alioth shootout?
     
    Isaac Gouy, Jun 10, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dang Griffith

    Re: any benchmark results for python?

    Dang Griffith, Jun 25, 2003, in forum: Python
    Replies:
    0
    Views:
    1,281
    Dang Griffith
    Jun 25, 2003
  2. Bowen Chiu
    Replies:
    1
    Views:
    548
    Skip Montanaro
    Aug 9, 2003
  3. Carl
    Replies:
    52
    Views:
    1,203
    Jacek Generowicz
    Jan 14, 2004
  4. Michael Gebhart

    Benchmark Mono - Ruby

    Michael Gebhart, Feb 6, 2005, in forum: Ruby
    Replies:
    19
    Views:
    229
    Michael Gebhart
    Feb 8, 2005
  5. Phil Tomson
    Replies:
    7
    Views:
    210
    Pit Capitain
    Jun 16, 2005
Loading...

Share This Page