the new great computer language shootout

Discussion in 'Ruby' started by Phil Tomson, Jun 17, 2004.

  1. Phil Tomson

    Phil Tomson Guest

    FYI: The great computer language shootout is now being hosted by one of
    the Debian groups. It's been updated to use the latest
    compilers/interpreters (Ruby 1.8.1 is in there now).

    You can find it here:
    http://shootout.alioth.debian.org/index.php

    It's interesting that Python has now moved ahead of Perl in performance -
    in the older shootout Python was still slower than Perl. Ruby is
    somewhere near the middle of the scale ( slower than both Perl and
    Python).

    Phil
     
    Phil Tomson, Jun 17, 2004
    #1
    1. Advertising

  2. Part of the speedup appears to be from their use of the Psyco JIT
    compiler for the Python code -- the links to source code for Python
    end in '.psyco'.

    I would guess that some of the Ruby code needs to be optimized, as
    well; for example, in the Array Access test, Ruby places dead last.
    That's rediculous, and I don't think it's indicative of the array
    performance for general cases.

    Lennon
     
    Lennon Day-Reynolds, Jun 17, 2004
    #2
    1. Advertising

  3. Phil Tomson

    Eric Hodel Guest

    --4C6bbPZ6c/S1npyF
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline
    Content-Transfer-Encoding: quoted-printable

    Lennon Day-Reynolds () wrote:

    > I would guess that some of the Ruby code needs to be optimized, as
    > well; for example, in the Array Access test, Ruby places dead last.
    > That's rediculous, and I don't think it's indicative of the array
    > performance for general cases.


    This is because of the use of Numeric#step. The implementation is
    flawed.

    --=20
    Eric Hodel - - http://segment7.net
    All messages signed with fingerprint:
    FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04


    --4C6bbPZ6c/S1npyF
    Content-Type: application/pgp-signature
    Content-Disposition: inline

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.2 (FreeBSD)

    iD8DBQFA0hysMypVHHlsnwQRAn7lAKC7pTzxm5piJAGGMyE8LeBDRtsSzQCeNAe0
    kpYB9vsrPkXq6ggbsRhEjyw=
    =FSDa
    -----END PGP SIGNATURE-----

    --4C6bbPZ6c/S1npyF--
     
    Eric Hodel, Jun 17, 2004
    #3
  4. Phil Tomson

    Eric Hodel Guest

    --9Q2l3mYpK16UQ/iv
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline
    Content-Transfer-Encoding: quoted-printable

    Eric Hodel () wrote:

    > This is because of the use of Numeric#step. The implementation is
    > flawed.


    Never mind, I am smoking crack.

    The real problem appears to be iteration in general.

    --=20
    Eric Hodel - - http://segment7.net
    All messages signed with fingerprint:
    FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04


    --9Q2l3mYpK16UQ/iv
    Content-Type: application/pgp-signature
    Content-Disposition: inline

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.2 (FreeBSD)

    iD8DBQFA0iMnMypVHHlsnwQRAmZ2AKDKWgyuQqXnAlx+dGAo1m2wUPuiKwCgqto+
    E0Ov0cb46p3ABluneTelINM=
    =L/2K
    -----END PGP SIGNATURE-----

    --9Q2l3mYpK16UQ/iv--
     
    Eric Hodel, Jun 18, 2004
    #4
  5. To be honest, some of the numbers seem highly suspect to me. For
    example, on my P4/Fedora Core 1 box at least, the Sieve of
    Eratosthenes example runs 20-30% *faster* in Ruby than it does in
    Python, while his dataset shows is taking almost twice as long.

    Lennon
     
    Lennon Day-Reynolds, Jun 18, 2004
    #5
  6. Phil Tomson

    Mark Hubbart Guest

    On Jun 17, 2004, at 4:03 PM, Eric Hodel wrote:

    > Eric Hodel () wrote:
    >
    >> This is because of the use of Numeric#step. The implementation is
    >> flawed.

    >
    > Never mind, I am smoking crack.
    >
    > The real problem appears to be iteration in general.


    surprisingly, using += on line 18 give a 12% speedup on my box.

    cheers,
    Mark
     
    Mark Hubbart, Jun 18, 2004
    #6
  7. On Fri, 18 Jun 2004 08:19:24 +0900, Lennon Day-Reynolds
    <> wrote:
    >
    > To be honest, some of the numbers seem highly suspect to me. For
    > example, on my P4/Fedora Core 1 box at least, the Sieve of
    > Eratosthenes example runs 20-30% *faster* in Ruby than it does in
    > Python, while his dataset shows is taking almost twice as long.


    Are you using psyco? As has been mentioned, it looks as if the python
    stuff has been done using that.
     
    Michael Campbell, Jun 18, 2004
    #7
  8. The Sieve results I got weren't using Psyco; using it, the difference
    in performance is about what the shootout lists, proportionally.
    However, it requires source-level changes to the files, and the source
    code on the shootout site doesn't included those changes. It's also
    not mentioned in the language information page for Python, or listed
    as a seperate implementation.

    Psyco does some great things for the performance of certain
    algorithms, but it's not part of the core Python distribution for any
    number of reasons: only works on x86, breaks several standard
    introspection and debugging libraries, and perhaps most importantly,
    prevents redefinition of methods at runtime on optimized objects.

    In short, it's almost more like a specialized reimplementation of
    Python (ala Stackless) than an add-on library. I don't think it's
    significantly more appropriate to use for this kind of benchmarking
    than, say, Ruby or Perl scripts rewritten to use one of the Inline
    libraries to do the heavy listing in C...especially when it's not
    explicitly mentioned on the benchmark site.

    Lennon
     
    Lennon Day-Reynolds, Jun 18, 2004
    #8
  9. On Fri, 18 Jun 2004 08:57:59 +0900, Lennon Day-Reynolds
    <> wrote:
    >
    > The Sieve results I got weren't using Psyco; using it, the difference
    > in performance is about what the shootout lists, proportionally.
    > However, it requires source-level changes to the files, and the source
    > code on the shootout site doesn't included those changes. It's also
    > not mentioned in the language information page for Python, or listed
    > as a seperate implementation.


    ...

    No disputes from me, I was just mentioning I think that's what they
    did, and that it might explain the discrepancy.
     
    Michael Campbell, Jun 18, 2004
    #9
  10. Phil Tomson

    Phil Tomson Guest

    In article <>,
    Michael Campbell <> wrote:
    >On Fri, 18 Jun 2004 08:19:24 +0900, Lennon Day-Reynolds
    ><> wrote:
    >>
    >> To be honest, some of the numbers seem highly suspect to me. For
    >> example, on my P4/Fedora Core 1 box at least, the Sieve of
    >> Eratosthenes example runs 20-30% *faster* in Ruby than it does in
    >> Python, while his dataset shows is taking almost twice as long.

    >
    >Are you using psyco? As has been mentioned, it looks as if the python
    >stuff has been done using that.
    >


    Isn't psycho kind of a special varient of Python? Is it fair to compare
    psycho Python in the language shootout (they should probalby use the more
    widely used PythonC).

    Phil
     
    Phil Tomson, Jun 18, 2004
    #10
  11. I agree that they really should stick strictly to Python code if they
    are comparing languages. We don't want to know how well C performs
    with a Python outer shell, we want to know how well Python performs.

    Carl

    On Fri, 18 Jun 2004 09:53:24 +0900, Phil Tomson <> wrote:
    >
    > In article <>,
    >
    >
    > Michael Campbell <> wrote:
    > >On Fri, 18 Jun 2004 08:19:24 +0900, Lennon Day-Reynolds
    > ><> wrote:
    > >>
    > >> To be honest, some of the numbers seem highly suspect to me. For
    > >> example, on my P4/Fedora Core 1 box at least, the Sieve of
    > >> Eratosthenes example runs 20-30% *faster* in Ruby than it does in
    > >> Python, while his dataset shows is taking almost twice as long.

    > >
    > >Are you using psyco? As has been mentioned, it looks as if the python
    > >stuff has been done using that.
    > >

    >
    > Isn't psycho kind of a special varient of Python? Is it fair to compare
    > psycho Python in the language shootout (they should probalby use the more
    > widely used PythonC).
    >
    > Phil
    >
    >
     
    Carl Youngblood, Jun 18, 2004
    #11
  12. Phil Tomson

    GGarramuno Guest

    Eric Hodel <> wrote in message news:<>...
    > Lennon Day-Reynolds () wrote:
    >
    > > I would guess that some of the Ruby code needs to be optimized, as
    > > well; for example, in the Array Access test, Ruby places dead last.
    > > That's rediculous, and I don't think it's indicative of the array
    > > performance for general cases.

    >
    > This is because of the use of Numeric#step. The implementation is
    > flawed.


    Huh? In what way is it flawed? While there you can use slicing for
    some operations, other array operations need iteration thru the array
    as showed.
    I do find the shootout is pretty accurate in terms of real ruby
    performance overall.
    The new web page is pretty bad in that most links are broken due to
    typos and what not, so it is not clear to me the python tests are
    based on psyco. There does appear to be a separate psyco benchmark.
     
    GGarramuno, Jun 18, 2004
    #12
  13. Phil Tomson

    David Ross Guest

    On Friday 18 June 2004 01:03 am, GGarramuno wrote:
    > Eric Hodel <> wrote in message
    > news:<>...
    >
    > > Lennon Day-Reynolds () wrote:
    > > > I would guess that some of the Ruby code needs to be optimized, as
    > > > well; for example, in the Array Access test, Ruby places dead last.
    > > > That's rediculous, and I don't think it's indicative of the array
    > > > performance for general cases.

    > >
    > > This is because of the use of Numeric#step. The implementation is
    > > flawed.

    >



    > Huh? In what way is it flawed? While there you can use slicing for

    In what way is it flawed? The whole thing is flawed. It is not fair to judge
    the languages like this. Especially when they are compiled with GNU gcc, the
    optimizations are really poor. It would be fair if someone used an optimized
    compiler to compile ruby and the others then compete. As you can see Java
    apears as first on some of the lists. I will state the fact that Java is not
    compiled by GCC. They are probably using a optimized comiler. which makes
    everything faster. I would like to see someone compiler ruby by Intel's
    Compiler ICC. I'm sure Ruby would then be better than java in tests. Java has
    optimized code and assembly. theres no possible way much of anything can be
    better when they are the ones using optimized comiler and tossing out the
    binaries. ---David Ross

    > some operations, other array operations need iteration thru the array
    > as showed.
    > I do find the shootout is pretty accurate in terms of real ruby
    > performance overall.
    > The new web page is pretty bad in that most links are broken due to
    > typos and what not, so it is not clear to me the python tests are
    > based on psyco. There does appear to be a separate psyco benchmark.
     
    David Ross, Jun 18, 2004
    #13
  14. Phil Tomson wrote:
    > Isn't psycho kind of a special varient of Python? Is it fair
    > to compare psycho Python in the language shootout (they
    > should probalby use the more widely used PythonC).


    You may be thinking of Pyrex, a Python dialect that compiles to C and lets
    you mix C and Python data types:

    http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/

    Pyrex is really quite nifty, especially for writing C extensions to Python
    and for wrapping existing C libraries. You can still code in Python or close
    to it, and let Pyrex write all the boilerplate C code for you. For
    performance critical algorithms, if you're using C types the generated code
    should be as fast as any other C code.

    Psyco, OTOH, works with standard Python code. It's essentially a Just In
    Time compiler for Python:

    http://psyco.sourceforge.net/introduction.html

    Psyco isn't part of the standard Python distribution--you need to install
    the Psyco module and add this to the top of your source file:

    import psyco; psyco.full()

    But that's all there is to it. There's nothing special about the
    language--it's standard Python.

    Psyco is only for x86 at the moment, but it can really pep up Python code on
    the x86.

    What's fair for performance comparisons? Well, a thorough language benchmark
    would measure Psyco, standard cPython, and Jython times for x86, plus
    cPython and Jython for other popular processors.

    These are all Open Source, so don't let anybody stop you from porting them
    to Ruby... :)

    -Mike
     
    Michael Geary, Jun 18, 2004
    #14
  15. Phil Tomson

    Joao Pedrosa Guest

    Hi,

    > What's fair for performance comparisons? Well, a
    > thorough language benchmark
    > would measure Psyco, standard cPython, and Jython
    > times for x86, plus
    > cPython and Jython for other popular processors.
    >
    > These are all Open Source, so don't let anybody stop
    > you from porting them
    > to Ruby... :)


    That's a lot of pythons. :)

    I'll keep my precious Ruby, and worry about
    performance later, when my programs and libraries are
    written. :)

    Perhaps one of the main reasons to use a script
    language, is to use it as a glue language, and I'm
    fine with that.

    Cheers,
    Joao



    __________________________________
    Do you Yahoo!?
    New and Improved Yahoo! Mail - Send 10MB messages!
    http://promotions.yahoo.com/new_mail
     
    Joao Pedrosa, Jun 18, 2004
    #15
  16. On Fri, Jun 18, 2004 at 08:57:59AM +0900, Lennon Day-Reynolds wrote:
    > The Sieve results I got weren't using Psyco; using it, the difference
    > in performance is about what the shootout lists, proportionally.
    > However, it requires source-level changes to the files, and the source
    > code on the shootout site doesn't included those changes. It's also
    > not mentioned in the language information page for Python, or listed
    > as a seperate implementation.
    >
    > Psyco does some great things for the performance of certain
    > algorithms, but it's not part of the core Python distribution for any
    > number of reasons: only works on x86, breaks several standard
    > introspection and debugging libraries, and perhaps most importantly,
    > prevents redefinition of methods at runtime on optimized objects.


    Hmm, if we'd had namespace selectors in Ruby, such optimizations (e.g.
    optimizing integer/floating point operations) would become possible, at
    least theoretically, without breaking code or loosing introspection at
    the overall level. I might be wrong, of course :)

    Regards,

    Michael
     
    Michael Neumann, Jun 18, 2004
    #16
  17. On Fri, Jun 18, 2004 at 05:38:20PM +0900, Michael Neumann wrote:
    > > Psyco does some great things for the performance of certain
    > > algorithms, but it's not part of the core Python distribution for any
    > > number of reasons: only works on x86, breaks several standard
    > > introspection and debugging libraries, and perhaps most importantly,
    > > prevents redefinition of methods at runtime on optimized objects.

    >
    > Hmm, if we'd had namespace selectors in Ruby, such optimizations (e.g.
    > optimizing integer/floating point operations) would become possible, at
    > least theoretically, without breaking code or loosing introspection at


    They are actually possible with the current language; you'd "only"
    need a code specializer + careful code cache invalidation (take a look
    at the Self papers).

    > the overall level. I might be wrong, of course :)


    --
    Running Debian GNU/Linux Sid (unstable)
    batsman dot geo at yahoo dot com

    Q: What's the big deal about rm, I have been deleting stuff for years? And
    never lost anything.. oops!
    A: ...
    -- From the Frequently Unasked Questions
     
    Mauricio Fernández, Jun 18, 2004
    #17
  18. On Fri, Jun 18, 2004 at 05:56:30PM +0900, Mauricio Fernndez wrote:
    > On Fri, Jun 18, 2004 at 05:38:20PM +0900, Michael Neumann wrote:
    > > > Psyco does some great things for the performance of certain
    > > > algorithms, but it's not part of the core Python distribution for any
    > > > number of reasons: only works on x86, breaks several standard
    > > > introspection and debugging libraries, and perhaps most importantly,
    > > > prevents redefinition of methods at runtime on optimized objects.

    > >
    > > Hmm, if we'd had namespace selectors in Ruby, such optimizations (e.g.
    > > optimizing integer/floating point operations) would become possible, at
    > > least theoretically, without breaking code or loosing introspection at

    >
    > They are actually possible with the current language; you'd "only"
    > need a code specializer + careful code cache invalidation (take a look
    > at the Self papers).


    Hm, but isn't that much harder to implement? And you need all these
    tests at runtime (well, should be no performance penalty).

    For example:

    a = 1
    a += 2

    This piece of code would become invalid, when the Integer#+ method is
    redefined.

    Hm, indeed, doesn't look imposible for me. What are the advantages (of
    both)?

    Regards,

    Michael
     
    Michael Neumann, Jun 18, 2004
    #18
  19. On Fri, Jun 18, 2004 at 06:29:32PM +0900, Michael Neumann wrote:
    > > They are actually possible with the current language; you'd "only"
    > > need a code specializer + careful code cache invalidation (take a look
    > > at the Self papers).

    >
    > Hm, but isn't that much harder to implement? And you need all these

    ========================

    Indeed. The Self team needed many man-years and they knew what they were
    doing. It took Sun quite a long time to create an acceptable VM/JIT
    for a much less dynamic language like Java, and they had most of the
    original Self crew working for them :p

    > tests at runtime (well, should be no performance penalty).
    >
    > For example:
    >
    > a = 1
    > a += 2
    >
    > This piece of code would become invalid, when the Integer#+ method is
    > redefined.


    That's what I referred to with 'code cache invalidation'; on
    redefinition of Integer#+, all specialized code segments that used the
    old one would be flushed out. This is performed on method redefinition,
    which should be quite rare.

    > Hm, indeed, doesn't look imposible for me. What are the advantages (of
    > both)?


    "Both"? If you're referring to selector namespaces, they don't simplify
    the runtime needs that much, unless you forbid method redefinitions.
    There could be some small speed increase, since code invalidation would
    be a bit less frequent (would of course depend on the performance of
    the code specializer).

    --
    Running Debian GNU/Linux Sid (unstable)
    batsman dot geo at yahoo dot com

    Linux: Where Don't We Want To Go Today?
    -- Submitted by Pancrazio De Mauro, paraphrasing some well-known sales talk
     
    Mauricio Fernández, Jun 18, 2004
    #19
  20. Phil Tomson

    denis Guest

    Mark Hubbart <> wrote in message news:<>...
    > On Jun 17, 2004, at 4:03 PM, Eric Hodel wrote:
    >
    > > Eric Hodel () wrote:
    > >
    > >> This is because of the use of Numeric#step. The implementation is
    > >> flawed.

    > >
    > > Never mind, I am smoking crack.
    > >
    > > The real problem appears to be iteration in general.

    >
    > surprisingly, using += on line 18 give a 12% speedup on my box.
    >
    > cheers,
    > Mark


    It is not so surprising += is always faster in Ruby (from my
    experience).This change would place us between mzscheme and mawk.
    Note that many other languages (perl, icon lua...) uses += so it
    should be allowed.

    Ciao

    Denis
     
    denis, Jun 18, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jacob Lee

    code for Computer Language Shootout

    Jacob Lee, Mar 16, 2005, in forum: Python
    Replies:
    14
    Views:
    632
  2. Delaney, Timothy C (Timothy)

    RE: code for Computer Language Shootout

    Delaney, Timothy C (Timothy), Mar 16, 2005, in forum: Python
    Replies:
    3
    Views:
    314
    Raymond Hettinger
    Mar 17, 2005
  3. Replies:
    14
    Views:
    509
    Fredrik Lundh
    Nov 30, 2005
  4. Isaac Gouy
    Replies:
    0
    Views:
    118
    Isaac Gouy
    Aug 3, 2004
  5. Replies:
    9
    Views:
    199
    Isaac Gouy
    Mar 29, 2005
Loading...

Share This Page