JRuby performance questions answered

Discussion in 'Ruby' started by Charles Oliver Nutter, Oct 31, 2007.

  1. Many folks still seem to be in the dark about JRuby performance relative
    to Ruby 1.8.x. I figured a short post with some basic info could start a
    discussion for anyone wanting to know more.

    Long story short, JRuby 1.1 is now generally faster than Ruby 1.8.6, and
    for most execution benchmarks it is the fastest 1.8-compatible Ruby
    implementation available. This is based on the YARV benchmarks, the
    Alioth benchmarks, a number of benchmarks we created for JRuby, and a
    few other external benchmarks we run from time to time. Many (most?) of
    these are microbenchmarks, but a number are nontrivial code.

    For example, M. Ed. Borasky's MatrixBenchmark:

    JRuby:
    Hilbert matrix of dimension 64 times its inverse = identity? true
    18.784000 0.000000 18.784000 ( 18.784000)

    Ruby 1.8.6:
    Hilbert matrix of dimension 64 times its inverse = identity? true
    32.560000 0.110000 32.670000 ( 32.776915)

    Or the YARV "pentomino" benchmark:

    ~/NetBeansProjects/jruby $ time jruby -J-server
    test/bench/yarv/bm_app_pentomino.rb

    real 1m31.770s
    user 1m32.016s
    sys 0m1.278s
    ~/NetBeansProjects/jruby $ time ruby test/bench/yarv/bm_app_pentomino.rb

    real 1m48.100s
    user 1m47.489s
    sys 0m0.287s

    Ola Bini of the JRuby team recently ran all the YARV benchmarks and
    reported the results here:

    http://ola-bini.blogspot.com/2007/10/mystery-expos-on-jruby-performance.html

    And I periodically post about performance progress on my blog here:

    http://headius.blogspot.com

    I expect we'll see Alioth updated soon after the 1.1 or 1.1 beta 1 releases.

    What about Rails?

    One of the biggest uses for JRuby recently has been Rails. Rails
    performance is a more complicated thing to measure; there's a lot of
    non-execution bottlenecks that get in the way. But both Ola and Nick
    Sieger (also of JRuby team) have found that JRuby on Rails performance
    is either very near or exceeding Ruby 1.8.x on Rails:

    http://ola-bini.blogspot.com/2007/10/current-state-of-regular-expressions.html
    (toward the bottom of the post are updated Rails numbers)

    http://blog.nicksieger.com/articles/2007/10/25/jruby-on-rails-fast-enough

    Rails is probably the ultimate real-world benchmark, and we're excited
    to see that JRuby's starting to pass Ruby 1.8.x performance here as well.

    I won't drag this on any longer, but I'd encourage you to try out your
    own Ruby code and let us know how it performs. We've spent a lot of time
    on performance issues for the 1.1 release, and have many future plans to
    continue increasing performance. We want your input.

    Many people believed we'd never be faster than the C implementation, and
    many still think we're slower. Now that I've set that record straight,
    any questions?

    - Charlie
     
    Charles Oliver Nutter, Oct 31, 2007
    #1
    1. Advertising

  2. Will JRuby support continuations (via callcc)?

    I've been reading about Actors in Scala recently and the technical
    documentation mentioned problems with trying to implement
    continuations on the JVM due to lack of stack manipulation.
     
    Paul Stickney, Oct 31, 2007
    #2
    1. Advertising

  3. Paul Stickney wrote:
    > Will JRuby support continuations (via callcc)?
    >
    > I've been reading about Actors in Scala recently and the technical
    > documentation mentioned problems with trying to implement
    > continuations on the JVM due to lack of stack manipulation.


    Neither JRuby nor XRuby nor IronRuby nor Ruby.NET will support
    continuations in the near future. However, there is research happening
    now to support continuations on the JVM...if that takes, we'll piggy
    back off that support.

    - Charlie
     
    Charles Oliver Nutter, Nov 1, 2007
    #3
  4. Charles Oliver Nutter

    Guest

    Quoting Charles Oliver Nutter <>:

    > Many people believed we'd never be faster than the C implementation,
    > and many still think we're slower. Now that I've set that record
    > straight, any questions?


    1. How long will it be before Alioth has some *reasonable* numbers for
    jRuby? As of yesterday, they still have you significantly slower than
    MRI. So I need to take jRuby out of my slides for RubyConf :) ... I

    2. I haven't actually been benchmarking jRuby recently ... too busy
    profiling MRI. :)

    3. I'm using the "Pet Store" benchmark for Rails. It's as close to a
    "real Rails benchmark" as anything I can find.

    4. How's your performance on a SPARC relative to MRI?
    >
    > - Charlie
     
    , Nov 1, 2007
    #4
  5. wrote:
    > Quoting Charles Oliver Nutter <>:
    >
    >> Many people believed we'd never be faster than the C implementation,
    >> and many still think we're slower. Now that I've set that record
    >> straight, any questions?

    >
    > 1. How long will it be before Alioth has some *reasonable* numbers for
    > jRuby? As of yesterday, they still have you significantly slower than
    > MRI. So I need to take jRuby out of my slides for RubyConf :) ... I


    The current published Alioth numbers are based on JRuby 1.0(ish), which
    was generally 2-3x slower than MRI. I'm hoping the numbers will be
    updated soon after the 1.1 releases...but it probably won't happen until
    1.1 final comes out in December. If someone else wants to re-run them
    for us, it would make us very happy :)

    > 2. I haven't actually been benchmarking jRuby recently ... too busy
    > profiling MRI. :)


    Well, we know there's still a lot we can do for JRuby
    performance...including not only the 1.9 benchmarks, but a number of
    additional ideas we have that won't affect 1.8 compatibility.

    > 3. I'm using the "Pet Store" benchmark for Rails. It's as close to a
    > "real Rails benchmark" as anything I can find.


    Yeah, that's what Ola's been using, and we're *real close* to matching
    MRI on that one right now. Still hunting for bottlenecks.

    > 4. How's your performance on a SPARC relative to MRI?


    Haven't given it a try...I don't have access to a good sparc box at the
    moment (and yes, I know...I work at Sun, so I can have a sparc by
    snapping my fingers...it's a matter of having time to monkey with it).

    - Charlie
     
    Charles Oliver Nutter, Nov 1, 2007
    #5
  6. On Oct 31, 11:24 pm, Charles Oliver Nutter <>
    wrote:

    > Neither JRuby nor XRuby nor IronRuby nor Ruby.NET will support


    "Neither ... nor ..." is for two alternatives. Perhaps this will
    suffice.

    None of JRuby or XRuby or IronRuby or Ruby.NET will support
     
    William James, Nov 1, 2007
    #6
  7. Charles Oliver Nutter

    Guest

    William James wrote:
    > On Oct 31, 11:24 pm, Charles Oliver Nutter <>
    > wrote:
    >
    >> Neither JRuby nor XRuby nor IronRuby nor Ruby.NET will support

    >
    > "Neither ... nor ..." is for two alternatives. Perhaps this will
    > suffice.
    >
    > None of JRuby or XRuby or IronRuby or Ruby.NET will support


    None of snow or rain or heat or gloom of night stays these couriers from
    the swift completion of their appointed rounds.

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    , Nov 1, 2007
    #7
  8. Charles Oliver Nutter

    Guest

    William James wrote:
    > On Oct 31, 11:24 pm, Charles Oliver Nutter <>
    > wrote:
    >
    >> Neither JRuby nor XRuby nor IronRuby nor Ruby.NET will support

    >
    > "Neither ... nor ..." is for two alternatives. Perhaps this will
    > suffice.
    >
    > None of JRuby or XRuby or IronRuby or Ruby.NET will support


    Perhaps

    JRuby, XRuby, IronRuby, Ruby.NET; none of them will support

    Too poetic?

    - Charlie
     
    , Nov 2, 2007
    #8
  9. Charles Oliver Nutter

    Guest

    Quoting Charles Oliver Nutter <>:

    > wrote:
    >> Quoting Charles Oliver Nutter <>:
    >>
    >>> Many people believed we'd never be faster than the C implementation,
    >>> and many still think we're slower. Now that I've set that record
    >>> straight, any questions?

    >>
    >> 1. How long will it be before Alioth has some *reasonable* numbers
    >> for jRuby? As of yesterday, they still have you significantly
    >> slower than MRI. So I need to take jRuby out of my slides for
    >> RubyConf :) ... I

    >
    > The current published Alioth numbers are based on JRuby 1.0(ish), which
    > was generally 2-3x slower than MRI. I'm hoping the numbers will be
    > updated soon after the 1.1 releases...but it probably won't happen
    > until 1.1 final comes out in December. If someone else wants to re-run
    > them for us, it would make us very happy :)


    I can't help you with that, but if you'll give me the numbers you do
    have (time in seconds for each benchmark on the latest jRuby, MRI and
    KRI on a common platform) I'll run the boxplots off and post them to
    the list.
     
    , Nov 2, 2007
    #9
  10. Charles Oliver Nutter

    Guest

    On Fri, Nov 02, 2007 at 09:35:44AM +0900, Charles Oliver Nutter wrote:
    > William James wrote:
    > >On Oct 31, 11:24 pm, Charles Oliver Nutter <>
    > >wrote:
    > >
    > >>Neither JRuby nor XRuby nor IronRuby nor Ruby.NET will support

    > >
    > >"Neither ... nor ..." is for two alternatives. Perhaps this will
    > >suffice.
    > >
    > >None of JRuby or XRuby or IronRuby or Ruby.NET will support

    >
    > Perhaps
    >
    > JRuby, XRuby, IronRuby, Ruby.NET; none of them will support
    >
    > Too poetic?


    The original polysyndeton conveyed the intended meaning effectively IMO.
    How thin is the line between a deliberate figure of speech and a grammatical
    accident!

    This doesn't sound too bad to me, but I'm no native speaker:
    "Neither JRuby nor XRuby, IronRuby or Ruby.NET will ..."

    "Or" must bind more tightly than "nor" there in order for the corresponding
    logical proposition to be correct:

    AND(NOT(JRuby), NOT(OR(XRuby, IronRuby, Ruby.NET)))

    "None of JRuby, XRuby, IronRuby or Ruby.NET will ..." and
    "As for JRuby, XRuby, IronRuby and Ruby.NET, none [of them] will ..."
    seem OK too.

    --
    Mauricio Fernandez - http://eigenclass.org
     
    , Nov 2, 2007
    #10
  11. Charles Oliver Nutter

    Guest

    Mauricio Fernandez wrote:
    > On Fri, Nov 02, 2007 at 09:35:44AM +0900, Charles Oliver Nutter wrote:
    >> William James wrote:
    >>> On Oct 31, 11:24 pm, Charles Oliver Nutter <>
    >>> wrote:
    >>>
    >>>> Neither JRuby nor XRuby nor IronRuby nor Ruby.NET will support
    >>> "Neither ... nor ..." is for two alternatives. Perhaps this will
    >>> suffice.
    >>>
    >>> None of JRuby or XRuby or IronRuby or Ruby.NET will support

    >> Perhaps
    >>
    >> JRuby, XRuby, IronRuby, Ruby.NET; none of them will support
    >>
    >> Too poetic?

    >
    > The original polysyndeton conveyed the intended meaning effectively IMO.
    > How thin is the line between a deliberate figure of speech and a grammatical
    > accident!


    I agree, and thanks for teaching me a new word.

    > This doesn't sound too bad to me, but I'm no native speaker:
    > "Neither JRuby nor XRuby, IronRuby or Ruby.NET will ..."


    That could sound like the "Neither ... nor ..." clause modifies
    IronRuby. As in:

    Neither fish nor fowl, greens or libertarians will be the first choice
    of voters who are repelled by two-party politics.

    It might not even be grammatical, but it is clear that we are not
    talking about electing ducks and sturgeons. I'd rewrite it anyway.

    > "Or" must bind more tightly than "nor" there in order for the corresponding
    > logical proposition to be correct:


    I don't think it works that way, at least as I hear it.

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    , Nov 2, 2007
    #11
  12. Charles Oliver Nutter

    Guest

    [OT] Re: JRuby performance questions answered

    On Sat, Nov 03, 2007 at 04:40:41AM +0900, Joel VanderWerf wrote:
    > Mauricio Fernandez wrote:
    > >This doesn't sound too bad to me, but I'm no native speaker:
    > >"Neither JRuby nor XRuby, IronRuby or Ruby.NET will ..."

    >
    > That could sound like the "Neither ... nor ..." clause modifies
    > IronRuby. As in:
    >
    > Neither fish nor fowl, greens or libertarians will be the first choice
    > of voters who are repelled by two-party politics.
    >
    > It might not even be grammatical, but it is clear that we are not
    > talking about electing ducks and sturgeons. I'd rewrite it anyway.


    Yet another effect is achieved by reordering the phrase this way:
    Neither fish nor greens, fowl or libertarians
    This being a play on the standard idiom [1], the two parts are connected,
    yielding a new meaning.

    Is any sufficiently complex figure of speech undistinguishable from nonsense? :)

    > >"Or" must bind more tightly than "nor" there in order for the corresponding
    > >logical proposition to be correct:

    >
    > I don't think it works that way, at least as I hear it.


    I had to repeat it a few times before I convinced myself that this is how
    things should work, but the interpretation as a juxtaposition is very
    attractive. Maybe I'm drawn to it because my mother tongue is closer to Latin?

    A skilled writer could use the ambiguity for very effective double-entendres.

    [1] I had to look it up. In Spanish, French and German the idiom is "neither
    meat nor fish" (actually, "neither fish nor meat" in German; I wonder why).

    --
    Mauricio Fernandez - http://eigenclass.org
     
    , Nov 3, 2007
    #12
  13. Charles Oliver Nutter

    Guest

    Re: [OT] Re: JRuby performance questions answered

    On 11/3/07, Mauricio Fernandez <> wrote:
    > I had to repeat it a few times before I convinced myself that this is how
    > things should work, but the interpretation as a juxtaposition is very
    > attractive. Maybe I'm drawn to it because my mother tongue is closer to Latin?
    >
    > A skilled writer could use the ambiguity for very effective double-entendres.
    >
    > [1] I had to look it up. In Spanish, French and German the idiom is "neither
    > meat nor fish" (actually, "neither fish nor meat" in German; I wonder why).


    The switching of nouns may have more to do with culture than language roots.

    I couldn't resist an attempt at lolcat...

    Ther be JRuby, XRuby, IronRuby, Ruby.NET, and no s'port continue stufs now.

    Latin rooted languages allow a lot of latitude, certainly, and I think
    if they lost that latitude, they would make a lot of poets bored, as
    you suggested.

    I could have used "but" instead of "and" in that last sentence and
    still convey the same meaning, but it might have changed the intent if
    your vernacular idioms differ from mine.

    Todd
     
    , Nov 3, 2007
    #13
  14. Charles Oliver Nutter

    Guest

    >
    > Long story short, JRuby 1.1 is now generally faster than Ruby 1.8.6, and
    > for most execution benchmarks it is the fastest 1.8-compatible Ruby
    > implementation available.


    Congrats on such an important benchmark! What made the difference 1.0
    to 1.1?
    -Roger
    --
    Posted via http://www.ruby-forum.com/.
     
    , Nov 4, 2007
    #14
  15. Charles Oliver Nutter

    Guest

    Roger Pack wrote:
    >> Long story short, JRuby 1.1 is now generally faster than Ruby 1.8.6, and
    >> for most execution benchmarks it is the fastest 1.8-compatible Ruby
    >> implementation available.

    >
    > Congrats on such an important benchmark! What made the difference 1.0
    > to 1.1?


    Mostly the compiler, but there's been a number of optimizations across
    the whole runtime and most of the core classes.

    - Charlie
     
    , Nov 5, 2007
    #15
  16. Charles Oliver Nutter

    gga Guest

    Charles Oliver Nutter wrote:
    >
    > Many people believed we'd never be faster than the C implementation, and
    > many still think we're slower.


    Isn't that still a correct impression and statement? Currently the C
    implementation to benchmark against is 1.9, which is soon to become
    the official release (early next year, if all goes to plan). 1.8 is
    mostly 3 years-old code now or so.
    Or is JRuby now faster than 1.9?

    > Now that I've set that record straight, any questions?
    >


    I do have one, yes. How much work does the JRuby team place in
    actually improving Sun's JVM?

    Recently there was a question of actually turning off ObjectSpace for
    JRuby as a default, instead of actually taking the JVM and enhancing
    it to support some sort of ObjectSpace, which seemed more like the
    right answer to me.
     
    gga, Nov 5, 2007
    #16
  17. Charles Oliver Nutter

    Guest

    gga wrote:
    > Charles Oliver Nutter wrote:
    >> Many people believed we'd never be faster than the C implementation, and
    >> many still think we're slower.

    >
    > Isn't that still a correct impression and statement? Currently the C
    > implementation to benchmark against is 1.9, which is soon to become
    > the official release (early next year, if all goes to plan). 1.8 is
    > mostly 3 years-old code now or so.
    > Or is JRuby now faster than 1.9?


    Ruby 1.9 is not likely to be the standard, widely-used version of Ruby
    for some time. Matz estimated it probably wouldn't be the case until
    late 2008. He also mentioned at the conference that while 1.9.1 might be
    feature-stable, it's still a bit experimental and will probably not be a
    production-ready VM for a while.

    In my opinion, no, it's not really a direct comparison to put JRuby up
    against Ruby 1.9 because 1.9 contains several compatibility-breaking
    changes. Many of those changes enable faster performance at the cost of
    backward compatibility. JRuby is an implementation of Ruby 1.8, and
    under that definition, it's appropriate to compare it to other
    implementations of Ruby 1.8. Some example of compatibility-breaking
    changes in Ruby 1.9:

    - case/when with literals will be constant time, but not invoke ===
    - Fixnum math will be performed as direct operations, rather than
    dispatch through Fixnum#+, Fixnum#-, and so on.
    - block arguments can only be local variables

    And so on. We plan to make all the same optimizations, but also make
    them individually configurable, so that if you want to have fast integer
    math but everything else 1.8 semantics, you'll be able to. Or if you
    want to turn on 1.9 features as a whole, you can.

    Since you ask, yes, there's a few benchmarks where we're already faster
    than 1.9:

    JRuby:
    1m loops yielding three fixnums 10 times to block splatting and
    accessing them
    3.225000 0.000000 3.225000 ( 3.225000)
    3.189000 0.000000 3.189000 ( 3.189000)
    3.170000 0.000000 3.170000 ( 3.170000)
    3.152000 0.000000 3.152000 ( 3.151000)
    3.322000 0.000000 3.322000 ( 3.322000)

    1.9:
    1m loops yielding three fixnums 10 times to block splatting and
    accessing them
    4.940000 0.010000 4.950000 ( 4.973445)
    4.950000 0.020000 4.970000 ( 4.971358)
    4.950000 0.010000 4.960000 ( 4.984771)
    4.950000 0.010000 4.960000 ( 4.980683)
    4.950000 0.010000 4.960000 ( 4.973008)

    ~/NetBeansProjects/jruby $ jruby -J-server
    test/bench/bench_fib_iterative.rb
    11.610000 0.000000 11.610000 ( 11.611000)
    11.481000 0.000000 11.481000 ( 11.481000)
    ~/NetBeansProjects/jruby $ ../ruby1.9/ruby -I ../ruby1.9/lib
    test/bench/bench_fib_iterative.rb
    13.110000 4.700000 17.810000 ( 17.928527)
    12.930000 4.770000 17.700000 ( 17.818631)

    1.9's eval performance is also considerably degraded due to the cost of
    compilation...in some cases even slower than Ruby 1.8.

    >> Now that I've set that record straight, any questions?
    >>

    >
    > I do have one, yes. How much work does the JRuby team place in
    > actually improving Sun's JVM?
    >
    > Recently there was a question of actually turning off ObjectSpace for
    > JRuby as a default, instead of actually taking the JVM and enhancing
    > it to support some sort of ObjectSpace, which seemed more like the
    > right answer to me.


    The problems with ObjectSpace are not just problems you can flip a
    switch and make go away; they're systemic issues that will affect any
    modern GC. Look at it this way:

    In Ruby, where Objects do not move around in memory, it's easy to just
    walk through the entire set of objects in memory and return them in
    turn. There's no parallel threads to create more garbage, and there's no
    concurrent garbage collector moving objects around and compacting the
    heap.

    On the JVM, however, it's an entirely different situation. Not only are
    threads actually running in parallel, creating their own garbage...the
    GC itself runs in parallel on many versions of Java; so you'd not only
    have to stop all other threads from running, you'd have to stop the
    garbage collector itself.

    The other answer here is that...you already can do these things; they're
    used for profiling and debugging modes of the JVM. And oddly enough, the
    most common applications of ObjectSpace.each_object are for runtime
    profiling and debugging. The difference in the JVM (and in JRuby 1.1) is
    that we're not going to saddle users with the performance hit of what's
    typically a profiling/debugging feature that they'll never use on a
    production server.

    (and of course we're very keen on improving the JVM...but there's only
    so many hours in a day)

    - Charlie
     
    , Nov 5, 2007
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. kayodeok
    Replies:
    1
    Views:
    358
  2. SithVegeta

    Just need a few questions answered...

    SithVegeta, Nov 15, 2004, in forum: C Programming
    Replies:
    6
    Views:
    380
    SithVegeta
    Nov 15, 2004
  3. Replies:
    10
    Views:
    556
  4. Syam
    Replies:
    25
    Views:
    849
    Kai-Uwe Bux
    Dec 2, 2006
  5. Charles Oliver Nutter

    [JRuby] JRuby perf questions answered

    Charles Oliver Nutter, Oct 31, 2007, in forum: Ruby
    Replies:
    7
    Views:
    183
    Kevin Williams
    Nov 1, 2007
Loading...

Share This Page