Ruby vs JRuby Performance

[Victor Reyes]

[Note: parts of this message were removed to make it a legal post.]

I knew that there was a penalty to be paid when running JRuby, but I did not
know how high.
I developed a Sudoku solver, which actually solves simple to medium
difficult puzzles. I am still struggling to make it solve "hard" problems.
I just installed JRuby yesterday and wanted to compare it against Ruby.

I executed the same program:

The results are in:

ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

BTW, there is no GUI in my program. I hope to learn the GUI part with JRuby.
That's the main reason why I downloaded JRuby, NetBeans, Java JDK/JRE, Etc.
I guess there is trade-off which one must be willing to accept.

Am I in the ball-park?

Regards,

Victor

 

[Jochen Theodorou]

Victor Reyes schrieb:
[...]

ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

jruby has certainly higher startup costs than native Ruby, but that does
not mean its overall performance is bad. I suggest you modify the
program so it runs for example 10 times and then run again. I am sure
the JRuby program will be very much below 10s

bye Jochen

 

[ThoML]

jruby -v

ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

Try to run it with jruby -J-server ...

 

[M. Edward (Ed) Borasky]

I knew that there was a penalty to be paid when running JRuby, but I did not
know how high.
I developed a Sudoku solver, which actually solves simple to medium
difficult puzzles. I am still struggling to make it solve "hard" problems.
I just installed JRuby yesterday and wanted to compare it against Ruby.

I executed the same program:

The results are in:

ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

BTW, there is no GUI in my program. I hope to learn the GUI part with JRuby.
That's the main reason why I downloaded JRuby, NetBeans, Java JDK/JRE, Etc.
I guess there is trade-off which one must be willing to accept.

Am I in the ball-park?

Regards,

Victor

Send the source to Charlie Nutter ... he is always looking for jRuby
performance test cases.

 

[Lloyd Linklater]

I knew that there was a penalty to be paid when running JRuby, but I did
not
know how high.
I developed a Sudoku solver, which actually solves simple to medium
difficult puzzles. I am still struggling to make it solve "hard"
problems.
I just installed JRuby yesterday and wanted to compare it against Ruby.

I executed the same program:

The results are in:
*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

Am I in the ball-park?

According to the metrics posted here, JRuby runs about twice as fast,
takes longer to load and uses a lot more memory. Here is a little
something on the metrics:

http://www.ruby-forum.com/topic/129990#new
http://www.ruby-forum.com/topic/129996#new

 

[Victor Reyes]

[Note: parts of this message were removed to make it a legal post.]

*Ruby vs JRuby Performance <http://www.ruby-forum.com/topic/155783#686221>*
Posted by Victor Reyes (Guest)
on 09.06.2008 15:41
[image: (Received via mailing list)]

I knew that there was a penalty to be paid when running JRuby, but I did
not
know how high.
I developed a Sudoku solver, which actually solves simple to medium
difficult puzzles. I am still struggling to make it solve "hard"

problems.
I just installed JRuby yesterday and wanted to compare it against Ruby.

I executed the same program:

The results are in:

ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

BTW, there is no GUI in my program. I hope to learn the GUI part with

JRuby.
That's the main reason why I downloaded JRuby, NetBeans, Java JDK/JRE,
Etc.
I guess there is trade-off which one must be willing to accept.

Am I in the ball-park?

Regards,

Victor

Reply with quote<http://www.ruby-forum.com/topic/155783?reply_to=686221#postform>
*Re: Ruby vs JRuby Performance<http://www.ruby-forum.com/topic/155783#686235>
*
Posted by Jochen Theodorou (Guest)
on 09.06.2008 16:15
[image: (Received via mailing list)]

Victor Reyes schrieb:
[...]

jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

jruby has certainly higher startup costs than native Ruby, but that does

not mean its overall performance is bad. I suggest you modify the
program so it runs for example 10 times and then run again. I am sure
the JRuby program will be very much below 10s

bye Jochen

Reply with quote<http://www.ruby-forum.com/topic/155783?reply_to=686235#postform>
*Re: Ruby vs JRuby Performance<http://www.ruby-forum.com/topic/155783#686241>
*
Posted by M. Edward (Ed) Borasky (Guest)

ruby -v
Elapsed Time: 1.058 Secs*

Send the source to Charlie Nutter ... he is always looking for jRuby
performance test cases.

Reply with quote<http://www.ruby-forum.com/topic/155783?reply_to=686241#postform>
*Re: Ruby vs JRuby Performance<http://www.ruby-forum.com/topic/155783#686245>
*
Posted by ThoML (Guest)
on 09.06.2008 16:31

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

Try to run it with jruby -J-server ...

Reply with quote<http://www.ruby-forum.com/topic/155783?reply_to=686245#postform>
*Re: Ruby vs JRuby Performance<http://www.ruby-forum.com/topic/155783#686249>
*
Posted by Lloyd Linklater (lloyd <http://www.ruby-forum.com/user/show/6864>)

I knew that there was a penalty to be paid when running JRuby, but I did
not
know how high.
I developed a Sudoku solver, which actually solves simple to medium
difficult puzzles. I am still struggling to make it solve "hard"
problems.
I just installed JRuby yesterday and wanted to compare it against Ruby.

I executed the same program:

The results are in:
*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

Am I in the ball-park?

According to the metrics posted here, JRuby runs about twice as fast,
takes longer to load and uses a lot more memory. Here is a little
something on the metrics:

http://www.ruby-forum.com/topic/129990#new
http://www.ruby-forum.com/topic/129996#new


======================================================================================

Team,

First, thank you for all your comments.

As suggested by JoChen, I executed the program under JRuby 14 times.

To be fair to Ruby, I also executed it 14 times under Ruby.
You can see the results below.

As suggested by Ed Borasky, I will send the code to Charlie Nutter, if
I can get his email. Otherwise I can post it here. It is not great

code at all, but it is doing the work for now.

I executed the program using *jruby -J-server* as suggested by ThoML
and results are catastrophic! See the number belows! I only ran it 6
times.
I also can't believe that jruby has been clocked at twice the speed of
ruby since after all, jruby has more overhead than ruby. Then again,

I am not expert on the subject (or any subject).

*JRuby executions:
*Elapsed Time: 0.888
Elapsed Time: 0.881
Elapsed Time: 0.977
Elapsed Time: 0.888
Elapsed Time: 0.872

Elapsed Time: 0.884
Elapsed Time: 0.869
Elapsed Time: 0.892
Elapsed Time: 0.876
Elapsed Time: 0.872
Elapsed Time: 0.882
Elapsed Time: 0.882
Elapsed Time: 0.867
Elapsed Time: 0.881
Elapsed Time: 0.906

Elapsed Time: 0.881

*Ruby executions:*
Elapsed Time: 0.422
Elapsed Time: 0.391
Elapsed Time: 0.407
Elapsed Time: 0.406
Elapsed Time: 0.407

Elapsed Time: 0.422
Elapsed Time: 0.406
Elapsed Time: 0.438
Elapsed Time: 0.406
Elapsed Time: 0.406
Elapsed Time: 0.406
Elapsed Time: 0.39
Elapsed Time: 0.406
Elapsed Time: 0.406
Elapsed Time: 0.422

Elapsed Time: 0.39

*jruby -J-server sudoku01final.rb*
Elapsed Time: 2.474
Elapsed Time: 2.467
Elapsed Time: 2.347
Elapsed Time: 2.404
Elapsed Time: 2.326
Elapsed Time: 2.414

Thank you

Victor

 

[ThoML]

I executed the program using *jruby -J-server* as suggested by ThoML

and results are catastrophic! See the number belows! I only ran it 6
times.

I didn't realize how small the numbers already are. Have you checked
the result of

time ruby -v
time jruby -v

just to get a feeling for how much time is spent in startup alone?

 

[Jochen Theodorou]

Victor Reyes schrieb:
[...]

As suggested by JoChen, I executed the program under JRuby 14 times.

To be fair to Ruby, I also executed it 14 times under Ruby.
You can see the results below.

As suggested by Ed Borasky, I will send the code to Charlie Nutter, if
I can get his email. Otherwise I can post it here. It is not great

code at all, but it is doing the work for now.

I executed the program using *jruby -J-server* as suggested by ThoML
and results are catastrophic! See the number belows! I only ran it 6
times.
I also can't believe that jruby has been clocked at twice the speed of
ruby since after all, jruby has more overhead than ruby. Then again,

I am not expert on the subject (or any subject).

*JRuby executions:
*Elapsed Time: 0.888
Elapsed Time: 0.881 [...]
Elapsed Time: 0.881

*Ruby executions:*
Elapsed Time: 0.422
Elapsed Time: 0.391 [...]
Elapsed Time: 0.39

*jruby -J-server sudoku01final.rb*
Elapsed Time: 2.474 [...]
Elapsed Time: 2.414

well, looking at these numbers I would suggest you subscribe to the
jruby mailing list and ask why jruby is 100% slower than ruby The
server mode really looks awful... They need to know about this!

bye Jochen

 

[Charles Oliver Nutter]

well, looking at these numbers I would suggest you subscribe to the
jruby mailing list and ask why jruby is 100% slower than ruby The
server mode really looks awful... They need to know about this!

You are here trolling on the Ruby lists Jochen? Don't you have some
Groovy bugs to be fixing?

The server VM starts up a lot slower on most systems, which is the
reason for this test to be invalid. And in general, any benchmark under
5-10s is going to be a pretty poor test of JRuby performance, since the
code might not even get compiled or optimized on such a short run.

- Charlie

 

[Charles Oliver Nutter]

ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [i386-mswin32]

*Ruby:
ruby sudoku01final.rb
Elapsed Time: 0.437 Secs*

jruby -v
ruby 1.8.6 (2008-05-28 rev 6586) [x86-jruby1.1.2]
*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

Far too short.

~/amsterdam âž” time ruby -e "puts 1"
1

real 0m0.006s
user 0m0.003s
sys 0m0.004s

~/amsterdam âž” time jruby -e "puts 1"
1

real 0m0.594s
user 0m0.460s
sys 0m0.101s

Is JRuby 100 times slower than Ruby? No. It is too short a test. Run
something longer and you'll get more realistic results.

- Charlie

 

[Jochen Theodorou]

You are here trolling on the Ruby lists Jochen? Don't you have some
Groovy bugs to be fixing?

ah, I didn't try to troll, I put so many in.

The server VM starts up a lot slower on most systems, which is the
reason for this test to be invalid. And in general, any benchmark under
5-10s is going to be a pretty poor test of JRuby performance, since the
code might not even get compiled or optimized on such a short run.

well, ok, but it was:

*jruby -J-server sudoku01final.rb*
Elapsed Time: 2.474
Elapsed Time: 2.467
Elapsed Time: 2.347
Elapsed Time: 2.404
Elapsed Time: 2.326
Elapsed Time: 2.414

that's >12s... unless he did start over again and again, but I assumed
that is not the case because of:

*JRuby executions:* Elapsed Time: 0.888 [...]

vs.

*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

but maybe I was wrong.

bye Jochen

 

[Victor Reyes]

[Note: parts of this message were removed to make it a legal post.]

You are here trolling on the Ruby lists Jochen? Don't you have some Groovy
bugs to be fixing?

ah, I didn't try to troll, I put so many in.

The server VM starts up a lot slower on most systems, which is the reason

for this test to be invalid. And in general, any benchmark under 5-10s is
going to be a pretty poor test of JRuby performance, since the code might
not even get compiled or optimized on such a short run.

well, ok, but it was:

*jruby -J-server sudoku01final.rb*

Elapsed Time: 2.474
Elapsed Time: 2.467
Elapsed Time: 2.347
Elapsed Time: 2.404
Elapsed Time: 2.326
Elapsed Time: 2.414

that's >12s... unless he did start over again and again, but I assumed
that is not the case because of:

*JRuby executions:* Elapsed Time: 0.888 [...]

vs.

*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

but maybe I was wrong.

bye Jochen

Team,

I'll be more than happy to send the code to someone or just put it here as
an attachment.
The way I executed the code was by opening a DOS (cmd) session on XP.
I typed the command as described above: *jruby sudoku01final.rb* multiple
times. Then I did the same for ruby: *ruby sudoku01final.rb.*
Although the code is not great code, that does not really matter since the
same "bad" or "good" code was executed unmodified under both environments.
Let me know if anyone wants to see and run the code for testing!

Thank you

Victor

 

[Leslie Viljoen]

Jochen Theodorou wrote:

well, looking at these numbers I would suggest you subscribe to the jruby
mailing list and ask why jruby is 100% slower than ruby The server mode
really looks awful... They need to know about this!


You are here trolling on the Ruby lists Jochen? Don't you have some Groovy
bugs to be fixing?

ah, I didn't try to troll, I put so many in.

The server VM starts up a lot slower on most systems, which is the reason

for this test to be invalid. And in general, any benchmark under 5-10s is
going to be a pretty poor test of JRuby performance, since the code might
not even get compiled or optimized on such a short run.

well, ok, but it was:

*jruby -J-server sudoku01final.rb*

Elapsed Time: 2.474
Elapsed Time: 2.467
Elapsed Time: 2.347
Elapsed Time: 2.404
Elapsed Time: 2.326
Elapsed Time: 2.414

that's >12s... unless he did start over again and again, but I assumed
that is not the case because of:

*JRuby executions:* Elapsed Time: 0.888 [...]

vs.

*JRuby:
jruby sudoku01final.rb
Elapsed Time: 1.058 Secs*

but maybe I was wrong.

bye Jochen

Team,

I'll be more than happy to send the code to someone or just put it here as
an attachment.
The way I executed the code was by opening a DOS (cmd) session on XP.
I typed the command as described above: *jruby sudoku01final.rb* multiple
times. Then I did the same for ruby: *ruby sudoku01final.rb.*
Although the code is not great code, that does not really matter since the
same "bad" or "good" code was executed unmodified under both environments.
Let me know if anyone wants to see and run the code for testing!

Just put a loop in your Ruby program:

12.times do

...main code

end



by restarting the JVM each time, you are not seeing any of JRuby's benefits.

Les

 

[Mike McKinney]

[Note: parts of this message were removed to make it a legal post.]

The JVM is built for long running processes, optimizing things over time
(jruby takes advantage of this), MRI is going to remain the same speed over
time. (for the most part, see below)

here is a quick example... first the somewhat silly example code:
#######################################
require 'benchmark'
num = ARGV[0] || 10

puts "Running #{num} times"

puts Benchmark.measure {
num.to_i.times do
234 + 89239423 + 23482903 + 34892389423 + 234823
end
}.total
#######################################
now run it 10 times each: ( ruby/jruby speed.rb 10)
ruby: 0.0
jruby: 0.0519

10,000 times:
ruby:0.03
jruby:0.105

100,000 times:
ruby: 0.26
jruby: 0.217

1,000,000 times:
ruby: 2.61
jruby: 1.303

now with some additional optimization for the jvm: jruby -J-server speed.rb
1000000
jruby: 0.845

This is obviously a silly little test, but can possibly shed light on the
start-up cost diffs between JRuby and MRI... and where the benefit lies. So
who is faster? MRI? (yes) JRuby? (yes) just depends on what problem you
are trying to solve.

 

[Charles Oliver Nutter]

BTW, there is no GUI in my program. I hope to learn the GUI part with JRuby.
That's the main reason why I downloaded JRuby, NetBeans, Java JDK/JRE, Etc.
I guess there is trade-off which one must be willing to accept.

For what it's worth, 1.1.3 should include an update to the interpreter
that will help short-running apps a bit. But even then runs shorter than
5-10s aren't going to really hit their stride as far as JRuby goes.

- Charlie

 

[Charles Oliver Nutter]

I knew that there was a penalty to be paid when running JRuby, but I did not
know how high.
I developed a Sudoku solver, which actually solves simple to medium
difficult puzzles. I am still struggling to make it solve "hard" problems.
I just installed JRuby yesterday and wanted to compare it against Ruby.

Ugh, and I keep forgetting...yes, we'd love to see the code.

- Charlie

 

[Jochen Theodorou]

Victor Reyes schrieb:
[...]

I'll be more than happy to send the code to someone or just put it here as
an attachment.
The way I executed the code was by opening a DOS (cmd) session on XP.
I typed the command as described above: *jruby sudoku01final.rb* multiple
times.

ok, that was not what I intended. I did mean, what others also said, a
loop inside the program.

bye Jochen

 

[Charles Oliver Nutter]

I'll be more than happy to send the code to someone or just put it here as
an attachment.
The way I executed the code was by opening a DOS (cmd) session on XP.
I typed the command as described above: *jruby sudoku01final.rb* multiple
times. Then I did the same for ruby: *ruby sudoku01final.rb.*
Although the code is not great code, that does not really matter since the
same "bad" or "good" code was executed unmodified under both environments.
Let me know if anyone wants to see and run the code for testing!

I'd love to give it a try. If it does turn out to be slower, it will be
fun to investigate and figure out why.

- Charlie

 

[Victor Reyes]

[Note: parts of this message were removed to make it a legal post.]

I'll be more than happy to send the code to someone or just put it here as
an attachment.
The way I executed the code was by opening a DOS (cmd) session on XP.
I typed the command as described above: *jruby sudoku01final.rb* multiple
times. Then I did the same for ruby: *ruby sudoku01final.rb.*
Although the code is not great code, that does not really matter since the
same "bad" or "good" code was executed unmodified under both environments.
Let me know if anyone wants to see and run the code for testing!

I'd love to give it a try. If it does turn out to be slower, it will be fun
to investigate and figure out why.

- Charlie
[/QUOTE]
Charlie,

The code is attached to this thread.
However, I will send it to your email.
Also, remember that this is code written by a Rubist wannabe and not
necessary by one who knows the ins and outs of the language.
I'll be interested in knowing what you find!

Thank you

Victor