How fast does your Ruby run?

[ara.t.howard]

Raising an error might be a better way to ensure people don't run
code without ObjectSpace enabled, certainly. But maybe they don't
care?

they care.... why not just use autoload charles?

a @ http://codeforpeople.com/

 

[Charles Oliver Nutter]

they care.... why not just use autoload charles?

Because ObjectSpace needs to be loaded/enabled *before* the objects you
want to track come into existance. Unlike Ruby's simple GC and memory
model, in JRuby we have to explicitly add objects created to a walkable
list. There's no capability on most modern GCed VMs to walk the heap at
will. Because of that, any objects created before ObjectSpace is enabled
would not be visible during any subsequent walk. Generally, autoload
would be far too late.

- Charlie

 

[ara.t.howard]

Because ObjectSpace needs to be loaded/enabled *before* the objects
you want to track come into existance. Unlike Ruby's simple GC and
memory model, in JRuby we have to explicitly add objects created to
a walkable list. There's no capability on most modern GCed VMs to
walk the heap at will. Because of that, any objects created before
ObjectSpace is enabled would not be visible during any subsequent
walk. Generally, autoload would be far too late.

hmmm. an exception seems best then - force the require up front.
or, is there perhaps a lighter weight method to mark things so you
*could* bootstrap ObjectSpace once it was loaded?

cheers.

a @ http://codeforpeople.com/

 

[Charles Oliver Nutter]

hmmm. an exception seems best then - force the require up front. or,
is there perhaps a lighter weight method to mark things so you *could*
bootstrap ObjectSpace once it was loaded?

Not really; we've made it as lightweight as possible. The problem is
that there's no capability in the JVM to say "let me walk all objects",
largely because of how that would impact GC operation and security
(should objects created in other classloaders be walkable? internal JVM
objects? should you be able to reach objects you'd *never* be able to
construct or reference yourself? so many questions).

So the only way to ever be able to reach objects in the future is to
have a reference to them somewhere. So we create a very lightweight
linked list of references to all objects created. However it gets more
complicated: ObjectSpace references shouldn't keep objects from getting
collected. So every reference is a Java weak reference, and we
periodically clean the list of dead references.

Tracking every object created + weak references + periodic cleanup = slow.

So ObjectSpace either needs to be there when the object is created, or
you must accept that you can't ever walk it.

- Charlie

 

[Joel VanderWerf]

In general, the tricky bit is that there's at least one key library in
stdlib that depends on ObjectSpace: test/unit, which uses it to locate
tests. However I've implemented in JRuby the ability to walk all child
classes from a parent class, which provides the each_object(Class) used
in test/unit.

The dependence on ObjectSpace is kind of nasty, isn't it? A library
shouldn't use ObjectSpace in this way, because what it finds there may
depend on whether GC has run recently or not.

Good to hear you found a way to support it anyway, though.

Does your mechanism to walk child classes prevent them from being
collected, if there are no other references to them? If so, then it
isn't quite bug-compatible (though it is unlikely to matter):

require 'test/unit'

def add_a_test_case str
puts "adding test case #{str}"
Class.new(Test::Unit::TestCase).class_eval do
define_method :test_foo do
puts "running test case #{str}"
end
end
end

at_exit { GC.start }
at_exit { add_a_test_case 1 }
at_exit { add_a_test_case 2 }
at_exit { add_a_test_case 3 }
at_exit { add_a_test_case 4 }

__END__

adding test case 4
adding test case 3
adding test case 2
adding test case 1
Loaded suite -
Started
running test case 1

 

[Joel VanderWerf]

Does your mechanism to walk child classes prevent them from being
collected, if there are no other references to them? If so, then it
isn't quite bug-compatible (though it is unlikely to matter):

Oh, just saw your other post: you use weak refs, so the behavior should
be like ObjectSpace.

 

[Charles Oliver Nutter]

The dependence on ObjectSpace is kind of nasty, isn't it? A library
shouldn't use ObjectSpace in this way, because what it finds there may
depend on whether GC has run recently or not.

Yes, ObjectSpace is a useful tool for walking the heap when you really
want to, but it's not deterministic about whether or when it will find
the object(s) you want. So I agree it's a bad feature to use as part of
libraries you want to work the same way every time.

In the case of test/unit, most TestCase descendants get hard-referenced
as classes assigned to constants somewhere. However if you constructed a
test case as follows:

Class.new(Test::Unit::TestCase) { def test_something; end }

...it may or may not run with the default loader.

Good to hear you found a way to support it anyway, though.

Does your mechanism to walk child classes prevent them from being
collected, if there are no other references to them? If so, then it
isn't quite bug-compatible (though it is unlikely to matter):

No, the references to child classes are all weak references; if they go
away, the references will go away. It's a much smaller price to pay than
attaching weak references to *all* constructed objects (to my knowledge,
it costs almost nothing, and test/unit still works.

The typical way I think people would fix test/unit's use of ObjectSpace
would be to add a Class#inherited hook that records all subclasses of
TestCase. But nobody's gone ahead with that change yet.

- Charlie

 

[ara.t.howard]

Not really; we've made it as lightweight as possible. The problem
is that there's no capability in the JVM to say "let me walk all
objects", largely because of how that would impact GC operation and
security (should objects created in other classloaders be walkable?
internal JVM objects? should you be able to reach objects you'd
*never* be able to construct or reference yourself? so many
questions).

better you than me ;-)

So the only way to ever be able to reach objects in the future is
to have a reference to them somewhere. So we create a very
lightweight linked list of references to all objects created.
However it gets more complicated: ObjectSpace references shouldn't
keep objects from getting collected. So every reference is a Java
weak reference, and we periodically clean the list of dead references.

Tracking every object created + weak references + periodic cleanup
= slow.

So ObjectSpace either needs to be there when the object is created,
or you must accept that you can't ever walk it.

hmmm. ok i'm brainstorming here which you can ignore if you like as
i know less that nothing about jvms or implementing ruby but here
goes: what if you could invert the problem? what i objects knew
about the global ObjectSpaceThang and could be forced to register
themselves on demand somehow? without a reference i've no idea how,
just throwing that out there. or, another stupid idea, what if the
objects themselves were the tree/graph of weak references parent ->
children. crawling it would be, um, fun - but you could prune dead
objects *only* when walking the graph. this should be possible in
ruby since you always have the notion of a parent object - which is
Object - so all objects should be either reachable or leaks. now
back to drinking my regularly scheduled beer...

cheers.

a @ http://codeforpeople.com/

 

[Daniel Schömer]

[...]
1.8.6 5000 i686-linux 13.607635 73488
[...]

What is "patch 5000"??

That's what ruby from the svn checkout
http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8 thinks is
it's patchlevel.

| # cat ruby_1_8/version.h
| #define RUBY_VERSION "1.8.6"
| #define RUBY_RELEASE_DATE "2007-10-26"
| #define RUBY_VERSION_CODE 186
| #define RUBY_RELEASE_CODE 20071026
| #define RUBY_PATCHLEVEL 5000
|
| #define RUBY_VERSION_MAJOR 1
| #define RUBY_VERSION_MINOR 8
| #define RUBY_VERSION_TEENY 6
| #define RUBY_RELEASE_YEAR 2007
| #define RUBY_RELEASE_MONTH 10
| #define RUBY_RELEASE_DAY 26
|
| #ifdef RUBY_EXTERN
| RUBY_EXTERN const char ruby_version[];
| RUBY_EXTERN const char ruby_release_date[];
| RUBY_EXTERN const char ruby_platform[];
| RUBY_EXTERN const int ruby_patchlevel;
| #endif

Daniel

 

[Daniel Schömer]

Gentoo Linux on Intel Pentium 4 2.40GHz (512 KB cache):

Ruby Patch Platform Seconds Iterations per second
===== ===== =================== ========= =====================
1.8.6 111 i686-linux (gentoo) 32.801483 30486
1.8.6 5000 i686-linux 13.607635 73488
1.9.0 0 i686-linux 3.988949 250692
1.8.5 3876 java 63.589 15725
1.8.5 3876 java (-J-server) 44.173 22638
1.8.5 3876 java (-O) 36.917 27087
1.8.5 3876 java (-O -J-server) 15.194 65815

The first is the system installed ruby.

IIRC on my Gentoo I got +80% on this particular benchmark with
USE="-threads". Given this huge difference I looked at the speedup for
the test suite of one of my Rails applications and got only ~ +10%.
[...]

Now I recompiled ruby without threads:

Ruby Patch Platform Seconds Iterations/s
===== ===== ============================ ========= ============
1.8.6 111 i686-linux (gentoo) 32.801483 30486
1.8.6 111 i686-linux (gentoo -threads) 15.776961 63383

Wow, that's better. Since I only have a few packages compiled
with threads, I try to recompile them without.

Daniel

 

[James Edward Gray II]

Well, you need to explicitly reference ObjectSpace in most cases to
access it, right?

In general, the tricky bit is that there's at least one key library
in stdlib that depends on ObjectSpace: test/unit, which uses it to
locate tests.

Couldn't test/unit define an inherited() hook and locate test cases
that way? I haven't looked at the code, but it seems like that could
work.

James Edward Gray II

 

[Ola Bini]

Couldn't test/unit define an inherited() hook and locate test cases
that way? I haven't looked at the code, but it seems like that could
work.

James Edward Gray II

Yes, it's very easy to do so. In fact, it's so easy that I implemented
it a few months back:
http://ola-bini.blogspot.com/2007/07/objectspace-to-have-or-not-to-have.html

I didn't integrate it with test/unit at that point, but it should be
easy-easy to do.

Cheers

--
Ola Bini (http://ola-bini.blogspot.com)
JRuby Core Developer
Developer, ThoughtWorks Studios (http://studios.thoughtworks.com)
Practical JRuby on Rails (http://apress.com/book/view/9781590598818)

"Yields falsehood when quined" yields falsehood when quined.

 

[Charles Oliver Nutter]

Yes, it's very easy to do so. In fact, it's so easy that I implemented
it a few months back:
http://ola-bini.blogspot.com/2007/07/objectspace-to-have-or-not-to-have.html


I didn't integrate it with test/unit at that point, but it should be
easy-easy to do.

I think Ryan Davis (current test/unit maintainer) would probably like to
have that patch. Send it his way.

- Charlie

 

[Daniel Berger]

How fast does your Ruby run?

How fast does my Ruby run?! You've never heard of The Millenium Ruby?
It's the version that made the GNU/Kessel run in less than 12
parsecs!

It's run faster than Imperial programming languages. Not the local
bulk Java installs, mind you. I'm talking the big MS C# installs.

She's fast enough for you, old man.

Regards,

Dan

 

[Robert McGovern]

Hand built 1.8.6 on 2ghz MacBook, 1gig ram:

Followup, with the newly released 1.8.6p100

robert-mcgoverns-computer:~/Documents/Programming robertmcgovern$ ruby test.rb
55

Ruby 1.8.6 patch 110 on i686-darwin8.10.1
It took 9.955669 seconds to run. 100445 iterations persecond.

Slightly slower, running on battery but I don't think it makes any difference.

A followup on myself now that Leopard is out and installed on my MacBook.

On AC, with Normal performance selected the results are:

Ruby 1.8.6 patch 36 on universal-darwin9.0
It took 12.038167 seconds to run. 83069 iterations persecond.

On AC, with Better Performance selected:

Ruby 1.8.6 patch 36 on universal-darwin9.0
It took 12.112343 seconds to run. 82560 iterations persecond.

On Battery, Better Battery Life:

Ruby 1.8.6 patch 36 on universal-darwin9.0
It took 12.095676 seconds to run. 82674 iterations persecond.

On Battery, Better Performance:

Ruby 1.8.6 patch 36 on universal-darwin9.0
It took 12.10441 seconds to run. 82614 iterations persecond.

So roughly 3 seconds slower overall for Leopards own 1.8.6 vs hand
built 1.8.6 on Tiger. Need to build my own 1.8.6 on Leopard to see
what that brings. Also no real difference between various settings
(wasn't sure if any noticeable CPU throttling would occur).

 

[ara.t.howard]

How fast does my Ruby run?! You've never heard of The Millenium Ruby?
It's the version that made the GNU/Kessel run in less than 12
parsecs!

It's run faster than Imperial programming languages. Not the local
bulk Java installs, mind you. I'm talking the big MS C# installs.

She's fast enough for you, old man.

touche=B4

a @ http://codeforpeople.com/

 

[ghalsey]

windows xp. intel core2 1.86GHz

JRUBY:

C:\jruby\jruby-trunk\bin>jruby calculate.rb
55

It took 41.125 seconds to run. 24316 iterations per
second.

Not so good. Let's disable ObjectSpace:
C:\jruby\jruby-trunk\bin>jruby -O calculate.rb
55

It took 11.109 seconds to run. 90017 iterations per
second.

That was good. Now let's give it some more juice:
C:\jruby\jruby-trunk\bin>jruby -J-server -O calculate.rb
55

It took 5.75 seconds to run. 173913 iterations per
second.
--------------------------------------------
MRI:One-Click
ruby calculate.rb
55

Ruby 1.8.6 patch 0 on i386-mswin32
It took 15.828 seconds to run. 63179 iterations per
second.

#testing 1.9, yep it's fast;

C:\cygwin\ruby1.9\bin>ruby calculate.rb
55

Ruby 1.9.0 patch 0 on i386-cygwin
It took 3.847 seconds to run. 259942 iterations per
second.

 

[SpringFlowers AutumnMoon]

on a 1.86GHz Intel Core 2 Duo CPU (Intel 6320) running Win XP:


Ruby 1.9.1 patch 0 on i386-mswin32
It took 5.9375 seconds to run. 168421 iterations per second.

 

[Jian Lin]

Program changed a little:

n = 3_000_000

start_time = Time.now

t = 0
1.upto(n) do |i|
t = (1..10).inject {|x, y| x + y }
end

finish_time = Time.now

p t

puts
RUBY_VERSION ||= "unknown version"
RUBY_PATCHLEVEL ||= "unknown patchlevel"
RUBY_PLATFORM ||= "unknown platform"

print "Ruby ", RUBY_VERSION, " patch ", RUBY_PATCHLEVEL, " on ",
RUBY_PLATFORM

puts
print "It took #{finish_time - start_time} seconds to run."
print " #{(n / (finish_time - start_time)).to_i} iterations per
second.\n"


D:\>ruby calc.rb
55

Ruby 1.9.1 patch 378 on i386-mingw32
It took 6.414366 seconds to run. 467700 iterations per second.

Dell with Intel Quad Core Q6600

=================================================================

C:\>ruby19\bin\ruby.exe calc.rb
55

Ruby 1.9.1 patch 378 on i386-mingw32
It took 5.289302 seconds to run. 567182 iterations per second.

Dell with Intel i7 920, 2.67GHz

the same machine, but using Ruby 1.8.7:

Ruby 1.8.7 patch 249 on i386-mingw32
It took 20.26816 seconds to run. 148015 iterations per second.

 

[Josh Cheek]

[Note: parts of this message were removed to make it a legal post.]

Program changed a little:

n = 3_000_000

start_time = Time.now

t = 0
1.upto(n) do |i|
t = (1..10).inject {|x, y| x + y }
end

finish_time = Time.now

p t

puts
RUBY_VERSION ||= "unknown version"
RUBY_PATCHLEVEL ||= "unknown patchlevel"
RUBY_PLATFORM ||= "unknown platform"

print "Ruby ", RUBY_VERSION, " patch ", RUBY_PATCHLEVEL, " on ",
RUBY_PLATFORM

puts
print "It took #{finish_time - start_time} seconds to run."
print " #{(n / (finish_time - start_time)).to_i} iterations per
second.\n"


D:\>ruby calc.rb
55

Ruby 1.9.1 patch 378 on i386-mingw32
It took 6.414366 seconds to run. 467700 iterations per second.

Dell with Intel Quad Core Q6600

=================================================================

C:\>ruby19\bin\ruby.exe calc.rb
55

Ruby 1.9.1 patch 378 on i386-mingw32
It took 5.289302 seconds to run. 567182 iterations per second.

Dell with Intel i7 920, 2.67GHz

the same machine, but using Ruby 1.8.7:

Ruby 1.8.7 patch 249 on i386-mingw32
It took 20.26816 seconds to run. 148015 iterations per second.

$rvm list

rvm rubies

jruby-1.4.0 [ [x86_64-java] ]
jruby-1.5.0 [ [x86_64-java] ]
macruby-0.6 [ x86_64 ]
rbx-head [ x86_64 ]
ruby-1.8.6-p399 [ x86_64 ]
ruby-1.8.7-p174 [ x86_64 ]
ruby-1.8.7-p249 [ x86_64 ]
ruby-1.9.1-p378 [ x86_64 ]
ruby-1.9.2-preview3 [ x86_64 ]


# ===== 1.8.7 =====
$ rvm use 1.8.7-p249
info: Using ruby 1.8.7 p249

$ ruby calc.rb
Ruby 1.8.7 patch 249 on i686-darwin10.3.0
It took 22.411302 seconds to run. 133861 iterations per second.


# ===== 1.9.1 =====
$ rvm use 1.9.1
info: Using ruby 1.9.1 p378

$ ruby calc.rb
Ruby 1.9.1 patch 378 on i386-darwin10.3.0
It took 4.696109 seconds to run. 638826 iterations per second.


# ===== JRuby 1.5.0 =====
$ rvm use jruby
info: Using jruby 1.5.0

$ ruby calc.rb
Ruby 1.8.7 patch 249 on java
It took 9.12 seconds to run. 328947 iterations per second.


# ===== MacRuby 0.6 =====
$ rvm use macruby
info: Using macruby 0.6

$ ruby calc.rb
Ruby 1.9.0 patch 0 on universal-darwin10.0
It took 5.059672 seconds to run. 592923 iterations per second.


# ===== Rubinius 1.0 =====
$ rvm use rbx-head
info: Using rbx head

$ ruby -v
rubinius 1.0.0 (1.8.7 32151bb2 2010-05-14 JI) [x86_64-apple-darwin10.3.0]

$ ruby calc.rb
Ruby 1.8.7 patch 174 on x86_64-apple-darwin10.3.0
It took 19.90644 seconds to run. 150704 iterations per second.

$ruby calc.rb
Ruby 1.8.7 patch 174 on x86_64-apple-darwin10.3.0
It took 19.840837 seconds to run. 151203 iterations per second.





Results are interesting, I still use 1.8.7 most of the time, because,
regardless of what this test says, the only time I ever notice the speed of
my Ruby is when I am running specs or rake tasks and 1.8.7 runs them
noticeably faster (than 1.9.1, at least. I should try with some of the
others, but since my last rvm update, I just seem to be having a lot of
trouble with gems and versions).

My Rubinius result seems unlikely to be an accurate reflection of that
project, I thought the first time you run it it takes a long time b/c it has
to compile bytecode, then after that, it uses the bytecode, and is much
quicker. But it took just as long both times. I scanned their
getting_started, and FAQ, and didn't see anything to suggest I am doing it
wrong, so IDK. Perhaps I misunderstand the project, or need to optimize rbx
by hand rather than letting rvm deal with it.