structuring complexly-interdependent C/Ruby libraries

[Eric Peden]

First post. Hi.

I'm writing a ray-tracer in Ruby (I know, I know...) Perhaps some other
time I'll wax ecstatic over the advantages Ruby brings to such a
project, but at the moment I have less pleasant issues I'm hoping to
get some help with.

A pure-Ruby ray tracer is just a tad slow. Clearly, some core
functionality needs to be implemented in C for speed. The system is
broken into classes similar to the following:

Camera
Engine
Vector
Point
Ray
Shape
Sphere
Polygon

Of those classes, only a handful are performance critical, and of each
of those, only one or two methods are true bottlenecks. I'm trying to
keep as much of the code as possible in Ruby, while breaking out only
the real work horses into C. Vector, for example, is almost pure C,
whereas only the intersection-testing method in the Shape subclasses is
in C. Unfortunately, I keep running into dependency problems when
trying to build the C functions. My C code is organized like this:

ext/
extconf.rb
ext.c
vector.c
sphere.c

When built, this generates a single library that includes the code for
all of the optimized classes. 'ext.c' has an Init_ function that calls
initialization routines in the other files. Here's an example from
sphere.c:

Init_Sphere() {
VALUE cSphere = rb_const_get(rb_cObject, rb_intern("Sphere"));
}

sphere.rb would look something like this:

require 'vector'
require 'ext/optimized'

class Sphere < Shape
...
end

When I run this, I get a "undefined constant" error from the C code,
which makes sense: when the C library is included, Sphere hasn't been
defined yet. So I just move the "require 'ext/optimized'" to the bottom
of sphere.rb, right? Sadly, no. There's only one library entry point,
so I have to initialize all of my optimized classes at the same time.
'vector.rb' also requires 'ext/optimized', so Init_Sphere() would still
get called before Sphere is actually defined, and I'll still get my
error. Furthermore, since Vector gets used in several other files, I
have to worry about the order in which those files are parsed, as well.

I can see a few solutions: one is to break out every C file so that it
compiles into its own library. This would leave me with an ugly file
structure, since I'd need either a new extconf.rb for each C file, or a
sub-directory for each source file. I've tried doing something like
this:

VALUE cSphere = rb_eval_string("class Shape; end; class Sphere <
Shape; end");

but that feels extremely naughty, not to mention requires me to
duplicate the heritage of each subclass I want to provide C extensions
for. I suppose I could also create a "Optimizer" object in my C
library's Init() method, and attach methods to that class that would
call the class-specific Init_...() methods:

# a .rb file
require 'ext/optimized'
class Sphere < Shape
..
end
Optimizer.init_sphere

But that also seems rather distasteful.

I apologize for my wordiness... I'm trying to make my situation clear.
Am I making this more difficult than it really is? If the classes
involved where pure-C, or if there wasn't any subclassing, I'd be fine.
I just can't see how to manage the class dependencies while dealing
with a single library entry point, and so I've come to seek the
guidance of the wizards. Any help--or even architecture bashing, if
it's constructive--is appreciated.

 

[Pit Capitain]

First post. Hi.
Welcome!

(...code snipped...)

When I run this, I get a "undefined constant" error from the C code,
which makes sense: when the C library is included, Sphere hasn't been
defined yet.

I've never extended Ruby with C yet, but couldn't you create the optimized
classes in C and later add some Ruby methods to them?

Regards,
Pit

 

[leon breedt]

I apologize for my wordiness... I'm trying to make my situation clear.
Am I making this more difficult than it really is? If the classes
involved where pure-C, or if there wasn't any subclassing, I'd be fine.
I just can't see how to manage the class dependencies while dealing
with a single library entry point, and so I've come to seek the
guidance of the wizards. Any help--or even architecture bashing, if
it's constructive--is appreciated.

I always define classes with C elements in the extension itself.

You know you can define a class and the methods on it in Ruby as many
times as you want as long as you don't change the inheritance? So its
no problem to later in Ruby redefine the class with its pure-Ruby
methods.

Leon

 

[Michael Neumann]

First post. Hi.

I'm writing a ray-tracer in Ruby (I know, I know...) Perhaps some other
time I'll wax ecstatic over the advantages Ruby brings to such a
project, but at the moment I have less pleasant issues I'm hoping to get
some help with.

A pure-Ruby ray tracer is just a tad slow. Clearly, some core
functionality needs to be implemented in C for speed. The system is
broken into classes similar to the following:

Camera
Engine
Vector
Point
Ray
Shape
Sphere
Polygon

Of those classes, only a handful are performance critical, and of each
of those, only one or two methods are true bottlenecks. I'm trying to
keep as much of the code as possible in Ruby, while breaking out only
the real work horses into C. Vector, for example, is almost pure C,
whereas only the intersection-testing method in the Shape subclasses is
in C. Unfortunately, I keep running into dependency problems when trying
to build the C functions. My C code is organized like this:

ext/
extconf.rb
ext.c
vector.c
sphere.c

You might take a look at RubyInline:

http://rubyforge.org/projects/rubyinline/

It will allow you to put the C code right into your Ruby classes.

Regards,

Michael

 

[Kaspar Schiess]

Hello Eric,

Usually, you define classes in your C extension and likewise in your Ruby
code, doing things twice, which does not matter since Ruby will take your
meaning (creating one class).

-- myclass.c
Init_... {
cMyClass = rb_define_class_under( mMyModule, "MyClass", rb_cObject );
}

-- myclass.rb
require 'myclass.so'
class MyClass
# ...
end

But yeah, redefinition with a different superclass gives you a
SuperClassMismatch. So I would aim for different classes altogether, using
mixins to mix the c code into the Ruby code:

-- myclass.c
Init_... {
mMyClassMixin = rb_define_module_under( mMyModule, "MyClassCMixin" );
...
}

-- myclass.rb
require 'myclasscmixin'

class MyClass < MySuperClass
include MyClassCMixin
#...
end

This method is tested and works, and it looks clean enough to me. Now does
this solve all your problems ? Yes, if I read you right.

k

code manufacture & ruby lab at http://www.tua.ch/ruby

 

[Eric Peden]

You might take a look at RubyInline:

http://rubyforge.org/projects/rubyinline/

It will allow you to put the C code right into your Ruby classes.

I have looked at RubyInline (and in fact use it in a couple of places
in my code). It works great for shorter methods. However, for longer
methods, or classes where I C-ify several methods, I prefer breaking
the code out in a .c file. This makes my text editor happy, and lets me
work with C-specific syntax highlighting and autocompletion features.

There are also a few times where I want to work directly with the C
representation of another optimized class, and I had trouble doing this
with Inline. This was early on in my explorations, so it's likely I was
just doing things the wrong way.

Thanks for the suggestion!

 

[Eric Peden]

Usually, you define classes in your C extension and likewise in your
Ruby
code, doing things twice, which does not matter since Ruby will take
your
meaning (creating one class).

This was suggested by several people (thanks, Pit and Leon!), and not
only captures the essence of Ruby, but would work quite well... if it
weren't for my need for inheritance, as you point out shortly. I
_could_ also create "empty" versions of my superclasses in C, but then
I'm recreating the inheritance hierarchy again. Or...

But yeah, redefinition with a different superclass gives you a
SuperClassMismatch. So I would aim for different classes altogether,
using
mixins to mix the c code into the Ruby code:

This method is tested and works, and it looks clean enough to me. Now
does
this solve all your problems ? Yes, if I read you right.

...I could just do this. I believe this is exactly what I need. Thanks
for the help!

 

[Kaspar Schiess]

(In response to by Eric
Peden)

...I could just do this. I believe this is exactly what I need. Thanks
for the help!

np. Just found out about this too. Thanks for asking the cool question.

k

 

[leon breedt]

This was suggested by several people (thanks, Pit and Leon!), and not
only captures the essence of Ruby, but would work quite well... if it
weren't for my need for inheritance, as you point out shortly. I
_could_ also create "empty" versions of my superclasses in C, but then
I'm recreating the inheritance hierarchy again. Or...

I'm not sure you need to recreate it:

class Super
end

class Test < Super
def one
puts "one!"
end
end

t1 = Test.new

class Test
def two
puts "two!"
end
end

t2 = Test.new

puts t1.class.ancestors.inspect
puts t2.class.ancestors.inspect


It appears that Ruby "remembers" the ancestry from the first time that
class was defined using that name.

So if you define it in C first, and in your Ruby code, require the
extension before redefining the classes, you don't need to recreate
the inheritance hierarchy.

Leon