Problem wrapping the C++ STL

[Daniel Berger]

Hi all,

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

#include "ruby.h"
#include <vector>
using namespace std;

struct cvector{
vector<VALUE> array(0);
};

void Init_cvector(){
VALUE cVector = rb_define_class("CVector", rb_cObject);
}

When I try to compile this (using g++ on OS X) I get:

/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
enum 'rb_thread_status' without previous declaration
/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: invalid
type in declaration before ';' token
cvector.c:6: error: expected identifier before numeric constant
cvector.c:6: error: expected ',' or '...' before numeric constant
cvector.c: In function 'void Init_cvector()':
cvector.c:10: warning: unused variable 'cVector'
make: *** [cvector.o] Error 1

Thanks,

Dan

 

[Sylvain Joyeux]

Is it possible to wrap the C++ STL stuff? Say, for example, vector?
Yes. I wrapped set for instance

/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
enum 'rb_thread_status' without previous declaration

This is not a C++ bug. This is a bug in 1.8.6-p36, for which I know no
workaround. rb_thread_status is defined in node.h but ruby.h inclu

cvector.c:6: error: expected identifier before numeric constant
cvector.c:6: error: expected ',' or '...' before numeric constant

You're trying to initialize a struct field. You need to do
struct cvector {
vector<VALUE> array;
};
an resize the array if needed.

Note that you can wrap std::vector directly (no need to go through an
intermediate cvector structure)

Sylvain

 

[Eric Mahurin]

Hi all,

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

#include "ruby.h"
#include <vector>
using namespace std;

struct cvector{
vector<VALUE> array(0);
};

void Init_cvector(){
VALUE cVector =3D rb_define_class("CVector", rb_cObject);
}

When I try to compile this (using g++ on OS X) I get:

/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
enum 'rb_thread_status' without previous declaration
/usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: invalid
type in declaration before ';' token
cvector.c:6: error: expected identifier before numeric constant
cvector.c:6: error: expected ',' or '...' before numeric constant
cvector.c: In function 'void Init_cvector()':
cvector.c:10: warning: unused variable 'cVector'
make: *** [cvector.o] Error 1

Thanks,

Dan

Have you tried swig? Gonzalo Garramu=F1o has done some work to make the
ruby<->STL interface better (faster and more STL functionality). I've
also done similar python<->STL modifications for swig. I'd love to
see all of this completed with full STL functionality (containers,
iterators, algorithms) and the big-O performance preserved. C++ STL
is quite rich w/ a lot of thought put into big-O performance (great
when dealing with large datasets).

 

[Nobuyoshi Nakada]

Hi,

At Mon, 18 Jun 2007 23:15:26 +0900,
Daniel Berger wrote in [ruby-talk:256045]:

Is it possible to wrap the C++ STL stuff? Say, for example, vector?

In general, C++ isn't supported in 1.8.

 

[Sylvain Joyeux]

g++ -Wall -I. -I/opt/lib/ruby/1.8/i686-darwin8.9.1

-I/opt/lib/ruby/1.8/i686-darwin8.9.1 -I. -fno-common -g -O2 -pipe
-fno-common -c cvector.c
cvector.c: In function 'void Init_cvector()':
cvector.c:10: error: invalid conversion from 'VALUE (*)(int, VALUE*,
VALUE)' to 'VALUE (*)(...)'
cvector.c:10: error: initializing argument 3 of 'void
rb_define_method(VALUE, const char*, VALUE (*)(...), int)'
make: *** [cvector.o] Error 1

C++ is more pissy than C about type convertions. You have to use the
RUBY_METHOD_FUNC macro to convert cvector_init into the right type:

rb_define_method(cVector, "initialize",
RUBY_METHOD_FUNC(cvector_init), -1);

I guess I could store it as an instance variable within the constructor
and refer back to it that way. I don't think it's faster, though. Or,
did you have something else in mind?

The idea is *not* to use a cvector structure. Since you'll have to provide
an alloc/free method pair anyway, allocate std::vector in them. Check
value_set_alloc/value_set_free in
http://www.laas.fr/~sjoyeux/darcs/utilrb/ext/value_set.cc

Sylvain

 

[Daniel Berger]

g++ -Wall -I. -I/opt/lib/ruby/1.8/i686-darwin8.9.1
-I/opt/lib/ruby/1.8/i686-darwin8.9.1 -I. -fno-common -g -O2 -pipe
-fno-common -c cvector.c
cvector.c: In function 'void Init_cvector()':
cvector.c:10: error: invalid conversion from 'VALUE (*)(int, VALUE*,
VALUE)' to 'VALUE (*)(...)'
cvector.c:10: error: initializing argument 3 of 'void
rb_define_method(VALUE, const char*, VALUE (*)(...), int)'
make: *** [cvector.o] Error 1

C++ is more pissy than C about type convertions. You have to use the
RUBY_METHOD_FUNC macro to convert cvector_init into the right type:

rb_define_method(cVector, "initialize",
RUBY_METHOD_FUNC(cvector_init), -1);

Ah, thanks.

The idea is *not* to use a cvector structure. Since you'll have to provide
an alloc/free method pair anyway, allocate std::vector in them. Check
value_set_alloc/value_set_free inhttp://www.laas.fr/~sjoyeux/darcs/utilrb/ext/value_set.cc

Interesting, thank you. For kicks, I tried to compile your source code
on my Solaris 10 box (after installing boost). It built (with some
warnings), but I can't get it to load.

Here's the extconf.rb file I used:

require 'mkmf'
dir_config('set2')

case RUBY_PLATFORM
when /sunos|solaris/
CONFIG['CC'] = 'CC'
when /mswin/i
CONFIG["COMPILE_C"].sub!(/-Tc/, '-Tp')
else
CONFIG['CC'] = 'g++ -Wall'
end

create_makefile('set2')

Here was the result of the build step:

djberge-/export/home/djberge/workspace/set/ext-635>ruby extconf.rb --
with-set2-include=/opt/csw/include
creating Makefile

djberge-/export/home/djberge/workspace/set/ext-636>make
CC -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -I/opt/csw/include -KPIC -dalign -fns -
xbuiltin=%all -xlibmil -xtarget=ultra2e -xO5 -xipo -c set.c
"set.c", line 349: Warning (Anachronism): Formal argument 3 of type
extern "C" unsigned long(*)(...) in call to rb_iterate(extern "C"
unsigned long(*)(unsigned long), unsigned long, extern "C" unsigned
long(*)(...), unsigned long) is being passed unsigned long(*)(...).
"set.c", line 369: Warning (Anachronism): Formal argument 2 of type
extern "C" unsigned long(*)(unsigned long) in call to
rb_define_alloc_func(unsigned long, extern "C" unsigned long(*)
(unsigned long)) is being passed unsigned long(*)(unsigned long).
2 Warning(s) detected.
ld -G -o set2.so set.o -L'.' -L'/usr/local/lib' -R'/usr/local/lib' -
L. -lrt -lpthread -ldl -lcrypt -lm -lc

Ok, a couple warnings. I proceed to try to "require 'set2'" and I get
this:

djberge-/export/home/djberge/workspace/set/ext-637>ruby test.rb
/export/home/djberge/workspace/set/ext/set2.so: ld.so.1: ruby: fatal:
relocation error: file /export/home/djberge/workspace/set/ext/set2.so:
symbol __1cDstdJbad_allocG__vtbl_: referenced symbol not found - /
export/home/djberge/workspace/set/ext/set2.so (LoadError)
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
from test.rb:2

This was with Ruby 1.8.6-p38 (today's svn checkout of the 1.8.6
branch).

Any ideas?

Dan

 

[MenTaLguY]

Incidentally, I did want to throw in a note to be very, very careful about exceptions when wrapping C++ code as a Ruby extension.

It's necessary to catch and wrap/unwrap exceptions at all C++/C boundaries, since otherwise, Ruby cleanup (ensure, rescue, etc...) will be skipped when C++ exceptions unwind the stack (potentially trashing the interpreter), and C++ cleanup (destructors for objects in automatic storage, catch, etc...) will get skipped when Ruby exceptions unwind the stack.

Also, on some architectures, a C++ exception tearing down a C stack frame is by itself enough to cause badness.

boost.python takes care of all this for writing Python extensions in C++, but sadly there's not a maintained Ruby equivalent.

-mental

 

[Daniel Berger]

Incidentally, I did want to throw in a note to be very, very careful about exceptions when wrapping C++ code as a Ruby extension.

It's necessary to catch and wrap/unwrap exceptions at all C++/C boundaries, since otherwise, Ruby cleanup (ensure, rescue, etc...) will be skipped when C++ exceptions unwind the stack (potentially trashing the interpreter), and C++ cleanup (destructors for objects in automatic storage, catch, etc...) will get skipped when Ruby exceptions unwind the stack.

Also, on some architectures, a C++ exception tearing down a C stack frame is by itself enough to cause badness.

boost.python takes care of all this for writing Python extensions in C++, but sadly there's not a maintained Ruby equivalent.

-mental

Maybe we should just rewrite the interpreter in C++ then. Think of all
the STL and Boost stuff we could then integrate "for free". I can only
guess what other advantages this might bring in the long run.

I know Matz has said in the past that the C++ object model would get
in the way, but I'm convinced it could be done (by smarter minds than
I).

Dan

 

[MenTaLguY]

Maybe we should just rewrite the interpreter in C++ then. Think of all
the STL and Boost stuff we could then integrate "for free". I can only
guess what other advantages this might bring in the long run.

It should be nearly sufficient to replace the setjmp/longjump bits
with C++ exceptions and build the interpreter with a C++ compiler.

However, that just exchanges one problem for another since the bulk of Ruby
extensions are written in C, and embedding Ruby becomes even more
difficult.

I think we'd all be much better off if someone wrote a Ruby equivalent to
boost.python. There is a partially-completed one out there, but it's
unmaintained I'm not able to find it again at the moment.

-mental

 

[Daniel Berger]

It should be nearly sufficient to replace the setjmp/longjump bits
with C++ exceptions and build the interpreter with a C++ compiler.

However, that just exchanges one problem for another since the bulk of Ruby
extensions are written in C, and embedding Ruby becomes even more
difficult.

I'd make that trade. The potential benefits far exceed the pain of
reworking extensions IMHO, especially when you consider that *most*
people using Ruby don't do embedding or extending.

And this is coming from a guy who has written quite a few extensions.

Regards,

Dan