Problem wrapping the C++ STL

Discussion in 'Ruby' started by Daniel Berger, Jun 18, 2007.

  1. Hi all,

    Is it possible to wrap the C++ STL stuff? Say, for example, vector?

    #include "ruby.h"
    #include <vector>
    using namespace std;

    struct cvector{
    vector<VALUE> array(0);
    };

    void Init_cvector(){
    VALUE cVector = rb_define_class("CVector", rb_cObject);
    }

    When I try to compile this (using g++ on OS X) I get:

    /usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
    enum 'rb_thread_status' without previous declaration
    /usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: invalid
    type in declaration before ';' token
    cvector.c:6: error: expected identifier before numeric constant
    cvector.c:6: error: expected ',' or '...' before numeric constant
    cvector.c: In function 'void Init_cvector()':
    cvector.c:10: warning: unused variable 'cVector'
    make: *** [cvector.o] Error 1

    Thanks,

    Dan
     
    Daniel Berger, Jun 18, 2007
    #1
    1. Advertising

  2. > Is it possible to wrap the C++ STL stuff? Say, for example, vector?
    Yes. I wrapped set for instance

    > /usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
    > enum 'rb_thread_status' without previous declaration

    This is not a C++ bug. This is a bug in 1.8.6-p36, for which I know no
    workaround. rb_thread_status is defined in node.h but ruby.h inclu

    > cvector.c:6: error: expected identifier before numeric constant
    > cvector.c:6: error: expected ',' or '...' before numeric constant

    You're trying to initialize a struct field. You need to do
    struct cvector {
    vector<VALUE> array;
    };
    an resize the array if needed.

    Note that you can wrap std::vector directly (no need to go through an
    intermediate cvector structure)

    Sylvain
     
    Sylvain Joyeux, Jun 18, 2007
    #2
    1. Advertising

  3. Daniel Berger

    Eric Mahurin Guest

    On 6/18/07, Daniel Berger <> wrote:
    > Hi all,
    >
    > Is it possible to wrap the C++ STL stuff? Say, for example, vector?
    >
    > #include "ruby.h"
    > #include <vector>
    > using namespace std;
    >
    > struct cvector{
    > vector<VALUE> array(0);
    > };
    >
    > void Init_cvector(){
    > VALUE cVector =3D rb_define_class("CVector", rb_cObject);
    > }
    >
    > When I try to compile this (using g++ on OS X) I get:
    >
    > /usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: use of
    > enum 'rb_thread_status' without previous declaration
    > /usr/local/lib/ruby/1.8/i686-darwin8.9.1/intern.h:207: error: invalid
    > type in declaration before ';' token
    > cvector.c:6: error: expected identifier before numeric constant
    > cvector.c:6: error: expected ',' or '...' before numeric constant
    > cvector.c: In function 'void Init_cvector()':
    > cvector.c:10: warning: unused variable 'cVector'
    > make: *** [cvector.o] Error 1
    >
    > Thanks,
    >
    > Dan


    Have you tried swig? Gonzalo Garramu=F1o has done some work to make the
    ruby<->STL interface better (faster and more STL functionality). I've
    also done similar python<->STL modifications for swig. I'd love to
    see all of this completed with full STL functionality (containers,
    iterators, algorithms) and the big-O performance preserved. C++ STL
    is quite rich w/ a lot of thought put into big-O performance (great
    when dealing with large datasets).
     
    Eric Mahurin, Jun 18, 2007
    #3
  4. Hi,

    At Mon, 18 Jun 2007 23:15:26 +0900,
    Daniel Berger wrote in [ruby-talk:256045]:
    > Is it possible to wrap the C++ STL stuff? Say, for example, vector?


    In general, C++ isn't supported in 1.8.

    --
    Nobu Nakada
     
    Nobuyoshi Nakada, Jun 19, 2007
    #4
  5. > g++ -Wall -I. -I/opt/lib/ruby/1.8/i686-darwin8.9.1
    > -I/opt/lib/ruby/1.8/i686-darwin8.9.1 -I. -fno-common -g -O2 -pipe
    > -fno-common -c cvector.c
    > cvector.c: In function 'void Init_cvector()':
    > cvector.c:10: error: invalid conversion from 'VALUE (*)(int, VALUE*,
    > VALUE)' to 'VALUE (*)(...)'
    > cvector.c:10: error: initializing argument 3 of 'void
    > rb_define_method(VALUE, const char*, VALUE (*)(...), int)'
    > make: *** [cvector.o] Error 1


    C++ is more pissy than C about type convertions. You have to use the
    RUBY_METHOD_FUNC macro to convert cvector_init into the right type:

    rb_define_method(cVector, "initialize",
    RUBY_METHOD_FUNC(cvector_init), -1);

    > > Note that you can wrap std::vector directly (no need to go through an
    > > intermediate cvector structure)

    >
    > I guess I could store it as an instance variable within the constructor
    > and refer back to it that way. I don't think it's faster, though. Or,
    > did you have something else in mind?

    The idea is *not* to use a cvector structure. Since you'll have to provide
    an alloc/free method pair anyway, allocate std::vector in them. Check
    value_set_alloc/value_set_free in
    http://www.laas.fr/~sjoyeux/darcs/utilrb/ext/value_set.cc

    Sylvain
     
    Sylvain Joyeux, Jun 19, 2007
    #5
  6. On Jun 19, 1:07 am, Sylvain Joyeux <>
    wrote:
    > > g++ -Wall -I. -I/opt/lib/ruby/1.8/i686-darwin8.9.1
    > > -I/opt/lib/ruby/1.8/i686-darwin8.9.1 -I. -fno-common -g -O2 -pipe
    > > -fno-common -c cvector.c
    > > cvector.c: In function 'void Init_cvector()':
    > > cvector.c:10: error: invalid conversion from 'VALUE (*)(int, VALUE*,
    > > VALUE)' to 'VALUE (*)(...)'
    > > cvector.c:10: error: initializing argument 3 of 'void
    > > rb_define_method(VALUE, const char*, VALUE (*)(...), int)'
    > > make: *** [cvector.o] Error 1

    >
    > C++ is more pissy than C about type convertions. You have to use the
    > RUBY_METHOD_FUNC macro to convert cvector_init into the right type:
    >
    > rb_define_method(cVector, "initialize",
    > RUBY_METHOD_FUNC(cvector_init), -1);


    Ah, thanks.

    > > > Note that you can wrap std::vector directly (no need to go through an
    > > > intermediate cvector structure)

    >
    > > I guess I could store it as an instance variable within the constructor
    > > and refer back to it that way. I don't think it's faster, though. Or,
    > > did you have something else in mind?

    >
    > The idea is *not* to use a cvector structure. Since you'll have to provide
    > an alloc/free method pair anyway, allocate std::vector in them. Check
    > value_set_alloc/value_set_free inhttp://www.laas.fr/~sjoyeux/darcs/utilrb/ext/value_set.cc


    Interesting, thank you. For kicks, I tried to compile your source code
    on my Solaris 10 box (after installing boost). It built (with some
    warnings), but I can't get it to load.

    Here's the extconf.rb file I used:

    require 'mkmf'
    dir_config('set2')

    case RUBY_PLATFORM
    when /sunos|solaris/
    CONFIG['CC'] = 'CC'
    when /mswin/i
    CONFIG["COMPILE_C"].sub!(/-Tc/, '-Tp')
    else
    CONFIG['CC'] = 'g++ -Wall'
    end

    create_makefile('set2')

    Here was the result of the build step:

    djberge-/export/home/djberge/workspace/set/ext-635>ruby extconf.rb --
    with-set2-include=/opt/csw/include
    creating Makefile

    djberge-/export/home/djberge/workspace/set/ext-636>make
    CC -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
    ruby/1.8/sparc-solaris2.10 -I. -I/opt/csw/include -KPIC -dalign -fns -
    xbuiltin=%all -xlibmil -xtarget=ultra2e -xO5 -xipo -c set.c
    "set.c", line 349: Warning (Anachronism): Formal argument 3 of type
    extern "C" unsigned long(*)(...) in call to rb_iterate(extern "C"
    unsigned long(*)(unsigned long), unsigned long, extern "C" unsigned
    long(*)(...), unsigned long) is being passed unsigned long(*)(...).
    "set.c", line 369: Warning (Anachronism): Formal argument 2 of type
    extern "C" unsigned long(*)(unsigned long) in call to
    rb_define_alloc_func(unsigned long, extern "C" unsigned long(*)
    (unsigned long)) is being passed unsigned long(*)(unsigned long).
    2 Warning(s) detected.
    ld -G -o set2.so set.o -L'.' -L'/usr/local/lib' -R'/usr/local/lib' -
    L. -lrt -lpthread -ldl -lcrypt -lm -lc

    Ok, a couple warnings. I proceed to try to "require 'set2'" and I get
    this:

    djberge-/export/home/djberge/workspace/set/ext-637>ruby test.rb
    /export/home/djberge/workspace/set/ext/set2.so: ld.so.1: ruby: fatal:
    relocation error: file /export/home/djberge/workspace/set/ext/set2.so:
    symbol __1cDstdJbad_allocG__vtbl_: referenced symbol not found - /
    export/home/djberge/workspace/set/ext/set2.so (LoadError)
    from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
    custom_require.rb:27:in `require'
    from test.rb:2

    This was with Ruby 1.8.6-p38 (today's svn checkout of the 1.8.6
    branch).

    Any ideas?

    Dan
     
    Daniel Berger, Jun 19, 2007
    #6
  7. Daniel Berger

    MenTaLguY Guest

    Incidentally, I did want to throw in a note to be very, very careful about exceptions when wrapping C++ code as a Ruby extension.

    It's necessary to catch and wrap/unwrap exceptions at all C++/C boundaries, since otherwise, Ruby cleanup (ensure, rescue, etc...) will be skipped when C++ exceptions unwind the stack (potentially trashing the interpreter), and C++ cleanup (destructors for objects in automatic storage, catch, etc...) will get skipped when Ruby exceptions unwind the stack.

    Also, on some architectures, a C++ exception tearing down a C stack frame is by itself enough to cause badness.

    boost.python takes care of all this for writing Python extensions in C++, but sadly there's not a maintained Ruby equivalent.

    -mental
     
    MenTaLguY, Jun 19, 2007
    #7
  8. On Jun 19, 11:13 am, MenTaLguY <> wrote:
    > Incidentally, I did want to throw in a note to be very, very careful about exceptions when wrapping C++ code as a Ruby extension.
    >
    > It's necessary to catch and wrap/unwrap exceptions at all C++/C boundaries, since otherwise, Ruby cleanup (ensure, rescue, etc...) will be skipped when C++ exceptions unwind the stack (potentially trashing the interpreter), and C++ cleanup (destructors for objects in automatic storage, catch, etc...) will get skipped when Ruby exceptions unwind the stack.
    >
    > Also, on some architectures, a C++ exception tearing down a C stack frame is by itself enough to cause badness.
    >
    > boost.python takes care of all this for writing Python extensions in C++, but sadly there's not a maintained Ruby equivalent.
    >
    > -mental


    Maybe we should just rewrite the interpreter in C++ then. Think of all
    the STL and Boost stuff we could then integrate "for free". I can only
    guess what other advantages this might bring in the long run.

    I know Matz has said in the past that the C++ object model would get
    in the way, but I'm convinced it could be done (by smarter minds than
    I).

    Dan
     
    Daniel Berger, Jun 19, 2007
    #8
  9. Daniel Berger

    MenTaLguY Guest

    On Wed, 20 Jun 2007 02:37:39 +0900, Daniel Berger <> wrote:
    > Maybe we should just rewrite the interpreter in C++ then. Think of all
    > the STL and Boost stuff we could then integrate "for free". I can only
    > guess what other advantages this might bring in the long run.


    It should be nearly sufficient to replace the setjmp/longjump bits
    with C++ exceptions and build the interpreter with a C++ compiler.

    However, that just exchanges one problem for another since the bulk of Ruby
    extensions are written in C, and embedding Ruby becomes even more
    difficult.

    I think we'd all be much better off if someone wrote a Ruby equivalent to
    boost.python. There is a partially-completed one out there, but it's
    unmaintained I'm not able to find it again at the moment.

    -mental
     
    MenTaLguY, Jun 19, 2007
    #9
  10. On Jun 19, 11:47 am, MenTaLguY <> wrote:
    > On Wed, 20 Jun 2007 02:37:39 +0900, Daniel Berger <> wrote:
    > > Maybe we should just rewrite the interpreter in C++ then. Think of all
    > > the STL and Boost stuff we could then integrate "for free". I can only
    > > guess what other advantages this might bring in the long run.

    >
    > It should be nearly sufficient to replace the setjmp/longjump bits
    > with C++ exceptions and build the interpreter with a C++ compiler.
    >
    > However, that just exchanges one problem for another since the bulk of Ruby
    > extensions are written in C, and embedding Ruby becomes even more
    > difficult.


    I'd make that trade. The potential benefits far exceed the pain of
    reworking extensions IMHO, especially when you consider that *most*
    people using Ruby don't do embedding or extending.

    And this is coming from a guy who has written quite a few extensions.

    Regards,

    Dan
     
    Daniel Berger, Jun 19, 2007
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Allan Bruce

    To STL or not to STL

    Allan Bruce, Oct 16, 2003, in forum: C++
    Replies:
    41
    Views:
    1,071
    Christopher Benson-Manica
    Oct 17, 2003
  2. Replies:
    4
    Views:
    807
    Daniel T.
    Feb 16, 2006
  3. Replies:
    2
    Views:
    556
    klaus hoffmann
    Feb 22, 2006
  4. Replies:
    5
    Views:
    509
    Markus Schoder
    Apr 16, 2006
  5. Steve
    Replies:
    2
    Views:
    511
    Andre Kostur
    Nov 6, 2007
Loading...

Share This Page