[ANN] rocaml 0.6.0: fast, easy Ruby extensions in Objective Caml

M

Mauricio Fernandez

rocaml allows you to write Ruby extensions in Objective Caml.

http://eigenclass.org/hiki/rocaml

Developing Ruby extensions with rocaml is easier and more convenient than
writing a plain old C extension because rocaml performs Ruby <-> OCaml
conversions for a wide range of types, including abstract types and arrays,
tuples, variants and records of values of any supported type (e.g. arrays of
arrays of variants of tuples of ...).

Moreover, exceptions raised in the OCaml code are captured by the generated
extension and raised inside Ruby.

Making an extension with rocaml involves two steps:
* implementing the desired functionality in Objective Caml, and registering
the functions to be exported (using Callback.register : string -> 'a -> unit
or the included camlp4 extension)
* creating the extconf.rb file (just modify the sample extconf.rb distributed
with rocaml) defining the interface of your Objective Caml code.

** At no point is there any need to write a single line of C code. **

The mandatory trivial example
=============================

This example doesn't do justice to the usefulness of rocaml because the
extension is beyond trivial and you could as well have written it in C using
RubyInline. The advantages of rocaml (and of Objective Caml) usually become
visible when the extension takes more than two lines (take a look at the
3-line Marshal replacement that is 3 times faster than Ruby's, though...).
Here follows a minimal example however, merely to show how easily rocaml
extensions can be made.

Here's the OCaml code placed in fib.ml:
let rec fib n = if n < 2 then 1 else fib (n-1) + fib (n-2)
export fib

Here's the interface declaration in your extconf.rb:
Interface.generate("fib") do
def_module("Fib") do
fun "fib", INT => INT
end
end

That's it. The extension can be built like any run-of-the-mill C extension
with
ruby extconf.rb
make

The resulting Ruby extension that can be used as usual:
require 'fib'
p Fib.fib 10

Download
========
You can get rocaml at http://eigenclass.org/hiki/rocaml

License
=======
rocaml is distributed under the same terms as Ruby.

rocaml copyright (c) 2007 Mauricio Fernandez <[email protected]>
 
D

Daniel Berger

rocaml allows you to write Ruby extensions in Objective Caml.

<snip>

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.

What should I do?

Thanks,

Dan
 
D

Daniel Berger

<snip>

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.

What should I do?

I decided to grab camlp5 (http://pauillac.inria.fr/~ddr/camlp5/) and
try again. The extconf.rb script succeeds now. However, now the make
fails:

ocamldep.opt rubyOCamlUtil.ml > .depend
sh: ocamldep.opt: not found
*** Error code 1
make: Fatal error: Command failed for target `.depend'

I have ocamldep, but no ocamldep.opt. Where do I go from here?

Thanks,

Dan
 
M

Mauricio Fernandez

<snip>

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.

What should I do?

I see you solved this by installing camlp5, but I'd still like to fix the
camlp* selection code. What does camlp4 -v return for you?

$ camlp4 -v
Camlp4 version 3.10.0

camlp4 in 3.10.0 is incompatible with 3.09.2's (OTOH camlp5 *is* backwards
compatible).

The code that detects camlp5 is

have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?

For some reason it didn't work properly but I can't see why. Any idea?
 
D

Daniel Berger

I see you solved this by installing camlp5, but I'd still like to fix the
camlp* selection code. What does camlp4 -v return for you?

$ camlp4 -v
Camlp4 version 3.10.0

Same for me.
camlp4 in 3.10.0 is incompatible with 3.09.2's (OTOH camlp5 *is* backwards
compatible).

The code that detects camlp5 is

have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?

For some reason it didn't work properly but I can't see why. Any idea?

If camlp5 isn't found, you'll end up with nil. Changing 'empty?' to
'nil?' should do the trick, although people will still see a "sh:
camlp5: not found" echoed to stdout, which may be confusing.

Any ideas on the make failure?

Regards,

Dan
 
M

Mauricio Fernandez

rocaml allows you to write Ruby extensions in Objective Caml.
[...]
I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:

sh: camlp5: not found
*** extconf.rb failed ***

I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.
[...]
I decided to grab camlp5 (http://pauillac.inria.fr/~ddr/camlp5/) and
try again. The extconf.rb script succeeds now. However, now the make
fails:

ocamldep.opt rubyOCamlUtil.ml > .depend
sh: ocamldep.opt: not found
*** Error code 1
make: Fatal error: Command failed for target `.depend'

I have ocamldep, but no ocamldep.opt. Where do I go from here?

You didn't
make opt.opt
(or make world.opt) when you built ocaml-3.10.0, did you?
(BTW, on Debian the .opt binaries are in ocaml-native-compilers, in case
somebody is reading this)

For the time being, you can just install the native-code compilers or
hand-edit the generated Makefile; scroll down to

...
#############################################################################
# #
# Objective Caml #
# #
#############################################################################

OCAMLC = ocamlc.opt
OCAMLOPT = ocamlopt.opt
OCAMLDEP = ocamldep.opt
OFLAGS = -pp 'camlp5o -I . pa_rocaml.cmo'
OCAML_INCLUDES =
OCAML_LIBS = nums.cmxa
...

and remove the .opt extensions. I will add some code to detect whether the
opt binaries are available.

One last thing; what's your platform? rocaml extensions might or might not
work on AMD64 and OSX because their linkers aren't very smart (they can't
handle non-PIC code, it seems). According to OCaml's 3.10.0 changelog,
- Intel/AMD 64 bits: generate position-independent code by default.
but I don't know if that's enough or something special must be done at compile
time.

The next Objective Caml release will feature dynamic loading of native code,
so this won't remain a problem for long.
Meanwhile, static extensions can be used.
 
M

Mauricio Fernandez

I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:
sh: camlp5: not found
*** extconf.rb failed ***
I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.
What should I do?

I see you solved this by installing camlp5, but I'd still like to fix the
camlp* selection code. What does camlp4 -v return for you? [...]
The code that detects camlp5 is

have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?

For some reason it didn't work properly but I can't see why. Any idea?

If camlp5 isn't found, you'll end up with nil. Changing 'empty?' to
'nil?' should do the trick, although people will still see a "sh:
camlp5: not found" echoed to stdout, which may be confusing.

That's what 2>&1 is meant to prevent, but you're very right, it should be
nil?. Now I wonder why you didn't get a NoMethodError.
Any ideas on the make failure?

Yes, the native code compilers are missing, see my other message for a couple
solutions. There was another bug in the code that detects whether they are
present; I have pushed the fix to the repository at

http://eigenclass.org/repos/rocaml/head/

and it will be in the next tarball, 0.6.1, to follow shortly.


Wed Oct 17 23:29:10 CEST 2007 Mauricio Fernandez <[email protected]>
* rocaml_extconf.rb: fixed native compiler detection.
Wed Oct 17 23:21:08 CEST 2007 Mauricio Fernandez <[email protected]>
* rocaml_extconf.rb: fix camlp[45] detection.
diff -rN -u old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-17 23:29:42.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-17 23:29:42.000000000 +0200
@@ -43,7 +43,7 @@
exit
end

-maybe_opt = lambda{|x| opt = "#{x}.opt"; system(x) ? opt : x }
+maybe_opt = lambda{|x| opt = "#{x}.opt"; system(opt) ? opt : x }

if OCAML_PACKAGES.empty? then
OCAMLC = maybe_opt["ocamlc"]
@@ -91,9 +91,9 @@

# determine whether camlp4 (or camlp5) can be used:

-have_camlp5 = ! `camlp5 -v 2>&1`["version"].empty?
+have_camlp5 = ! `camlp5 -v 2>&1`["version"].nil?
camlp4version = `camlp4 -v 2>&1`[/version\s+(\d.*)/, 1]
-have_camlp4 = ! camlp4version.empty?
+have_camlp4 = ! camlp4version.nil?

pa_rocaml_revdeps = Dir["*.ml"].map do |f|
"#{f.sub(/\.ml$/, ".cmx")}: pa_rocaml.cmo"
 
D

Daniel Berger

rocaml allows you to write Ruby extensions in Objective Caml. [...]
I grabbed the latest and greatest Ocaml tarball and installed it.
However, when I try to install rocaml I get this:
sh: camlp5: not found
*** extconf.rb failed ***
I have no camlp5 in /usr/local/bin. I have camlp4, however, and the
comments in rocaml_extconf.rb suggest that either should suffice.
[...]
I decided to grab camlp5 (http://pauillac.inria.fr/~ddr/camlp5/) and
try again. The extconf.rb script succeeds now. However, now the make
fails:
ocamldep.opt rubyOCamlUtil.ml > .depend
sh: ocamldep.opt: not found
*** Error code 1
make: Fatal error: Command failed for target `.depend'
I have ocamldep, but no ocamldep.opt. Where do I go from here?

You didn't
make opt.opt
(or make world.opt) when you built ocaml-3.10.0, did you?

Ok, I did make world.opt and reinstalled. Then I rebuilt camlp5 with
"transitional". Now it seems I'm missing some header files:
ocamldep.opt rubyOCamlUtil.ml > .depend
cc -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -KPIC -g -c foo_rocaml_wrapper.c
"foo_rocaml_wrapper.c", line 14: cannot find include file: <caml/
mlvalues.h>
"foo_rocaml_wrapper.c", line 15: cannot find include file: <caml/
callback.h>
"foo_rocaml_wrapper.c", line 16: cannot find include file: <caml/
memory.h>
"foo_rocaml_wrapper.c", line 17: cannot find include file: <caml/
alloc.h>
"foo_rocaml_wrapper.c", line 18: cannot find include file: <caml/
fail.h>
...

I don't think OCaml's installation instructions could be any more
confusing. What option did I miss?

Regards,

Dan
 
M

Mauricio Fernandez

You didn't
make opt.opt
(or make world.opt) when you built ocaml-3.10.0, did you?

Ok, I did make world.opt and reinstalled. Then I rebuilt camlp5 with
"transitional". Now it seems I'm missing some header files:
ocamldep.opt rubyOCamlUtil.ml > .depend
cc -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -KPIC -g -c foo_rocaml_wrapper.c
"foo_rocaml_wrapper.c", line 14: cannot find include file: <caml/
mlvalues.h> [...]

I don't think OCaml's installation instructions could be any more
confusing. What option did I miss?

I'm sorry for all the issues and appreciate your patience. The basic problem
is that I'm using Debian and everything Just Works(TM) with its OCaml
packages, so it is harder for me to anticipate problems like those you ran
into. I see that you're using Solaris, which adds further uncertainty because
I'm not sure its linker can build shared objects with non-PIC code [1]. Some
magic incantations in the form of $LDFLAGS might be required. You're stepping
on new ground :)

The missing headers should be in /usr/local/lib/ocaml/3.10.0/caml (or maybe
/usr/local/lib/ocaml/caml; in my system I have a symlink from
/usr/include/caml to /usr/lib/ocaml/3.10.0/caml, which is why this never
happened to me). The extension should build correctly after symlinking or
applying this one-line patch:


Thu Oct 18 19:14:58 CEST 2007 Mauricio Fernandez <[email protected]>
* rocaml_extconf.rb: add ocaml_native_lib_path to the INCFLAGS (-Idir).
diff -rN -u -w old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-18 19:25:49.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-18 19:25:49.000000000 +0200
@@ -43,6 +43,8 @@
exit
end

+$INCFLAGS << " -I#{ocaml_native_lib_path}"
+
maybe_opt = lambda{|x| opt = "#{x}.opt"; system(opt) ? opt : x }

if OCAML_PACKAGES.empty? then


Note that the extension you're building ("foo") isn't complete. It includes
the wrappers but not the OCaml implementations of the corresponding functions;
in fact, it is only meant to serve as the extconf.rb template for new
extensions.

You can find the actual examples under examples/:
* marshal: 3-line specialized marshallers that can be over 3 times faster than
Ruby's Marshal (the largest speedup is achieved with float arrays)
* tree: a 30-line RB tree with 3X faster lookup than RBTree
(you'll need rbtree if you want to run the benchmarks in test_tree.rb)
* oo, records, variants: show how abstract types, records and variants are
converted between Ruby and OCaml. Abstract types become objects, records
turn into hashes with symbol keys and variants are mapped to symbols or
arrays.

Thank you,

[1] According to http://gcc.gnu.org/ml/gcc/1999-05n/msg00376.html

SunOS and Solaris will quite happily build .so from non-PIC objects - they
just don't "share" very well as the relocations cause copy-on-write and hence
private pages.

So there's a good chance it will work.
 
D

Daniel Berger

Ok, I did make world.opt and reinstalled. Then I rebuilt camlp5 with
"transitional". Now it seems I'm missing some header files:
ocamldep.opt rubyOCamlUtil.ml > .depend
cc -I. -I/usr/local/lib/ruby/1.8/sparc-solaris2.10 -I/usr/local/lib/
ruby/1.8/sparc-solaris2.10 -I. -KPIC -g -c foo_rocaml_wrapper.c
"foo_rocaml_wrapper.c", line 14: cannot find include file: <caml/
mlvalues.h> [...]

I don't think OCaml's installation instructions could be any more
confusing. What option did I miss?

I'm sorry for all the issues and appreciate your patience. The basic problem
is that I'm using Debian and everything Just Works(TM) with its OCaml
packages, so it is harder for me to anticipate problems like those you ran
into. I see that you're using Solaris, which adds further uncertainty because
I'm not sure its linker can build shared objects with non-PIC code [1]. Some
magic incantations in the form of $LDFLAGS might be required. You're stepping
on new ground :)

It would seem so. Your one line patch to rocaml_extconf.rb worked
(thanks!), but now I get this:

irb(main):002:0> require 'foo'
LoadError: ld.so.1: ruby: fatal: relocation error: file /export/home/
djberge/src/ruby/rocaml-0.6.1/foo.so: symbol __muldi3: referenced
symbol not found - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
from (irb):2

This is Sun Studio 12 on Sparc/Solaris 10, btw.

I already tried installing a gcc add-on found here: http://cooltools.sunsource.net/gcc/

No joy. Any ideas?

Oh, and these warnings showed up during the build. Dunno if you're
interested:

"foo_rocaml_wrapper.c", line 61: warning: statement not reached
"foo_rocaml_wrapper.c", line 68: warning: statement not reached
"foo_rocaml_wrapper.c", line 338: warning: statement not reached
"foo_rocaml_wrapper.c", line 388: warning: statement not reached
"foo_rocaml_wrapper.c", line 438: warning: statement not reached
"foo_rocaml_wrapper.c", line 485: warning: statement not reached
"foo_rocaml_wrapper.c", line 532: warning: statement not reached
"foo_rocaml_wrapper.c", line 582: warning: statement not reached
"foo_rocaml_wrapper.c", line 632: warning: statement not reached
"foo_rocaml_wrapper.c", line 681: warning: statement not reached
"foo_rocaml_wrapper.c", line 730: warning: statement not reached
"foo_rocaml_wrapper.c", line 779: warning: statement not reached
"foo_rocaml_wrapper.c", line 829: warning: statement not reached

Regards,

Dan
 
D

Daniel Berger

<snip>

Some more information that may or may not be useful:

jberge-/export/home/djberge/src/ruby/rocaml-0.6.1-697>ldd foo.so
librt.so.1 => /lib/librt.so.1
libpthread.so.1 => /lib/libpthread.so.1
libdl.so.1 => /lib/libdl.so.1
libcrypt_i.so.1 => /usr/lib/libcrypt_i.so.1
libm.so.2 => /lib/libm.so.2
libc.so.1 => /lib/libc.so.1
libaio.so.1 => /lib/libaio.so.1
libmd5.so.1 => /lib/libmd5.so.1
libgen.so.1 => /lib/libgen.so.1
/platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1
/platform/SUNW,Sun-Blade-100/lib/libmd5_psr.so.1

Regards,

Dan
 
M

Mauricio Fernandez

I'm sorry for all the issues and appreciate your patience. The basic problem
is that I'm using Debian and everything Just Works(TM) with its OCaml
packages, so it is harder for me to anticipate problems like those you ran
into. I see that you're using Solaris, which adds further uncertainty because
I'm not sure its linker can build shared objects with non-PIC code [1]. Some
magic incantations in the form of $LDFLAGS might be required. You're stepping
on new ground :)

It would seem so. Your one line patch to rocaml_extconf.rb worked
(thanks!), but now I get this:

irb(main):002:0> require 'foo'
LoadError: ld.so.1: ruby: fatal: relocation error: file /export/home/
djberge/src/ruby/rocaml-0.6.1/foo.so: symbol __muldi3: referenced
symbol not found - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
from (irb):2

It seems we're almost there (we went through the critical non-PIC code
relocation phase :)

According to google, __muldi3 is an internal GCC library function. Do you have
a native linker in addition to GCC's? GCC's will automatically link against
libgcc, but it seems that's not the case with the native one.

At any rate, linking against either libgcc.a or libgcc_s.so.1 should work;
something like this maybe?


diff -rN -u -w old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
@@ -89,7 +89,9 @@

# needed by mkmf's create_makefile
$LOCAL_LIBS = "#{CAML_TARGET} #{ocaml_native_lib_path}/libasmrun.a #{extra_caml_libs.join(" ")}"
-
+# try to add GCC's libgcc, required on Sparc
+libgcc = Dir["/lib/libgcc*"].first # maybe some other search path(s)
+$LOCAL_LIBS << " " << libgcc if libgcc

# determine whether camlp4 (or camlp5) can be used:


You might have to change the search path. On my platform, libgcc can also be
found in /usr/lib/gcc/i486-linux-gnu/<gcc version>. Linking against either
libgcc.a or libgcc_s.so should do.

I haven't pushed the patch yet; want to know if /lib/libgcc* is OK or if the
path needs to be changed. Also, can I rely on say
require 'rbconfig'
libgcc_needed = Config::CONFIG["arch"] =~ "sparc"
?
Oh, and these warnings showed up during the build. Dunno if you're
interested:

"foo_rocaml_wrapper.c", line 61: warning: statement not reached
"foo_rocaml_wrapper.c", line 68: warning: statement not reached
"foo_rocaml_wrapper.c", line 338: warning: statement not reached

These are all things like

rb_raise(klass, "%s", s);
return Qnil; /* not reached */

or

if(exception == Qnil && !status) {
return ret;
} else if(status) { /* exception in Ruby -> caml conversions */
rb_jump_tag(status);
} else { /* OCaml exception*/
rb_raise(rb_eStandardError, StringValuePtr(exception));
}

return Qnil; /* never reached */

It seems your GCC is smarter than mine and knows that rb_raise, rb_jump_tag,
etc. do not return. Know any macros in ruby.h to handle such things?
They are harmless and can be safely ignored anyway.
 
D

Daniel Berger

It seems we're almost there (we went through the critical non-PIC code
relocation phase :)

According to google, __muldi3 is an internal GCC library function. Do you have
a native linker in addition to GCC's? GCC's will automatically link against
libgcc, but it seems that's not the case with the native one.

Did I mention that I built Ruby with Sun Studio 12, but OCaml with gcc
4? Ocaml seemed to demand it. :|
At any rate, linking against either libgcc.a or libgcc_s.so.1 should work;
something like this maybe?

diff -rN -u -w old-rocaml/rocaml_extconf.rb new-rocaml/rocaml_extconf.rb
--- old-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
+++ new-rocaml/rocaml_extconf.rb 2007-10-18 21:03:41.000000000 +0200
@@ -89,7 +89,9 @@

# needed by mkmf's create_makefile
$LOCAL_LIBS = "#{CAML_TARGET} #{ocaml_native_lib_path}/libasmrun.a #{extra_caml_libs.join(" ")}"
-
+# try to add GCC's libgcc, required on Sparc
+libgcc = Dir["/lib/libgcc*"].first # maybe some other search path(s)
+$LOCAL_LIBS << " " << libgcc if libgcc

# determine whether camlp4 (or camlp5) can be used:

You might have to change the search path. On my platform, libgcc can also be
found in /usr/lib/gcc/i486-linux-gnu/<gcc version>. Linking against either
libgcc.a or libgcc_s.so should do.

In my particular case it's /opt/csw/gcc4/lib/libgcc which I added
directly:

djberge-/opt/csw/gcc4/lib-663>ll libg*
lrwxrwxrwx 1 root root 15 Dec 12 2006 libgcc_s.so -> ./
libgcc_s.so.1
-rwxr-xr-x 1 root bin 63928 Oct 10 2005 libgcc_s.so.1

It builds correctly. But then I get a different linker error:

LoadError: ld.so.1: ruby: fatal: libgcc_s.so.1: open failed: No such
file or directory - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
from (irb):2

LDD led me to the fix:

djberge-/export/home/djberge/src/ruby/rocaml-0.6.1-546>ldd foo.so
libgcc_s.so.1 => (file not found)
librt.so.1 => /lib/librt.so.1
libpthread.so.1 => /lib/libpthread.so.1
libdl.so.1 => /lib/libdl.so.1
libcrypt_i.so.1 => /usr/lib/libcrypt_i.so.1
libm.so.2 => /lib/libm.so.2
libc.so.1 => /lib/libc.so.1
libaio.so.1 => /lib/libaio.so.1
libmd5.so.1 => /lib/libmd5.so.1
libgen.so.1 => /lib/libgen.so.1
/platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1
/platform/SUNW,Sun-Blade-100/lib/libmd5_psr.so.1

So, I slapped $LDFLAGS += " -L/opt/csw/gcc4/lib -R/opt/csw/gcc4/lib"
into the rocaml_extconf.rb file. Looks better after that:

djberge-/export/home/djberge/src/ruby/rocaml-0.6.1-605>ldd foo.so
libgcc_s.so.1 => /opt/csw/gcc4/lib/libgcc_s.so.1
librt.so.1 => /lib/librt.so.1
libpthread.so.1 => /lib/libpthread.so.1
libdl.so.1 => /lib/libdl.so.1
libcrypt_i.so.1 => /usr/lib/libcrypt_i.so.1
libm.so.2 => /lib/libm.so.2
libc.so.1 => /lib/libc.so.1
libaio.so.1 => /lib/libaio.so.1
libmd5.so.1 => /lib/libmd5.so.1
libgen.so.1 => /lib/libgen.so.1
/platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1
/platform/SUNW,Sun-Blade-100/lib/libmd5_psr.so.1

Back to irb:

irb(main):002:0> require 'foo'
NameError: (eval): uninitialized constant Kernel::Some
from (eval)
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
from (irb):2

Hu...what? WTF? Never seen that one before. I dug through /usr/local/
lib/ruby to make sure there wasn't some other "foo" it was picking up
- there isn't.

Any ideas?

Thanks,

Dan
 
M

Mauricio Fernandez

It seems we're almost there (we went through the critical non-PIC code
relocation phase :)

According to google, __muldi3 is an internal GCC library function. Do you have
a native linker in addition to GCC's? GCC's will automatically link against
libgcc, but it seems that's not the case with the native one.

Did I mention that I built Ruby with Sun Studio 12, but OCaml with gcc
4? Ocaml seemed to demand it. :|
At any rate, linking against either libgcc.a or libgcc_s.so.1 should work;
something like this maybe?
[...]
It builds correctly. But then I get a different linker error:

LoadError: ld.so.1: ruby: fatal: libgcc_s.so.1: open failed: No such
file or directory - /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /export/home/djberge/src/ruby/rocaml-0.6.1/foo.so
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/
custom_require.rb:27:in `require'
from (irb):2 [...]
So, I slapped $LDFLAGS += " -L/opt/csw/gcc4/lib -R/opt/csw/gcc4/lib"
into the rocaml_extconf.rb file. Looks better after that: [...]
Back to irb:

irb(main):002:0> require 'foo'
NameError: (eval): uninitialized constant Kernel::Some [...]
Hu...what? WTF? Never seen that one before. I dug through /usr/local/
lib/ruby to make sure there wasn't some other "foo" it was picking up
- there isn't.

Any ideas?

Congrats, it seems you got it to work :)

This is what you missed in a previous message:

Note that the extension you're building ("foo") isn't complete. It includes
the wrappers but not the OCaml implementations of the corresponding functions;
in fact, it is only meant to serve as the extconf.rb template for new
extensions.

You can find the actual examples under examples/:
* marshal: 3-line specialized marshallers that can be over 3 times faster than
Ruby's Marshal (the largest speedup is achieved with float arrays)
* tree: a 30-line RB tree with 3X faster lookup than RBTree
(you'll need rbtree if you want to run the benchmarks in test_tree.rb)
* oo, records, variants: show how abstract types, records and variants are
converted between Ruby and OCaml. Abstract types become objects, records
turn into hashes with symbol keys and variants are mapped to symbols or
arrays.


BTW, is the /opt/csw/gcc4/lib (de facto) "standard" in any way?
I'd like to have as much as possible work out of the box (trying to detect
common cases) and offer a convenient way to specify the things that cannot be
inferred. Could you give me the output of
$ ruby -v -rrbconfig -rpp -e "pp Config::CONFIG"
?

Thanks,
 
D

Daniel Berger

Congrats, it seems you got it to work :)

This is what you missed in a previous message:

Note that the extension you're building ("foo") isn't complete. It includes
the wrappers but not the OCaml implementations of the corresponding functions;
in fact, it is only meant to serve as the extconf.rb template for new
extensions.

You can find the actual examples under examples/:
* marshal: 3-line specialized marshallers that can be over 3 times faster than
Ruby's Marshal (the largest speedup is achieved with float arrays)
* tree: a 30-line RB tree with 3X faster lookup than RBTree
(you'll need rbtree if you want to run the benchmarks in test_tree.rb)
* oo, records, variants: show how abstract types, records and variants are
converted between Ruby and OCaml. Abstract types become objects, records
turn into hashes with symbol keys and variants are mapped to symbols or
arrays.

Oh, heh, whoops. :)
BTW, is the /opt/csw/gcc4/lib (de facto) "standard" in any way?
I'd like to have as much as possible work out of the box (trying to detect
common cases) and offer a convenient way to specify the things that cannot be
inferred.

No, I'm afraid not. The "/opt/csw" prefix is the standard blastwave
package directory (as in, Solaris packages installed from
blastwave.org). I think you'll have to setup a "--with-gcc-dir" option
or something.
Could you give me the output of
$ ruby -v -rrbconfig -rpp -e "pp Config::CONFIG"

Sure, here it is:
ruby -v -rrbconfig -rpp -e "pp Config::CONFIG"
ruby 1.8.6 (2007-09-23 patchlevel 110) [sparc-solaris2.10]
{"sitedir"=>"/usr/local/lib/ruby/site_ruby",
"MAKEFILES"=>"Makefile",
"LIBRUBY"=>"libruby-static.a",
"MAKEDIRS"=>"mkdir -p",
"PACKAGE_VERSION"=>"",
"GREP"=>"/usr/sfw/bin/ggrep",
"prefix"=>"/usr/local",
"COMMON_LIBS"=>"",
"rubylibdir"=>"/usr/local/lib/ruby/1.8",
"target"=>"sparc-sun-solaris2.10",
"DLLWRAP"=>"",
"AR"=>"ar",
"target_alias"=>"",
"MANTYPE"=>"man",
"docdir"=>"/usr/local/share/doc/$(PACKAGE)",
"RDOCTARGET"=>"",
"dvidir"=>"/usr/local/share/doc/$(PACKAGE)",
"AS"=>"as",
"GNU_LD"=>"no",
"MAINLIBS"=>"",
"WINDRES"=>"",
"XCFLAGS"=>" -DRUBY_EXPORT",
"datarootdir"=>"/usr/local/share",
"RUBY_INSTALL_NAME"=>"ruby",
"LN_S"=>"ln -s",
"archdir"=>"/usr/local/lib/ruby/1.8/sparc-solaris2.10",
"LINK_SO"=>"",
"MINIRUBY"=>"./miniruby",
"DLDLIBS"=>" -lc",
"EXEEXT"=>"",
"target_vendor"=>"sun",
"RUNRUBY"=>"./miniruby $(srcdir)/runruby.rb --extout=.ext --",
"DESTDIR"=>"",
"sbindir"=>"/usr/local/sbin",
"LIBPATHENV"=>"LD_LIBRARY_PATH",
"sitearch"=>"sparc-solaris2.10",
"psdir"=>"/usr/local/share/doc/$(PACKAGE)",
"host_cpu"=>"sparc",
"DLEXT2"=>"",
"LIBS"=>"-lrt -lpthread -ldl -lcrypt -lm ",
"localedir"=>"/usr/local/share/locale",
"rubyw_install_name"=>"",
"EXTOUT"=>".ext",
"ECHO_C"=>"",
"OBJDUMP"=>"",
"arch"=>"sparc-solaris2.10",
"MAJOR"=>"1",
"ruby_version"=>"1.8",
"CC"=>"cc",
"EGREP"=>"/usr/sfw/bin/ggrep -E",
"COMMON_HEADERS"=>"",
"COMMON_MACROS"=>"",
"PACKAGE_TARNAME"=>"",
"build_cpu"=>"sparc",
"build_vendor"=>"sun",
"host_alias"=>"",
"RANLIB"=>"ranlib",
"LDSHARED"=>"ld -G",
"LIBRUBYARG_SHARED"=>"-R /usr/local/lib -L/usr/local/lib ",
"YFLAGS"=>"",
"htmldir"=>"/usr/local/share/doc/$(PACKAGE)",
"MINOR"=>"8",
"INSTALL_SCRIPT"=>"/opt/csw/bin/ginstall -c",
"EXPORT_PREFIX"=>"",
"LIBRUBY_ALIASES"=>"libruby.so",
"LDFLAGS"=>"-L. ",
"datadir"=>"/usr/local/share",
"NM"=>"",
"includedir"=>"/usr/local/include",
"infodir"=>"/usr/local/share/info",
"host_os"=>"solaris2.10",
"build"=>"sparc-sun-solaris2.10",
"host"=>"sparc-sun-solaris2.10",
"INSTALL_DATA"=>"/opt/csw/bin/ginstall -c -m 644",
"build_os"=>"solaris2.10",
"DLDFLAGS"=>"",
"ruby_install_name"=>"ruby",
"DLEXT"=>"so",
"LIBRUBY_SO"=>"libruby.so.1.8.6",
"TEENY"=>"6",
"CPP"=>"cc -E",
"ALLOCA"=>"",
"sysconfdir"=>"/usr/local/etc",
"exec_prefix"=>"/usr/local",
"PATH_SEPARATOR"=>":",
"LIBEXT"=>"a",
"mandir"=>"/usr/local/share/man",
"libdir"=>"/usr/local/lib",
"build_alias"=>"",
"target_cpu"=>"sparc",
"ECHO_N"=>"-n",
"sharedstatedir"=>"/usr/local/com",
"YACC"=>"bison -y",
"configure_args"=>" '--enable-pthread' 'CC=cc'",
"SOLIBS"=>"",
"host_vendor"=>"sun",
"TRY_LINK"=>"",
"PACKAGE_STRING"=>"",
"target_os"=>"solaris2.10",
"oldincludedir"=>"/usr/include",
"CP"=>"cp",
"SET_MAKE"=>"",
"LIBRUBYARG_STATIC"=>"-lruby-static",
"LIBRUBYARG"=>"-lruby-static",
"RUBYW_INSTALL_NAME"=>"",
"PACKAGE_NAME"=>"",
"pdfdir"=>"/usr/local/share/doc/$(PACKAGE)",
"EXTSTATIC"=>"",
"ECHO_T"=>"",
"RPATHFLAG"=>" -R'%1$-s'",
"SHELL"=>"/bin/bash",
"STATIC"=>"",
"ASFLAGS"=>"",
"INSTALL"=>"/opt/csw/bin/ginstall -c",
"sitearchdir"=>"/usr/local/lib/ruby/site_ruby/1.8/sparc-solaris2.10",
"STRIP"=>"strip",
"ARCHFILE"=>"",
"LIBRUBY_DLDFLAGS"=>"",
"OBJEXT"=>"o",
"XLDFLAGS"=>"",
"LIBRUBY_LDSHARED"=>"ld -G",
"ENABLE_SHARED"=>"no",
"RM"=>"rm -f",
"CCDLFLAGS"=>" -KPIC",
"setup"=>"Setup",
"CPPOUTFILE"=>"-o conftest.i",
"topdir"=>"/usr/local/lib/ruby/1.8/sparc-solaris2.10",
"RUBY_SO_NAME"=>"ruby",
"CFLAGS"=>"-g",
"localstatedir"=>"/usr/local/var",
"LIBPATHFLAG"=>" -L'%1$-s'",
"bindir"=>"/usr/local/bin",
"sitelibdir"=>"/usr/local/lib/ruby/site_ruby/1.8",
"NROFF"=>"/usr/bin/nroff",
"CPPFLAGS"=>"",
"INSTALL_PROGRAM"=>"/opt/csw/bin/ginstall -c",
"PACKAGE_BUGREPORT"=>"",
"libexecdir"=>"/usr/local/libexec",
"OUTFLAG"=>"-o ",
"LIBRUBY_A"=>"libruby-static.a",
"PREP"=>"miniruby",
"ARCH_FLAG"=>""}

BTW, I guess MS Windows is out of the question then, unless I want to
install Cygwin, correct?

Many thanks,

Dan
 
M

Mauricio Fernandez

Congrats, it seems you got it to work :) [...]
BTW, is the /opt/csw/gcc4/lib (de facto) "standard" in any way?
I'd like to have as much as possible work out of the box (trying to detect
common cases) and offer a convenient way to specify the things that cannot be
inferred.

No, I'm afraid not. The "/opt/csw" prefix is the standard blastwave
package directory (as in, Solaris packages installed from
blastwave.org). I think you'll have to setup a "--with-gcc-dir" option
or something.

Yes, that seems the most reasonable way.

[...]
BTW, I guess MS Windows is out of the question then, unless I want to
install Cygwin, correct?

I used to think so, but I've been googling a bit and it turns out that what I
thought would be the major problem (non-PIC code relocation in a shared
object) just can't happen: "On Win32, the standard executable format is
non-PIC"(!!) [1].

So it seems it is indeed possible to get rocaml to work on Win32. Quoting from
OCaml's README.win32:

There are no less than four ports of Objective Caml for MS Windows available:
- a native Win32 port, built with the Microsoft development tools;
- a native Win32 port, built with the Cygwin/MinGW development tools;
- a port consisting of the Unix sources compiled under the Cygwin
Unix-like environment for Windows;
- a native Win64 port (64-bit Windows), built with the Microsoft
development tools.

Building Ruby extensions on Windows has always been a PITA, since you have to
use the same compiler that was used to build Ruby itself. Fortunately, the
binary used by the One Click Installer is compatible with extensions built
with MinGW, so the MinGW ocaml build should work, as should the Cygwin one
along with Cygwin's ruby. And it might even be possible to get it to work with
MS's tools assuming Ruby itself can be compiled cleanly.

Of course, I'll have to add some options to make it possible to specify where
ocaml is installed, since the auto-detection I'm doing will obviously fail:

[rocaml_extconf.rb]
...
ocaml_native_lib_path = %w[
/usr/lib/ocaml/**/libasmrun.a
/usr/local/lib/ocaml/**/libasmrun.a
].map{|glob| Dir[glob]}.flatten.sort.map{|x| File.dirname(x)}.last
...

Some further LDFLAGS and/or CFLAGS magic might be needed, as was the case in
Solaris, and there might be some un*ix-isms in rocaml_extconf.rb here and
there, but it seems it can be made to work. I don't have access to a Windows
box so it all depends on people willing to explore new ways, though ;)

[1] https://www.gelato.unsw.edu.au/archives/comp-arch/2006-December/005669.html
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,054
Latest member
TrimKetoBoost

Latest Threads

Top