Compiling SpiderMonkey (libjs) on Mac OS X (for use with Python ctypes)

Discussion in 'Python' started by follower, Apr 28, 2006.

  1. follower

    follower Guest

    This post is mostly Google-bait for anyone else that might want to
    compile SpiderMonkey ( libjs / / libjs.dylib ) for OS X
    (10.4.5 in my case) and then use it with Python's ctypes. I can't say
    this will work for anyone else, but it worked for me...


    First up:

    $ tar xzvf js-1.5.tar.gz
    $ cd js/src

    Then, the broken way (so, don't do this if you want ctypes to work :)

    $ make -f Makefile.ref


    $ file Darwin_DBG.OBJ/
    Darwin_DBG.OBJ/ current ar archive

    $ python
    >>> from ctypes import *
    >>> cdll.load("Darwin_DBG.OBJ/")

    OSError: dlcompat: Can not open "Darwin_DBG.OBJ/"

    Now, the unbroken way (after a 'make -f Makefile.ref clean' if need

    $ export XCFLAGS="-fno-common";export XMKSHLIBOPTS="-dynamic
    -undefined suppress -flat_namespace"; export
    OTHER_LIBS="-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0/ -lgcc"

    $ make -f Makefile.ref # Hmmm, this only seems to work after
    a failed attempt with `make -f Makefile.ref` on its own...


    $ file Darwin_DBG.OBJ/
    Darwin_DBG.OBJ/ Mach-O dynamically linked shared library ppc

    $ python
    >>> from ctypes import *
    >>> cdll.load("Darwin_DBG.OBJ/")

    <CDLL 'Darwin_DBG.OBJ/', handle 1008000 at 7c310>

    Example use:
    >>> libjs = cdll.load("Darwin_DBG.OBJ/")
    >>> rt = libjs.JS_Init(0x100000)
    >>> cx = libjs.JS_NewContext(rt, 0x1000);


    A small bit of explanation as I understand things:

    === export XCFLAGS="-fno-common" ===

    This is required to avoid an issue demonstrated here:

    $ libtool -dynamic -o Darwin_DBG.OBJ/libjs.dylib
    Darwin_DBG.OBJ/libjs.a -lSystem
    ld: for architecture ppc
    ld: common symbols not allowed with MH_DYLIB output format with the
    -multi_module option
    Darwin_DBG.OBJ/libjs.a(jsatom.o) definition of common
    _js_atom_map_count (size 4)
    Darwin_DBG.OBJ/libjs.a(jsatom.o) definition of common
    _js_atom_map_hash_table_count (size 4)
    libtool: internal link edit command failed

    This method of conversion was discussed here:
    but I think I first ran into the error by including the '-dynamic' flag

    The use of the '-fno-common' flag is also described here

    "The generation of position-independent code (PIC) is the default on
    Darwin. Actually, PowerPC code is position-independent by design,
    there is no performance or space penalty involved. So, you don't
    to specify a PIC option when compiling code for a shared library or
    module. However, the linker doesn't allow "common" symbols in
    libraries, so you must use the -fno-common compiler option."

    === export XMKSHLIBOPTS="-dynamic -undefined suppress -flat_namespace"

    The '-dynamic' flag makes the library in the Mach-O format ctypes on
    Mac OS X understands.

    Without the '-undefined suppress' flag I get this:

    ld: for architecture ppc
    ld: Undefined symbols:
    libtool: internal link edit command failed

    Without the '-flat_namespace' flag I get this (and I remembered reading
    about the flat namespace flag somewhere... :) ):

    ld: for architecture ppc
    ld: -undefined error or -undefined define_a_way must be used when
    -twolevel_namespace is in effect
    libtool: internal link edit command failed

    But some references
    <> say this is
    a bad idea:

    "In any case, two-level namespaces are highly recommended, as flat
    namespaces do NOT behave identially to the flat namespaces of other
    popular linkers. Most importantly: the first definition of a symbol
    used, rather than the last, which can cause serious problems with
    plugin loading."

    === export OTHER_LIBS="-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0/
    -lgcc" ===

    This is required to avoid:

    ld: for architecture ppc
    ld: Darwin_DBG.OBJ/jsdate.o has external relocation entries in
    non-writable section (__TEXT,__text) for symbols:
    libtool: internal link edit command failed

    According to some references
    this is related to having multiple versions of gcc (e.g. Apple and
    Fink) installed.

    My "solution" is somewhat cargo-culted so I wouldn't be surprised to
    learn something in my approach is ill-advised. :) But, it's working
    for me for the moment so it's at least a start.

    The best reference I've found (after a *lot* of searching) for this
    issue was <> which
    includes the following:

    2.1 Shared Libraries vs. Loadable Modules

    One Mach-O feature that hits many people by surprise is the strict
    distinction between shared libraries and dynamically loadable modules.
    On ELF systems both are the same; any piece of shared code can be used
    as a library and for dynamic loading. Use otool -hv some_file to see
    the filetype of some_file.

    Mach-O shared libraries have the file type MH_DYLIB and carry the
    extension .dylib. They can be linked against with the usual static
    linker flags, e.g. -lfoo for libfoo.dylib. However, they can not be
    loaded as a module. (Side note: Shared libraries can be loaded
    dynamically through an API. However, that API is different from the API
    for bundles and the semantics make it useless for an dlopen()
    emulation. Most notably, shared libraries can not be unloaded.)

    Loadable modules are called "bundles" in Mach-O speak. They have the
    file type MH_BUNDLE. Since no component involved cares about it, they
    can carry any extension. The extension .bundle is recommended by Apple,
    but most ported software uses .so for the sake of compatibility.
    Bundles can be dynamically loaded and unloaded via dyld APIs, and there
    is a wrapper that emulates dlopen() on top of that API. It is not
    possible to link against bundles as if they were shared libraries.
    However, it is possible that a bundle is linked against real shared
    libraries; those will be loaded automatically when the bundle is

    This suggests I still may be getting the two confused, so feel free to
    correct me. :)

    Further comments on libtool, Mac OS X and porting are found here:


    Anyway, hopefully this will be of use to someone.

    Now to stop shaving yaks and get back to what I was *really* working
    on... :)

    follower, Apr 28, 2006
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Henk Punt
    Henk Punt
    Jul 23, 2004
  2. zowtar


    zowtar, Sep 5, 2007, in forum: Python
    Sep 5, 2007
  3. Replies:
  4. elca
  5. Trans

    SpiderMonkey / OSSP js?

    Trans, Aug 20, 2006, in forum: Ruby
    Jeremy Tregunna
    Aug 24, 2006

Share This Page