F
follower
This post is mostly Google-bait for anyone else that might want to
compile SpiderMonkey ( libjs / libjs.so / libjs.dylib ) for OS X
(10.4.5 in my case) and then use it with Python's ctypes. I can't say
this will work for anyone else, but it worked for me...
Using:
http://ftp.mozilla.org/pub/mozilla.org/js/js-1.5.tar.gz
First up:
$ tar xzvf js-1.5.tar.gz
$ cd js/src
Then, the broken way (so, don't do this if you want ctypes to work
):
$ make -f Makefile.ref
...
$ file Darwin_DBG.OBJ/libjs.so
Darwin_DBG.OBJ/libjs.so: current ar archive
$ python OSError: dlcompat: Can not open "Darwin_DBG.OBJ/libjs.so"
Now, the unbroken way (after a 'make -f Makefile.ref clean' if need
be):
$ export XCFLAGS="-fno-common";export XMKSHLIBOPTS="-dynamic
-undefined suppress -flat_namespace"; export
OTHER_LIBS="-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0/ -lgcc"
$ make -f Makefile.ref libjs.so # Hmmm, this only seems to work after
a failed attempt with `make -f Makefile.ref` on its own...
...
$ file Darwin_DBG.OBJ/libjs.so
Darwin_DBG.OBJ/libjs.so: Mach-O dynamically linked shared library ppc
$ python <CDLL 'Darwin_DBG.OBJ/libjs.so', handle 1008000 at 7c310>
Example use: ...
A small bit of explanation as I understand things:
=== export XCFLAGS="-fno-common" ===
This is required to avoid an issue demonstrated here:
$ libtool -dynamic -o Darwin_DBG.OBJ/libjs.dylib
Darwin_DBG.OBJ/libjs.a -lSystem
ld: for architecture ppc
ld: common symbols not allowed with MH_DYLIB output format with the
-multi_module option
Darwin_DBG.OBJ/libjs.a(jsatom.o) definition of common
_js_atom_map_count (size 4)
Darwin_DBG.OBJ/libjs.a(jsatom.o) definition of common
_js_atom_map_hash_table_count (size 4)
libtool: internal link edit command failed
This method of conversion was discussed here:
<http://groups.google.com/group/comp.sys.mac.programmer.help/msg/f3b4ad292965fe94?hl=en&>
but I think I first ran into the error by including the '-dynamic' flag
in XMKSHLIBOPTS.
The use of the '-fno-common' flag is also described here
<http://fink.sourceforge.net/doc/porting/shared.php>:
"The generation of position-independent code (PIC) is the default on
Darwin. Actually, PowerPC code is position-independent by design,
so
there is no performance or space penalty involved. So, you don't
need
to specify a PIC option when compiling code for a shared library or
module. However, the linker doesn't allow "common" symbols in
shared
libraries, so you must use the -fno-common compiler option."
=== export XMKSHLIBOPTS="-dynamic -undefined suppress -flat_namespace"
===
The '-dynamic' flag makes the library in the Mach-O format ctypes on
Mac OS X understands.
Without the '-undefined suppress' flag I get this:
ld: for architecture ppc
ld: Undefined symbols:
_fprintf$LDBLStub
_sprintf$LDBLStub
__Unwind_GetCFA
__Unwind_GetGR
__Unwind_GetIP
libtool: internal link edit command failed
Without the '-flat_namespace' flag I get this (and I remembered reading
about the flat namespace flag somewhere... ):
ld: for architecture ppc
ld: -undefined error or -undefined define_a_way must be used when
-twolevel_namespace is in effect
libtool: internal link edit command failed
But some references
<http://google.com/search?q=cache:www.metapkg.org/wiki/12> say this is
a bad idea:
"In any case, two-level namespaces are highly recommended, as flat
namespaces do NOT behave identially to the flat namespaces of other
popular linkers. Most importantly: the first definition of a symbol
is
used, rather than the last, which can cause serious problems with
plugin loading."
=== export OTHER_LIBS="-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0/
-lgcc" ===
This is required to avoid:
ld: for architecture ppc
ld: Darwin_DBG.OBJ/jsdate.o has external relocation entries in
non-writable section (__TEXT,__text) for symbols:
restFP
saveFP
libtool: internal link edit command failed
According to some references
<http://lists.apple.com/archives/Fortran-dev/2004/Jul/msg00036.html>
this is related to having multiple versions of gcc (e.g. Apple and
Fink) installed.
My "solution" is somewhat cargo-culted so I wouldn't be surprised to
learn something in my approach is ill-advised. But, it's working
for me for the moment so it's at least a start.
The best reference I've found (after a *lot* of searching) for this
issue was <http://fink.sourceforge.net/doc/porting/shared.php> which
includes the following:
"""
2.1 Shared Libraries vs. Loadable Modules
One Mach-O feature that hits many people by surprise is the strict
distinction between shared libraries and dynamically loadable modules.
On ELF systems both are the same; any piece of shared code can be used
as a library and for dynamic loading. Use otool -hv some_file to see
the filetype of some_file.
Mach-O shared libraries have the file type MH_DYLIB and carry the
extension .dylib. They can be linked against with the usual static
linker flags, e.g. -lfoo for libfoo.dylib. However, they can not be
loaded as a module. (Side note: Shared libraries can be loaded
dynamically through an API. However, that API is different from the API
for bundles and the semantics make it useless for an dlopen()
emulation. Most notably, shared libraries can not be unloaded.)
Loadable modules are called "bundles" in Mach-O speak. They have the
file type MH_BUNDLE. Since no component involved cares about it, they
can carry any extension. The extension .bundle is recommended by Apple,
but most ported software uses .so for the sake of compatibility.
Bundles can be dynamically loaded and unloaded via dyld APIs, and there
is a wrapper that emulates dlopen() on top of that API. It is not
possible to link against bundles as if they were shared libraries.
However, it is possible that a bundle is linked against real shared
libraries; those will be loaded automatically when the bundle is
loaded.
"""
This suggests I still may be getting the two confused, so feel free to
correct me.
Further comments on libtool, Mac OS X and porting are found here:
<http://fink.sourceforge.net/doc/porting/libtool.php?phpLang=en>
Anyway, hopefully this will be of use to someone.
Now to stop shaving yaks and get back to what I was *really* working
on...
--Phil.
compile SpiderMonkey ( libjs / libjs.so / libjs.dylib ) for OS X
(10.4.5 in my case) and then use it with Python's ctypes. I can't say
this will work for anyone else, but it worked for me...
Using:
http://ftp.mozilla.org/pub/mozilla.org/js/js-1.5.tar.gz
First up:
$ tar xzvf js-1.5.tar.gz
$ cd js/src
Then, the broken way (so, don't do this if you want ctypes to work
):
$ make -f Makefile.ref
...
$ file Darwin_DBG.OBJ/libjs.so
Darwin_DBG.OBJ/libjs.so: current ar archive
$ python OSError: dlcompat: Can not open "Darwin_DBG.OBJ/libjs.so"
Now, the unbroken way (after a 'make -f Makefile.ref clean' if need
be):
$ export XCFLAGS="-fno-common";export XMKSHLIBOPTS="-dynamic
-undefined suppress -flat_namespace"; export
OTHER_LIBS="-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0/ -lgcc"
$ make -f Makefile.ref libjs.so # Hmmm, this only seems to work after
a failed attempt with `make -f Makefile.ref` on its own...
...
$ file Darwin_DBG.OBJ/libjs.so
Darwin_DBG.OBJ/libjs.so: Mach-O dynamically linked shared library ppc
$ python <CDLL 'Darwin_DBG.OBJ/libjs.so', handle 1008000 at 7c310>
Example use: ...
A small bit of explanation as I understand things:
=== export XCFLAGS="-fno-common" ===
This is required to avoid an issue demonstrated here:
$ libtool -dynamic -o Darwin_DBG.OBJ/libjs.dylib
Darwin_DBG.OBJ/libjs.a -lSystem
ld: for architecture ppc
ld: common symbols not allowed with MH_DYLIB output format with the
-multi_module option
Darwin_DBG.OBJ/libjs.a(jsatom.o) definition of common
_js_atom_map_count (size 4)
Darwin_DBG.OBJ/libjs.a(jsatom.o) definition of common
_js_atom_map_hash_table_count (size 4)
libtool: internal link edit command failed
This method of conversion was discussed here:
<http://groups.google.com/group/comp.sys.mac.programmer.help/msg/f3b4ad292965fe94?hl=en&>
but I think I first ran into the error by including the '-dynamic' flag
in XMKSHLIBOPTS.
The use of the '-fno-common' flag is also described here
<http://fink.sourceforge.net/doc/porting/shared.php>:
"The generation of position-independent code (PIC) is the default on
Darwin. Actually, PowerPC code is position-independent by design,
so
there is no performance or space penalty involved. So, you don't
need
to specify a PIC option when compiling code for a shared library or
module. However, the linker doesn't allow "common" symbols in
shared
libraries, so you must use the -fno-common compiler option."
=== export XMKSHLIBOPTS="-dynamic -undefined suppress -flat_namespace"
===
The '-dynamic' flag makes the library in the Mach-O format ctypes on
Mac OS X understands.
Without the '-undefined suppress' flag I get this:
ld: for architecture ppc
ld: Undefined symbols:
_fprintf$LDBLStub
_sprintf$LDBLStub
__Unwind_GetCFA
__Unwind_GetGR
__Unwind_GetIP
libtool: internal link edit command failed
Without the '-flat_namespace' flag I get this (and I remembered reading
about the flat namespace flag somewhere...
):ld: for architecture ppc
ld: -undefined error or -undefined define_a_way must be used when
-twolevel_namespace is in effect
libtool: internal link edit command failed
But some references
<http://google.com/search?q=cache:www.metapkg.org/wiki/12> say this is
a bad idea:
"In any case, two-level namespaces are highly recommended, as flat
namespaces do NOT behave identially to the flat namespaces of other
popular linkers. Most importantly: the first definition of a symbol
is
used, rather than the last, which can cause serious problems with
plugin loading."
=== export OTHER_LIBS="-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.0/
-lgcc" ===
This is required to avoid:
ld: for architecture ppc
ld: Darwin_DBG.OBJ/jsdate.o has external relocation entries in
non-writable section (__TEXT,__text) for symbols:
restFP
saveFP
libtool: internal link edit command failed
According to some references
<http://lists.apple.com/archives/Fortran-dev/2004/Jul/msg00036.html>
this is related to having multiple versions of gcc (e.g. Apple and
Fink) installed.
My "solution" is somewhat cargo-culted so I wouldn't be surprised to
learn something in my approach is ill-advised.
But, it's workingfor me for the moment so it's at least a start.
The best reference I've found (after a *lot* of searching) for this
issue was <http://fink.sourceforge.net/doc/porting/shared.php> which
includes the following:
"""
2.1 Shared Libraries vs. Loadable Modules
One Mach-O feature that hits many people by surprise is the strict
distinction between shared libraries and dynamically loadable modules.
On ELF systems both are the same; any piece of shared code can be used
as a library and for dynamic loading. Use otool -hv some_file to see
the filetype of some_file.
Mach-O shared libraries have the file type MH_DYLIB and carry the
extension .dylib. They can be linked against with the usual static
linker flags, e.g. -lfoo for libfoo.dylib. However, they can not be
loaded as a module. (Side note: Shared libraries can be loaded
dynamically through an API. However, that API is different from the API
for bundles and the semantics make it useless for an dlopen()
emulation. Most notably, shared libraries can not be unloaded.)
Loadable modules are called "bundles" in Mach-O speak. They have the
file type MH_BUNDLE. Since no component involved cares about it, they
can carry any extension. The extension .bundle is recommended by Apple,
but most ported software uses .so for the sake of compatibility.
Bundles can be dynamically loaded and unloaded via dyld APIs, and there
is a wrapper that emulates dlopen() on top of that API. It is not
possible to link against bundles as if they were shared libraries.
However, it is possible that a bundle is linked against real shared
libraries; those will be loaded automatically when the bundle is
loaded.
"""
This suggests I still may be getting the two confused, so feel free to
correct me.
Further comments on libtool, Mac OS X and porting are found here:
<http://fink.sourceforge.net/doc/porting/libtool.php?phpLang=en>
Anyway, hopefully this will be of use to someone.
Now to stop shaving yaks and get back to what I was *really* working
on...
--Phil.