distutils, PyBison and Mac OS X

[Maarten Sneep]

I'm trying to build PyBison[1] on Mac OS X, and I'm running into some
problems with the distutils.

Just for starters: PyBison requires Pyrex[2]. This is not a problem,
and Pyrex seems to work without problems, at least the primes sample
module shows a nice 25 to 30 fold speed increase over the pure python
version.

I used the distutils to create the module from the primes.pyx sample,
following the setup.py from the PyBison distrubution:

from distutils.core import setup, Extension
from Pyrex.Distutils import build_ext

setup( name = "primes", version = "1.0",
  description="Pyrex sample for calculating prime numbers",
  author='Greg Ewing', 
  url='http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/',
  ext_modules=[Extension("primes", ["primes.pyx"])],
  cmdclass = {'build_ext': build_ext}
)

I mention this, because PyBison seems to use the distutils to compile
the bison and flex source code on the fly, but fails to pick up some
parameters, which prevents it from working correctly.

Although Mac OS X uses gcc, and is an otherwise pretty complete unix
install, it is missing some components: shared libraries are not
linked with -shared but use -bundle instead. A succesful build of
primes is linked with:

export MACOSX_DEPLOYMENT_TARGET=10.3; gcc -Wl,-F. -Wl,-F. \
-bundle -undefined dynamic_lookup \
build/temp.darwin-7.9.0-Power_Macintosh-2.3/primes.o \
-o primes.so

(Another difference that caused major headaches: the absense of a
'dl' library has been resolved with version 10.3 ("Panther"). A dl
library is included as a wrapper around the bundles mechanism that
were present all along. Since the linux version uses the dl library,
I tried to compile the same source, and that part seems to run just
fine).

The issues:

- distutils.ccompiler uses the unixcompiler, which has hard coded a
cc -shared to create shared objects. I've tried to override this,
but no luck there (other issues seem to crop up). Is this a known
issue with distutils?

I installed pybison using the linux C-source, and tried it out
using the 'calc' example. After setting verbose=1 in the run.py,
and enabling the "int lineno = 0" in calc.py, the compilation phase
finishes without a hitch, but linking fails badly:

bisonCmd=['bison', '-d', '-v', '-t', 'tmp.y']
renaming bison output files
tmp.tab.c => tmp.bison.c
tmp.tab.h => tokens.h
cc: unrecognized option `-shared'
ld: Undefined symbols:
_main
_PyInt_FromLong
_PyObject_GetAttrString
_PyObject_HasAttrString
_PyObject_SetAttrString
_PyString_AsString
_PyString_FromString
_PyTuple_New
_PyTuple_SetItem
__Py_NoneStruct
Traceback (most recent call last):
File "run.py", line 7, in ?
parser = calc.Parser(verbose=1, keepfiles=1)
File ". . ./python2.3/site-packages/bison.py", line 308,
in __init__ self.engine = ParserEngine(self)
File "bison_.pyx", line 187, in bison_.ParserEngine.__init__
File "bison_.pyx", line 202, in
bison_.ParserEngine.openCurrentLib
File "bison_.pyx", line 540, in bison_.ParserEngine.buildLib
File ". . ./python2.3/distutils/ccompiler.py", line 843,
in link_shared_object extra_preargs, extra_postargs,
build_temp, target_lang)
File ". . ./python2.3/distutils/unixccompiler.py", line 178,
in link raise LinkError, msg
distutils.errors.LinkError: command 'cc' failed with exit status 1

- Would it be possible to modify PyBison to use a call to the
distutils.core.setup(. . .) function and call that as if 'python
setup.py build_ext --inplace' had been called from the command line,
since that seems to pick up the correct options.

- Where do I start: distutils is pretty large, and it seems to have
the right options, but they just aren't used in PyBison.

Thanks in advance for any insights that get me going,

Maarten

Links:
[1] http://www.freenet.org.nz/python/pybison/
[2] http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/