Building things with setup.py

[James Stroud]

Let's take a specific failure, namely the line

/auto_nfs/data10/users/jstroud/Programs/bin/g77
-L/data10/users/jstroud/Programs/lib
-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3
build/temp.linux-i686-2.5/numpy/core/blasdot/_dotblas.o
-L/data10/users/jstroud/Programs/lib -lblas -lg2c -o
build/lib.linux-i686-2.5/numpy/core/_dotblas.so

This gives errors like

build/temp.linux-i686-2.5/numpy/core/blasdot/_dotblas.o(.text+0x758):numpy/core/blasdot/_dotblas.c:226:
undefined reference to `PyTuple_New'

That's not too surprising: this line tries to link the input
*as an executable program*, despite calling the output
_dotblas.so. In an executable program, all symbols need to
be defined; that's why it it complains about so many missing
symbols (including MAIN__ - which should never be missing
in a library). Even though adding python2.5.a to the linker
link makes these symbols appear, the result still won't
work, as you can't use an executable program file as if
it were a shared library.

Now, compare this to a succeeding build of a C extension
module,

gcc -pthread -shared -L/data10/users/jstroud/Programs/lib
-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3
-I/data10/users/jstroud/Programs/include
-I/data10/users/jstroud/Programs/qt/include -I/usr/include
-I/data10/users/jstroud/Programs/include/freetype2/freetype
build/temp.linux-i686-2.5/build/src.linux-i686-2.5/numpy/core/src/umathmodule.o
-lm -o build/lib.linux-i686-2.5/numpy/core/umath.so

Notice that this passes -shared to the compiler, requesting
construction of a shared library. This is the problem with
the g77 linker line; try invoking

/auto_nfs/data10/users/jstroud/Programs/bin/g77 -shared
-L/data10/users/jstroud/Programs/lib
-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3
build/temp.linux-i686-2.5/numpy/core/blasdot/_dotblas.o
-L/data10/users/jstroud/Programs/lib -lblas -lg2c -o
build/lib.linux-i686-2.5/numpy/core/_dotblas.so

manually (whether or not -pthread is also necessary
or supported for g77, I don't know). This should at least
make the complaints about missing symbols go away; you
should then see whether the resulting module can be
imported in Python.

If that solves the problem, the question is why the
-shared option isn't passed automatically; your setting
LDFLAGS might indeed be a cause.

Regards,
Martin

As per your and Robert Kern's suggestions, unsetenv'ing $LDFLAGS and
$CPPFLAGS indeed did the trick for numpy, scipy, and mysqldb. What a
tough lesson in distutils!

<COMMENTARY>Though great for self development, I'm not so sure such
lessons should be necessary to build these tools.</COMMENTARY>

James

James

 

[James Stroud]

Okay, this is possibly part of the problem. numpy.distutils handles
FORTRAN code differently than other extension modules; this is why pure
C extension modules both inside numpy and elsewehere were linking fine.
Try unsetenving $CPPFLAGS and $LDFLAGS and adding those arguments to
build_ext, if necessary.

Looking on line 516 of numpy/fcompiler/__init__.py (why there's a class
definition in __init__.py, I'll never know), it does look like the
linker flags which are carefully constructed by numpy.distutils are
indeed overridden by the user's setting of $LDFLAGS.

I unset $LDFLAGS and $CPPFLAGS (not trying one at a time), and this did
the trick. Everything built without a hitch.


James

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

<COMMENTARY>Though great for self development, I'm not so sure such
lessons should be necessary to build these tools.</COMMENTARY>

Yes. The lesson you should take from this is: don't try to be smarter
than the authors of the software. It should build out of the box, if
you follow the build instructions literally. Deviate to the tiniest
from the instructions, and you are on your own.

If it doesn't work out of the box even though you followed the
instructions, don't try to fix it. You didn't write it, fixing
it will be a huge effort. Instead, get help from somebody, and/or
report a bug.

Always strive for the simplest possible setup. If you find that you
have to customize something, and the instructions did not tell you
to do so, question this customization. It is likely wrong or
unnecessary.

Regards,
Martin