pyconfig on 64-bit machines with distutils vs 32-bit legacy code

[Rob Clewley]

Hi,

I have a problem compiling legacy C code using distutils on a
colleague's 64 bit intel machine running linux. The legacy code is
linked to user-generated code from python and distutils is a
convenient platform independent way to call whatever compiler is
available to create a DLL as a python module via SWIG. My code works
fine on 32 bit machines, but on a 64 bit machine the error

/data/home/nwagner/local/lib/python2.5/pyport.h:734:2: #error
"LONG_BIT definition appears wrong for platform (bad gcc/glibc
config?)."

was generated.

I looked this error up and found it referred to a bug in python about
pyconfig.h not being automatically adaptive when mixing 32-bit code
and 64-bit libraries. There were various vague references to solutions
and a supposed fix in python, but the posts were at least a couple of
years old and didn't make a lot of sense to me. I tried adding the
'-m32' option to the gcc call using extra_compile_args but that was
not enough. According to a different suggestion on a different mailing
list, I tried replacing pyconfig.h with a version that used 32-bit
settings for the size of LONG and the other 64-bit specific sizes.
That did not help at all.

The gcc call created from my PyDSTool package's code via distutils is
(slightly reformatted for easy reading):

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC
-I/data/home/nwagner/local/lib/python2.5/site-packages/numpy/core/include
-I/data/home/nwagner/local/lib/python2.5/site-packages/numpy/numarray
-I/data/home/nwagner/svn/PyDSTool/PyDSTool/tests
-I/data/home/nwagner/svn/PyDSTool/PyDSTool/integrator
-I/data/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp
-I/data/home/nwagner/local/include/python2.5
-c /data/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.c
-o /data/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/data/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.o
-w -m32 -D__DOPRI__

Can anyone offer any advice as to what I might be missing or
misunderstanding? I assumed it would be possible to do this
compilation even if a 64 bit version of python has been installed on
this linux box.

I'm somewhat out of my depth talking about these internal gubbins so
please go easy on me.

Thanks in advance,
Rob

 

[Martin v. Löwis]

/data/home/nwagner/local/lib/python2.5/pyport.h:734:2: #error

"LONG_BIT definition appears wrong for platform (bad gcc/glibc
config?)."


Can anyone offer any advice as to what I might be missing or
misunderstanding?

You need to understand where the error comes from:
1. what is the *actual* value of SIZEOF_LONG (it should be 4)?
2. what is the actual value of LONG_BIT, and where does it come
from? (it should be 32)

To understand that better, I recommend to edit the command line
of gcc in the following way (assuming you use gcc 4.x):
1. replace -c with -E -dD
2. remove the -o <file> option

This will generate preprocessor output to stdout, which you then
need to search for SIZEOF_LONG and LONG_BIT. Searching back for
# <number> lines will tell you where the definition was made.

If that doesn't make it clear what the problem is, post your
findings here.

Regards,
Martin

 

[Martin v. Löwis]

/data/home/nwagner/local/lib/python2.5/pyport.h:734:2: #error

"LONG_BIT definition appears wrong for platform (bad gcc/glibc
config?)."


Can anyone offer any advice as to what I might be missing or
misunderstanding?

You need to understand where the error comes from:
1. what is the *actual* value of SIZEOF_LONG (it should be 4)?
2. what is the actual value of LONG_BIT, and where does it come
from? (it should be 32)

To understand that better, I recommend to edit the command line
of gcc in the following way (assuming you use gcc 4.x):
1. replace -c with -E -dD
2. remove the -o <file> option

This will generate preprocessor output to stdout, which you then
need to search for SIZEOF_LONG and LONG_BIT. Searching back for
# <number> lines will tell you where the definition was made.

If that doesn't make it clear what the problem is, post your
findings here.

Regards,
Martin

 

[Rob Clewley]

Thanks for replying, Martin.

I got my colleague (Nils) to run exactly the gcc call you described in
your post (see below for what he ran) but it only returns the
following:

/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.c:124:20:
error: Python.h: Datei oder Verzeichnis nicht gefunden
/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.c:2495:4:
error: #error "This python version requires swigto be run with the
'-classic' option"
In file included from
/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:61,
from
/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/arrayobject.h:14,
from
/home/nwagner/local/lib64/python2.6/site-packages/numpy/numarray/numpy/libnumarray.h:7,
from
/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.c:2758:
/home/nwagner/local/lib64/python2.6/site-packages/numpy/core/include/numpy/npy_common.h:71:2:
error: #error Must use Python with unicode enabled.

The command was

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall
-Wstrict-prototypes -fPIC
-I/data/home/nwagner/local/lib/python2.5/site-packages/numpy/core/include
-I/data/home/nwagner/local/lib/python2.5/site-packages/numpy/numarray
-I/data/home/nwagner/svn/PyDSTool/PyDSTool/tests
-I/data/home/nwagner/svn/PyDSTool/PyDSTool/integrator
-I/data/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp
-I/data/home/nwagner/local/include/python2.5 -E -dD
/data/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.c
-w -m32 -D__DOPRI__

Maybe Nils can pick up the thread from here.
Thanks,
Rob

 

[Martin v. Löwis]

I got my colleague (Nils) to run exactly the gcc call you described in
your post (see below for what he ran) but it only returns the
following:

Sehr seltsam. Welche gcc-Version ist das denn? (gcc -v)

/home/nwagner/svn/PyDSTool/PyDSTool/tests/dopri853_temp/dop853_HHnet_vf_wrap.c:124:20:
error: Python.h: Datei oder Verzeichnis nicht gefunden

Auch sehr seltsam. Ist Python.h auf dem Suchpfad vorhanden?

Wenn man eine Datei nur mit

#include <Python.h>

schreibt, und dann

gcc -E -<die ganze I-Optionen> datei.c

schreibt, ergibt sich dann eine Ausgabe? Wenn man dann -dD hinzunimmt:
was passiert?

Viel Erfolg,

Martin