building extensions for Windows Python

[JW]

I have a lousy little Python extension, generated with the generous help
of Pyrex. In Linux, things are simple. I compile the extension, link it
against some C stuff, and *poof*! everything works.

My employer wants me to create a Windows version of my extension that
works with the vanilla Python 2.5 from python.org.

My employment contract states that I won't be required to run Windows, and
I desperately want to honor that clause. Ideally, I'd somehow MinGw cross
compile like I do with C/C++ and *poof*!, out would pop a file I could
hand out to someone who wanted to "import <my_lousy_extension>".

In Windows, things aren't so simple. I'm immediately bedazzled by a vast,
unfamiliar nomenclature: "VC6", "VC7.1", "msvcrt", "msvcr71", "msvcr80",
"VS2005", "VS2007", "MSVC2003", "MSVC2005", "Express Edition", ".NET SDK",
etc.

The python.org "recommended compiler" is apparently no longer
available, but someone posted a link to an obscure Microsoft webpage
artifact where it could still be obtained. Running it under WINE is less
than ideal, but I decided to try and make it work. However, when I tried
installing the compiler, it failed because my "Internet Explorer [was] out
of date." I'm not sure how IE comes into the picture, but I'm just a
programmer, after all.

I am further confused by the python.org docs on using MinGw to build
extensions: "These instructions only apply if you're using a version of
Python prior to 2.4.1 with a MinGW prior to 3.0.0", but further on:
"pcexports python25.dll >python25.def" seems to contradict that. It
really makes no difference. The referenced webpage where the "pcexports"
tool is found seems defunct.

My main problem is that I don't really grasp the big picture. Can someone
give me an overview of the requirements to build extensions for Windows
Python, circa 2.5? Or, can I tell my employer she'll just have to
hire/contract a Windows expert to do the mud wrestling?

Jim Wilson
Gainesville, FL

 

[olsongt]

My main problem is that I don't really grasp the big picture. Can someone
give me an overview of the requirements to build extensions for Windows
Python, circa 2.5? Or, can I tell my employer she'll just have to
hire/contract a Windows expert to do the mud wrestling?

Jim Wilson
Gainesville, FL

To build an extension that runs on a generic python 2.5 you need a
Windows OS (suprise!) and VS.NET 2003. Some people may have had some
success with MinGW32 (on windows) but it's not going to work
'out-of-the-box'.

If your code is straightforward crossplatform C, you could setup a
distutils configuration script and pass it off to someone else to build
the windows binaries, but chances are some debugging will have to be
done.

 

[Michael L Torrie]

I have a lousy little Python extension, generated with the generous help
of Pyrex. In Linux, things are simple. I compile the extension, link it
against some C stuff, and *poof*! everything works.

My employer wants me to create a Windows version of my extension that
works with the vanilla Python 2.5 from python.org.

My employment contract states that I won't be required to run Windows, and
I desperately want to honor that clause. Ideally, I'd somehow MinGw cross
compile like I do with C/C++ and *poof*!, out would pop a file I could
hand out to someone who wanted to "import <my_lousy_extension>".

I took a very simple pyrex file that compiled and worked fine on linux
and built it for windows using my mingw cross compiler.

First I borrowed a complete windows python installation and stuck it on
my linux drive (in my case in /usr/i386-pc-mingw32/Python2.4.

Then I just did:

$ pyrex test.pyx

$ i386-pc-mingw32-gcc -o test.dll test.c -I/usr/i386-pc-
mingw32/include/python2.4 -mwindows -shared -lpython24 -L/usr/i386-pc-
mingw32/Python2.4/lib

Since there were no special windows libraries used, there were no code
changes required at all. So look into the mingw cross compiler.
Usually you have to build it yourself. see
http://www.torriefamily.org/~torriem/cross for more information. The
information on the windows target is in the second half of the document.
This document used the OS X as the build host, but the same steps work
on linux. I prefer to use /usr/local or /usr as my prefix (ending up
with /usr/local/bin/i386-pc-mingw32-gcc etc)

Michael

In Windows, things aren't so simple. I'm immediately bedazzled by a vast,
unfamiliar nomenclature: "VC6", "VC7.1", "msvcrt", "msvcr71", "msvcr80",
"VS2005", "VS2007", "MSVC2003", "MSVC2005", "Express Edition", ".NET SDK",
etc.

The python.org "recommended compiler" is apparently no longer
available, but someone posted a link to an obscure Microsoft webpage
artifact where it could still be obtained. Running it under WINE is less
than ideal, but I decided to try and make it work. However, when I tried
installing the compiler, it failed because my "Internet Explorer [was] out
of date." I'm not sure how IE comes into the picture, but I'm just a
programmer, after all.

I am further confused by the python.org docs on using MinGw to build
extensions: "These instructions only apply if you're using a version of
Python prior to 2.4.1 with a MinGW prior to 3.0.0", but further on:
"pcexports python25.dll >python25.def" seems to contradict that. It
really makes no difference. The referenced webpage where the "pcexports"
tool is found seems defunct.

My main problem is that I don't really grasp the big picture. Can someone
give me an overview of the requirements to build extensions for Windows
Python, circa 2.5? Or, can I tell my employer she'll just have to
hire/contract a Windows expert to do the mud wrestling?

Jim Wilson
Gainesville, FL

 

[Nick Craig-Wood]

I have a lousy little Python extension, generated with the generous help
of Pyrex. In Linux, things are simple. I compile the extension, link it
against some C stuff, and *poof*! everything works.
;-)

My employer wants me to create a Windows version of my extension that
works with the vanilla Python 2.5 from python.org.

My employment contract states that I won't be required to run Windows, and
I desperately want to honor that clause. Ideally, I'd somehow MinGw cross
compile like I do with C/C++ and *poof*!, out would pop a file I could
hand out to someone who wanted to "import <my_lousy_extension>".

You can do that no trouble - we do. We use mingw on linux to compile
stuff for windows all the time. (We use the mingw package under
debian)

We build extensions using mingw but linked to the link library of the
official python2.4 build. The extensions then run with the standard
python.org install.

Here are some (slightly dated) instructions which you'll need to adapt
to your setup

/misc/windows is a smb mounted windows machine

# Linking with the distributed python
#
# http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/cygwin.html
#
# On a windows machine
# install the latest windows python (2.4.3.msi) from www.python.org
# Copy the header files into the mingw installation
cp -av /misc/windows/Python24/include /usr/i586-mingw32msvc/include/python2.4
# Download pexports from here
# http://www.emmestech.com/software/cygwin/pexports-0.43/download_pexports.html
# unpack pexports.exe
unzip pexports-0.43.zip
# Fetch python dll from the windows machine
cp -av /misc/windows/WINNT/system32/python24.dll .
# Extract the exported symbols
wine pexports python24.dll > python24.def
# Create the link library
/usr/i586-mingw32msvc/bin/dlltool --dllname python24.dll --def python24.def --output-lib libpython2.4.a
# Move the files into the correct place
mv -i python24.dll python24.def libpython2.4.a /usr/i586-mingw32msvc/lib/

After that lot you can build python extensions with mingw under linux,
using -lpython2.4 which will run under windows.

 

[Nick Craig-Wood]

To build an extension that runs on a generic python 2.5 you need a
Windows OS (suprise!) and VS.NET 2003. Some people may have had some
success with MinGW32 (on windows) but it's not going to work
'out-of-the-box'.

Neither of those statements are true. It is fairly easy to build
python extensions using mingw hosted on linux which work with the
standard python.org install - see my other post in this thread.

 

[JW]

Thanks to Michael and Nick, I can now cross-compile my Pyrex extensions
for bog-standard Python 2.5. As I stumbled around in the dark trying to
bump into a solution, I was bolstered by the belief that at least two
other people had found the light at the end of the tunnel.

I had been using a crufty old prebuilt MinGW that linked to MSVCRT.DLL but
didn't include support for the newer VC runtime DLLs. I had to build the
current version which adds support for MVCR71 (used by Python 2.5) and
MSVCR80 (used by Microsoft's current ".NET" offering).

(Note: The build script proffered by mingw.org fails really cryptically if
you don't have the scanner tool, "flex", installed. But that's another
story. After some hand-to-hand combat, I had a working MinGW, nee GCC
3.4.2.)

Microsoft advises that it's very bad mojo to mix calls to MSVCRT.DLL and
MSVCR71.DLL in one application, but http://tinyurl.com/ydndnn (or a page
it references) advises how to avert such a disaster with a simple change
to MinGW's "spec" file (which I never knew existed).

The upshot is that I can now use Linux to cross-build my extension for
Windows, and my preliminary testing (under WINE -- See the original post.
I must adhere to my employment contract!) seems to show that it works.

Had I not known someone else claimed success, I would have thrown up my
hands in frustration.

Jim Wilson
Gainesville, FL

 

[Sybren Stuvel]

JW enlightened us with:

Thanks to Michael and Nick, I can now cross-compile my Pyrex
extensions for bog-standard Python 2.5 [...] I can now use Linux to
cross-build my extension for Windows, and my preliminary testing
(under WINE -- See the original post. I must adhere to my
employment contract!) seems to show that it works.

Congratulations!

Sybren