extending python with a C-written dll

  • Thread starter Jean-Baptiste PERIN
  • Start date
J

Jean-Baptiste PERIN

Hi,

I'm trying to make a windows dll reachable from a python script .. and
I'm encountering troubles during link step ..

I use "lcc" to compile and link
I use python 2.3 under win XP

Here's the compile command which seems to work well :
-----------------------------------------------------
lcclnk.exe -entry inithello -dll -o hello.dll hello.obj -L
"C:\python23\libs" python23.lib


Here's the link command which fails:
------------------------------------
lcclnk.exe -entry inithello -dll -o hello.dll hello.obj -L
"C:\python23\libs" python23.lib


Here are the errors :
---------------------
Error c:\...\plugins\hello.c 14 undefined reference to __imp__Py_BuildValue
Error c:\...\plugins\hello.c 46 undefined reference to __imp__Py_InitModule4



Here's the C code which compile but does not link :
--------------------------------------------------

#include <Python.h>

PyObject *helloHello(PyObject *self, PyObject *args)
{
printf("Hello World!");
return Py_BuildValue("i", 1);
}

PyObject *helloHelloString(PyObject *self, PyObject *args)
{
return Py_BuildValue("s", "Hello World!");
}

static char MHello__doc__[] = "This is a simple Python C extension example";
static char Hello__doc__[] = "Prints in the console";
static char HelloString__doc__[] = "Returns message as string";

PyMethodDef helloMethods[] = {
{"Hello", helloHello, METH_VARARGS, Hello__doc__},
{"HelloString", helloHelloString, METH_VARARGS, HelloString__doc__},
{NULL, NULL}
};

void __declspec(dllexport) inithello()
{
(void)Py_InitModule3("hello", helloMethods, MHello__doc__);
}

Can anyone help me?
 
G

Grant Edwards

I'm trying to make a windows dll reachable from a python script..

FWIW, you can call dll's using the ctypes modules without
mucking around in C. There may be performance reasons to build
a "real" python module, but I haven't run across them yet.
 
J

Jean-Baptiste PERIN

Grant Edwards a écrit :
FWIW, you can call dll's using the ctypes modules without
mucking around in C. There may be performance reasons to build
a "real" python module, but I haven't run across them yet.

Not really a performance reason .. but mainly because I have a huge code
produced in C and no time to port it in python ..
Moreover .. I need the python program to interact with the C code
...(perfom calls to dll-defined functions)

I don't know ctypes module .. do you really think it can help ?
Do you have a link to quickly learn about it ?

The only thing I found is :
http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html
 
G

Grant Edwards

Not really a performance reason .. but mainly because I have a
huge code produced in C and no time to port it in python ..
Moreover .. I need the python program to interact with the C
code ..(perfom calls to dll-defined functions)

I don't know ctypes module .. do you really think it can help?

It lets you easily call functions in DLLs.

I think that is unrelated.
 
J

Jean-Baptiste PERIN

Grant Edwards a écrit :
It lets you easily call functions in DLLs.




I think that is unrelated.

OK .. ctypes looks great .. and I plan to use it for futur purposes

The fact is that the python interpreter I use is not a standard one
It is a python interpreter delivered within a software named Blender.
I don't know whether it is possible or not to add ctypes to it ..(I
don't even have a python shell to perform the setup)

I'm sure it is possible to use dll-built with it ..
So please .. can you tell me where I can find Py_BuildValue and
Py_InitModule4 to link with ?
 
G

Grant Edwards

OK .. ctypes looks great .. and I plan to use it for futur
purposes

The fact is that the python interpreter I use is not a
standard one It is a python interpreter delivered within a
software named Blender. I don't know whether it is possible or
not to add ctypes to it ..(I don't even have a python shell to
perform the setup)

I'm sure it is possible to use dll-built with it .. So please
.. can you tell me where I can find Py_BuildValue and
Py_InitModule4 to link with ?

Sorry, I don't know anything about doing C lanugage extensions
under Windows.
 
D

Diez B. Roggisch

The fact is that the python interpreter I use is not a standard one
It is a python interpreter delivered within a software named Blender.
I don't know whether it is possible or not to add ctypes to it ..(I
don't even have a python shell to perform the setup)

I'm sure it is possible to use dll-built with it ..
So please .. can you tell me where I can find Py_BuildValue and
Py_InitModule4 to link with ?

I doubt that adding new modules as dlls is easier than incorporating
pure-python ones or a mixture of both - as ctypes most probably is. After
all, its pythons module loading mechanism that kicks in for all cases.

If ctypes suits your needs outside of blender, I'd go for trying to
incorporate it into the blender interpreter.
 
T

Thomas Heller

Jean-Baptiste PERIN said:
Grant Edwards a écrit :


OK .. ctypes looks great .. and I plan to use it for futur purposes

The fact is that the python interpreter I use is not a standard one
It is a python interpreter delivered within a software named Blender.
I don't know whether it is possible or not to add ctypes to it ..(I
don't even have a python shell to perform the setup)

You could try to download the ctypes windows installer, use winzip or
equivalent to unpack the contained files (you need _ctypes.pyd, and the
ctypes directory containing __init__.py), and copy them to some
directory where python can find them.

Thomas
 
J

Jean-Baptiste PERIN

thanks for your valuable help .. ctypes is not lost for ever ..I'm
going to explore a little bit .. it may suit my need in the end ..


I had read :

http://www.python.org/dev/doc/devel/ext/intro.html

and eveything looked so simple in it ..
I'm sad to realize that it only works for linux plateform :(

I've tried to build and install ctypes from sources

I ran the following command:

and I got the following message:
extensions need to
be built with the same version of the compiler, but it isn't installed.


:( :( :( :( So sad ....
 
J

Jean-Baptiste PERIN

Diez B. Roggisch a écrit :
I doubt that adding new modules as dlls is easier than incorporating
pure-python ones or a mixture of both - as ctypes most probably is. After
all, its pythons module loading mechanism that kicks in for all cases.

If ctypes suits your needs outside of blender, I'd go for trying to
incorporate it into the blender interpreter.


I've tried to build and install ctypes from sources

I ran the following command:

and I got the following message:
extensions need to
be built with the same version of the compiler, but it isn't installed.


:( :( :( :( So sad ....
 
J

Jean-Baptiste PERIN

Thomas Heller a écrit :
You could try to download the ctypes windows installer, use winzip or
equivalent to unpack the contained files (you need _ctypes.pyd, and the
ctypes directory containing __init__.py), and copy them to some
directory where python can find them.

Thomas

I took a look at the ctypes tutorial .. Here's what I saw ..
>>> from ctypes import *
>>> printf = cdll.msvcrt.printf
>>> printf.argtypes = [c_char_p, c_char_p, c_int, c_double]
>>> printf("String '%s', Int %d, Double %f\n", "Hi", 10, 2.2)

just to perfom a printf !!!

now look at http://www.python.org/dev/doc/devel/ext/intro.html

and look at the expected result .. it is far simpler from the python
side view

I think ctype is uselful when dealing with dll you're not the author of

But if noone can help me to extend python with my own dll .. I going to
explore ctypes .. but I find it's bothering .. and frustrating not to
achieve what I wanted to do ..

Moreover .. I tried to build and install ctypes from source ..
I ran the command

and I got the following message:
extensions need to
be built with the same version of the compiler, but it isn't installed.


SO SAD :( :( :(
 
D

Diez B. Roggisch

and look at the expected result .. it is far simpler from the python
side view

But you have to do all the C-Work!
I think ctype is uselful when dealing with dll you're not the author of

It is useful, if you don't want to write an extension for the extra work
that is needed for that - for example if you are missing a C compiler.
extensions need to
be built with the same version of the compiler, but it isn't installed.

How do you expect to be able to write your own extension if you don't have
the environment to do so in the first place? that error has nothing to do
with python itself, but with you not having installed the right compiler.
THe same problem will arise building your own extension.

Others suggested trying the ctypes binary installer. Do so.
 
S

Scott David Daniels

Jean-Baptiste PERIN said:
Hi,

I'm trying to make a windows dll reachable from a python script .. and
I'm encountering troubles during link step ..

I use "lcc" to compile and link
I use python 2.3 under win XP

Note that MinGW32 is a tested path for building (free) C extensions. I
don't know if anyone has a tested way of using lcc.

With MinGW32, your build step will be the delightfully easy:

python setup.py build --compiler=mingw32

--Scott David Daniels
(e-mail address removed)
 
J

Jean-Baptiste PERIN

How do you expect to be able to write your own extension if you don't have
the environment to do so in the first place? that error has nothing to do
with python itself, but with you not having installed the right compiler.
THe same problem will arise building your own extension.

I know it has nothing to do with python ..
It just makes me sad to see that python cannot get rid of Visual Studoi
to be extended . It's just amazing !!

Python is free
The compiler I use is free
The code I plan to produce will be free

Where can I find a free version of Visual Studio ?

the only thing I use that is not free is the operating system and it's
because I had no choice .. my company decided for it
Others suggested trying the ctypes binary installer. Do so.

Yes .. that's what I'm going to do .. but I find it frustrating ..
 
S

Steve Holden

Jean-Baptiste PERIN said:
I know it has nothing to do with python ..
It just makes me sad to see that python cannot get rid of Visual Studoi
to be extended . It's just amazing !!

Python is free
The compiler I use is free
The code I plan to produce will be free

Where can I find a free version of Visual Studio ?

the only thing I use that is not free is the operating system and it's
because I had no choice .. my company decided for it



Yes .. that's what I'm going to do .. but I find it frustrating ..

Perhaps you aren't aware that Microsoft make a free tool chain available
that can be used to create binary extensions? There has been some
discussion of this recently, and I managed to compile MySQLdb and a
couple of the mx extensions, so it's eminently doable [the PIL is
proving a little less tractable due to dependencies on other libraries
that I have yet to resolve].

The downloads are quite large, but if you want to try it, take a look at

http://www.vrplumber.com/programming/mstoolkit/

regards
Steve
 
G

Grant Edwards

I know it has nothing to do with python ..
It just makes me sad to see that python cannot get rid of Visual Studoi
to be extended.

That's got nothing to do with Python. You have to compile
extensions using a compiler that has an ABI that's compatible
with the ABI of the compiler used to compile Python.

You appear to be using a binary distribution of Python that was
compiled with MSVS 6.0, therefore, you have to compile
extensions with a compiler that has an ABI compatible with MSVS
6.0.

It's no use complaining to us bout your C compiler not being
ABI compatible with MSVS 6.0.

If you don't like MSVS, then your options are to either use a
different Python binary distro that was compiled with an ABI
that's compatible with your C compiler, or build just build it
yourself using whatever compiler you want. I suspect the latter
will also require re-compiling Blender.
 
J

John Machin

Jean-Baptiste PERIN said:
Hi,

I'm trying to make a windows dll reachable from a python script .. and
I'm encountering troubles during link step ..

I use "lcc" to compile and link
I use python 2.3 under win XP [snip]
Here are the errors :
---------------------
Error c:\...\plugins\hello.c 14 undefined reference to __imp__Py_BuildValue
Error c:\...\plugins\hello.c 46 undefined reference to __imp__Py_InitModule4
[snip]
return Py_BuildValue("i", 1); [snip]
void __declspec(dllexport) inithello()
{
(void)Py_InitModule3("hello", helloMethods, MHello__doc__);
Can anyone help me?

Yes. Option 1: *Maybe* you can help yourself. Have you not noticed from
the above that the compiler is magically prepending "__imp__" to the
names of external functions? However the .lib that comes with Python
won't have that "__imp__". You would have to bridge that gap, somehow.
Have you searched the lcc documentation? Are you prepared for the next
several problems that may arise?

By the way, lcc also *appears* to change "...Module3" to "...Module4"
:)

Option 2: You have been helped already. There are two free compilers
which work quite well for developing extensions in C for win32
platforms; start reading here:

http://docs.python.org/inst/tweak-flags.html#SECTION000620000000000000000
 
F

Fredrik Lundh

John said:
By the way, lcc also *appears* to change "...Module3" to "...Module4" :)

$ more Include/modsupport.h

....

#define Py_InitModule3(name, methods, doc) \
Py_InitModule4(name, methods, doc, (PyObject *)NULL, \
PYTHON_API_VERSION)

....

</F>
 
J

Jean-Baptiste PERIN

That's got nothing to do with Python. You have to compile
extensions using a compiler that has an ABI that's compatible
with the ABI of the compiler used to compile Python.

You appear to be using a binary distribution of Python that was
compiled with MSVS 6.0, therefore, you have to compile
extensions with a compiler that has an ABI compatible with MSVS
6.0.

It's no use complaining to us bout your C compiler not being
ABI compatible with MSVS 6.0.

If you don't like MSVS, then your options are to either use a
different Python binary distro that was compiled with an ABI
that's compatible with your C compiler, or build just build it
yourself using whatever compiler you want. I suspect the latter
will also require re-compiling Blender.

According to what I read in this forum, MinGW is an interesting option..
do you know if it is ABI compatible with MSVC?

otherwise.. I'm gonna explore the solution which consist in building the
dll with cygwin's gcc .. I've already recompiled Blender with gcc .. I
know that, in Blender code, there are C++ function that are available
from Blender's python ..
I've started exploring the makefile.. apparently they are using dlltool
in conjonction with gcc ..

thank you very much for your help ..
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,733
Messages
2,569,439
Members
44,829
Latest member
PIXThurman

Latest Threads

Top