extending python with a C-written dll

Discussion in 'Python' started by Jean-Baptiste PERIN, Dec 20, 2004.

  1. Hi,

    I'm trying to make a windows dll reachable from a python script .. and
    I'm encountering troubles during link step ..

    I use "lcc" to compile and link
    I use python 2.3 under win XP

    Here's the compile command which seems to work well :
    -----------------------------------------------------
    lcclnk.exe -entry inithello -dll -o hello.dll hello.obj -L
    "C:\python23\libs" python23.lib


    Here's the link command which fails:
    ------------------------------------
    lcclnk.exe -entry inithello -dll -o hello.dll hello.obj -L
    "C:\python23\libs" python23.lib


    Here are the errors :
    ---------------------
    Error c:\...\plugins\hello.c 14 undefined reference to __imp__Py_BuildValue
    Error c:\...\plugins\hello.c 46 undefined reference to __imp__Py_InitModule4



    Here's the C code which compile but does not link :
    --------------------------------------------------

    #include <Python.h>

    PyObject *helloHello(PyObject *self, PyObject *args)
    {
    printf("Hello World!");
    return Py_BuildValue("i", 1);
    }

    PyObject *helloHelloString(PyObject *self, PyObject *args)
    {
    return Py_BuildValue("s", "Hello World!");
    }

    static char MHello__doc__[] = "This is a simple Python C extension example";
    static char Hello__doc__[] = "Prints in the console";
    static char HelloString__doc__[] = "Returns message as string";

    PyMethodDef helloMethods[] = {
    {"Hello", helloHello, METH_VARARGS, Hello__doc__},
    {"HelloString", helloHelloString, METH_VARARGS, HelloString__doc__},
    {NULL, NULL}
    };

    void __declspec(dllexport) inithello()
    {
    (void)Py_InitModule3("hello", helloMethods, MHello__doc__);
    }

    Can anyone help me?
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #1
    1. Advertising

  2. On 2004-12-20, Jean-Baptiste PERIN <> wrote:

    > I'm trying to make a windows dll reachable from a python script..


    FWIW, you can call dll's using the ctypes modules without
    mucking around in C. There may be performance reasons to build
    a "real" python module, but I haven't run across them yet.

    --
    Grant Edwards grante Yow! I KAISER ROLL?! What
    at good is a Kaiser Roll
    visi.com without a little COLE SLAW
    on the SIDE?
     
    Grant Edwards, Dec 20, 2004
    #2
    1. Advertising

  3. Grant Edwards a écrit :
    > On 2004-12-20, Jean-Baptiste PERIN <> wrote:
    >
    >
    >>I'm trying to make a windows dll reachable from a python script..

    >
    >
    > FWIW, you can call dll's using the ctypes modules without
    > mucking around in C. There may be performance reasons to build
    > a "real" python module, but I haven't run across them yet.
    >


    Not really a performance reason .. but mainly because I have a huge code
    produced in C and no time to port it in python ..
    Moreover .. I need the python program to interact with the C code
    ...(perfom calls to dll-defined functions)

    I don't know ctypes module .. do you really think it can help ?
    Do you have a link to quickly learn about it ?

    The only thing I found is :
    http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #3
  4. On 2004-12-20, Jean-Baptiste PERIN <> wrote:

    >>>I'm trying to make a windows dll reachable from a python script..

    >>
    >> FWIW, you can call dll's using the ctypes modules without
    >> mucking around in C. There may be performance reasons to build
    >> a "real" python module, but I haven't run across them yet.

    >
    > Not really a performance reason .. but mainly because I have a
    > huge code produced in C and no time to port it in python ..
    > Moreover .. I need the python program to interact with the C
    > code ..(perfom calls to dll-defined functions)
    >
    > I don't know ctypes module .. do you really think it can help?


    It lets you easily call functions in DLLs.

    > Do you have a link to quickly learn about it ?


    http://starship.python.net/crew/theller/ctypes/
    http://starship.python.net/crew/theller/ctypes/tutorial.html

    > The only thing I found is :
    > http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html


    I think that is unrelated.

    --
    Grant Edwards grante Yow! I HAVE a towel.
    at
    visi.com
     
    Grant Edwards, Dec 20, 2004
    #4
  5. Grant Edwards a écrit :

    > On 2004-12-20, Jean-Baptiste PERIN <> wrote:
    >
    >
    >>>>I'm trying to make a windows dll reachable from a python script..
    >>>
    >>>FWIW, you can call dll's using the ctypes modules without
    >>>mucking around in C. There may be performance reasons to build
    >>>a "real" python module, but I haven't run across them yet.

    >>
    >>Not really a performance reason .. but mainly because I have a
    >>huge code produced in C and no time to port it in python ..
    >>Moreover .. I need the python program to interact with the C
    >>code ..(perfom calls to dll-defined functions)
    >>
    >>I don't know ctypes module .. do you really think it can help?

    >
    >
    > It lets you easily call functions in DLLs.
    >
    >
    >>Do you have a link to quickly learn about it ?

    >
    >
    > http://starship.python.net/crew/theller/ctypes/
    > http://starship.python.net/crew/theller/ctypes/tutorial.html
    >
    >
    >>The only thing I found is :
    >>http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html

    >
    >
    > I think that is unrelated.
    >


    OK .. ctypes looks great .. and I plan to use it for futur purposes

    The fact is that the python interpreter I use is not a standard one
    It is a python interpreter delivered within a software named Blender.
    I don't know whether it is possible or not to add ctypes to it ..(I
    don't even have a python shell to perform the setup)

    I'm sure it is possible to use dll-built with it ..
    So please .. can you tell me where I can find Py_BuildValue and
    Py_InitModule4 to link with ?
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #5
  6. On 2004-12-20, Jean-Baptiste PERIN <> wrote:

    > OK .. ctypes looks great .. and I plan to use it for futur
    > purposes
    >
    > The fact is that the python interpreter I use is not a
    > standard one It is a python interpreter delivered within a
    > software named Blender. I don't know whether it is possible or
    > not to add ctypes to it ..(I don't even have a python shell to
    > perform the setup)
    >
    > I'm sure it is possible to use dll-built with it .. So please
    > .. can you tell me where I can find Py_BuildValue and
    > Py_InitModule4 to link with ?


    Sorry, I don't know anything about doing C lanugage extensions
    under Windows.

    --
    Grant Edwards grante Yow! Boy, am I glad it's
    at only 1971...
    visi.com
     
    Grant Edwards, Dec 20, 2004
    #6
  7. > The fact is that the python interpreter I use is not a standard one
    > It is a python interpreter delivered within a software named Blender.
    > I don't know whether it is possible or not to add ctypes to it ..(I
    > don't even have a python shell to perform the setup)
    >
    > I'm sure it is possible to use dll-built with it ..
    > So please .. can you tell me where I can find Py_BuildValue and
    > Py_InitModule4 to link with ?


    I doubt that adding new modules as dlls is easier than incorporating
    pure-python ones or a mixture of both - as ctypes most probably is. After
    all, its pythons module loading mechanism that kicks in for all cases.

    If ctypes suits your needs outside of blender, I'd go for trying to
    incorporate it into the blender interpreter.

    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Dec 20, 2004
    #7
  8. Jean-Baptiste PERIN <> writes:

    > Grant Edwards a écrit :
    >
    >> On 2004-12-20, Jean-Baptiste PERIN <> wrote:
    >>
    >>>>>I'm trying to make a windows dll reachable from a python script..
    >>>>
    >>>>FWIW, you can call dll's using the ctypes modules without
    >>>>mucking around in C. There may be performance reasons to build
    >>>>a "real" python module, but I haven't run across them yet.
    >>>
    >>>Not really a performance reason .. but mainly because I have a
    >>>huge code produced in C and no time to port it in python ..
    >>>Moreover .. I need the python program to interact with the C
    >>>code ..(perfom calls to dll-defined functions)
    >>>
    >>>I don't know ctypes module .. do you really think it can help?

    >> It lets you easily call functions in DLLs.
    >>
    >>>Do you have a link to quickly learn about it ?

    >> http://starship.python.net/crew/theller/ctypes/
    >> http://starship.python.net/crew/theller/ctypes/tutorial.html
    >>
    >>>The only thing I found is :
    >>>http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html

    >> I think that is unrelated.
    >>

    >
    > OK .. ctypes looks great .. and I plan to use it for futur purposes
    >
    > The fact is that the python interpreter I use is not a standard one
    > It is a python interpreter delivered within a software named Blender.
    > I don't know whether it is possible or not to add ctypes to it ..(I
    > don't even have a python shell to perform the setup)


    You could try to download the ctypes windows installer, use winzip or
    equivalent to unpack the contained files (you need _ctypes.pyd, and the
    ctypes directory containing __init__.py), and copy them to some
    directory where python can find them.

    Thomas
     
    Thomas Heller, Dec 20, 2004
    #8
  9. thanks for your valuable help .. ctypes is not lost for ever ..I'm
    going to explore a little bit .. it may suit my need in the end ..


    I had read :

    http://www.python.org/dev/doc/devel/ext/intro.html

    and eveything looked so simple in it ..
    I'm sad to realize that it only works for linux plateform :(

    I've tried to build and install ctypes from sources

    I ran the following command:

    >> python setup.py build


    and I got the following message:

    >> error: Python was built with version 6 of Visual Studio, and

    extensions need to
    be built with the same version of the compiler, but it isn't installed.


    :( :( :( :( So sad ....
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #9
  10. Diez B. Roggisch a écrit :

    >>The fact is that the python interpreter I use is not a standard one
    >>It is a python interpreter delivered within a software named Blender.
    >>I don't know whether it is possible or not to add ctypes to it ..(I
    >>don't even have a python shell to perform the setup)
    >>
    >>I'm sure it is possible to use dll-built with it ..
    >>So please .. can you tell me where I can find Py_BuildValue and
    >>Py_InitModule4 to link with ?

    >
    >
    > I doubt that adding new modules as dlls is easier than incorporating
    > pure-python ones or a mixture of both - as ctypes most probably is. After
    > all, its pythons module loading mechanism that kicks in for all cases.
    >
    > If ctypes suits your needs outside of blender, I'd go for trying to
    > incorporate it into the blender interpreter.
    >



    I've tried to build and install ctypes from sources

    I ran the following command:

    >> python setup.py build


    and I got the following message:

    >> error: Python was built with version 6 of Visual Studio, and

    extensions need to
    be built with the same version of the compiler, but it isn't installed.


    :( :( :( :( So sad ....
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #10
  11. Thomas Heller a écrit :

    > Jean-Baptiste PERIN <> writes:
    >
    >
    >>Grant Edwards a écrit :
    >>
    >>
    >>>On 2004-12-20, Jean-Baptiste PERIN <> wrote:
    >>>
    >>>
    >>>>>>I'm trying to make a windows dll reachable from a python script..
    >>>>>
    >>>>>FWIW, you can call dll's using the ctypes modules without
    >>>>>mucking around in C. There may be performance reasons to build
    >>>>>a "real" python module, but I haven't run across them yet.
    >>>>
    >>>>Not really a performance reason .. but mainly because I have a
    >>>>huge code produced in C and no time to port it in python ..
    >>>>Moreover .. I need the python program to interact with the C
    >>>>code ..(perfom calls to dll-defined functions)
    >>>>
    >>>>I don't know ctypes module .. do you really think it can help?
    >>>
    >>>It lets you easily call functions in DLLs.
    >>>
    >>>
    >>>>Do you have a link to quickly learn about it ?
    >>>
    >>>http://starship.python.net/crew/theller/ctypes/
    >>>http://starship.python.net/crew/theller/ctypes/tutorial.html
    >>>
    >>>
    >>>>The only thing I found is :
    >>>>http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html
    >>>
    >>>I think that is unrelated.
    >>>

    >>
    >>OK .. ctypes looks great .. and I plan to use it for futur purposes
    >>
    >>The fact is that the python interpreter I use is not a standard one
    >>It is a python interpreter delivered within a software named Blender.
    >>I don't know whether it is possible or not to add ctypes to it ..(I
    >>don't even have a python shell to perform the setup)

    >
    >
    > You could try to download the ctypes windows installer, use winzip or
    > equivalent to unpack the contained files (you need _ctypes.pyd, and the
    > ctypes directory containing __init__.py), and copy them to some
    > directory where python can find them.
    >
    > Thomas


    I took a look at the ctypes tutorial .. Here's what I saw ..

    >>> from ctypes import *
    >>> printf = cdll.msvcrt.printf
    >>> printf.argtypes = [c_char_p, c_char_p, c_int, c_double]
    >>> printf("String '%s', Int %d, Double %f\n", "Hi", 10, 2.2)


    just to perfom a printf !!!

    now look at http://www.python.org/dev/doc/devel/ext/intro.html

    and look at the expected result .. it is far simpler from the python
    side view

    I think ctype is uselful when dealing with dll you're not the author of

    But if noone can help me to extend python with my own dll .. I going to
    explore ctypes .. but I find it's bothering .. and frustrating not to
    achieve what I wanted to do ..

    Moreover .. I tried to build and install ctypes from source ..
    I ran the command

    >> python setup.py build


    and I got the following message:

    >> error: Python was built with version 6 of Visual Studio, and

    extensions need to
    be built with the same version of the compiler, but it isn't installed.


    SO SAD :( :( :(
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #11
  12. > and look at the expected result .. it is far simpler from the python
    > side view


    But you have to do all the C-Work!

    >
    > I think ctype is uselful when dealing with dll you're not the author of


    It is useful, if you don't want to write an extension for the extra work
    that is needed for that - for example if you are missing a C compiler.

    > >> error: Python was built with version 6 of Visual Studio, and

    > extensions need to
    > be built with the same version of the compiler, but it isn't installed.


    How do you expect to be able to write your own extension if you don't have
    the environment to do so in the first place? that error has nothing to do
    with python itself, but with you not having installed the right compiler.
    THe same problem will arise building your own extension.

    Others suggested trying the ctypes binary installer. Do so.
    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Dec 20, 2004
    #12
  13. Jean-Baptiste PERIN wrote:
    > Hi,
    >
    > I'm trying to make a windows dll reachable from a python script .. and
    > I'm encountering troubles during link step ..
    >
    > I use "lcc" to compile and link
    > I use python 2.3 under win XP


    Note that MinGW32 is a tested path for building (free) C extensions. I
    don't know if anyone has a tested way of using lcc.

    With MinGW32, your build step will be the delightfully easy:

    python setup.py build --compiler=mingw32

    --Scott David Daniels
     
    Scott David Daniels, Dec 20, 2004
    #13

  14. >
    >> >> error: Python was built with version 6 of Visual Studio, and

    >>extensions need to
    >>be built with the same version of the compiler, but it isn't installed.

    >
    >
    > How do you expect to be able to write your own extension if you don't have
    > the environment to do so in the first place? that error has nothing to do
    > with python itself, but with you not having installed the right compiler.
    > THe same problem will arise building your own extension.


    I know it has nothing to do with python ..
    It just makes me sad to see that python cannot get rid of Visual Studoi
    to be extended . It's just amazing !!

    Python is free
    The compiler I use is free
    The code I plan to produce will be free

    Where can I find a free version of Visual Studio ?

    the only thing I use that is not free is the operating system and it's
    because I had no choice .. my company decided for it

    > Others suggested trying the ctypes binary installer. Do so.


    Yes .. that's what I'm going to do .. but I find it frustrating ..
     
    Jean-Baptiste PERIN, Dec 20, 2004
    #14
  15. Jean-Baptiste PERIN

    Steve Holden Guest

    Jean-Baptiste PERIN wrote:

    >
    >>
    >>> >> error: Python was built with version 6 of Visual Studio, and
    >>> extensions need to
    >>> be built with the same version of the compiler, but it isn't installed.

    >>
    >>
    >>
    >> How do you expect to be able to write your own extension if you don't
    >> have
    >> the environment to do so in the first place? that error has nothing to do
    >> with python itself, but with you not having installed the right compiler.
    >> THe same problem will arise building your own extension.

    >
    >
    > I know it has nothing to do with python ..
    > It just makes me sad to see that python cannot get rid of Visual Studoi
    > to be extended . It's just amazing !!
    >
    > Python is free
    > The compiler I use is free
    > The code I plan to produce will be free
    >
    > Where can I find a free version of Visual Studio ?
    >
    > the only thing I use that is not free is the operating system and it's
    > because I had no choice .. my company decided for it
    >
    >> Others suggested trying the ctypes binary installer. Do so.

    >
    >
    > Yes .. that's what I'm going to do .. but I find it frustrating ..


    Perhaps you aren't aware that Microsoft make a free tool chain available
    that can be used to create binary extensions? There has been some
    discussion of this recently, and I managed to compile MySQLdb and a
    couple of the mx extensions, so it's eminently doable [the PIL is
    proving a little less tractable due to dependencies on other libraries
    that I have yet to resolve].

    The downloads are quite large, but if you want to try it, take a look at

    http://www.vrplumber.com/programming/mstoolkit/

    regards
    Steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
     
    Steve Holden, Dec 20, 2004
    #15
  16. On 2004-12-20, Jean-Baptiste PERIN <> wrote:

    > I know it has nothing to do with python ..
    > It just makes me sad to see that python cannot get rid of Visual Studoi
    > to be extended.


    That's got nothing to do with Python. You have to compile
    extensions using a compiler that has an ABI that's compatible
    with the ABI of the compiler used to compile Python.

    You appear to be using a binary distribution of Python that was
    compiled with MSVS 6.0, therefore, you have to compile
    extensions with a compiler that has an ABI compatible with MSVS
    6.0.

    It's no use complaining to us bout your C compiler not being
    ABI compatible with MSVS 6.0.

    If you don't like MSVS, then your options are to either use a
    different Python binary distro that was compiled with an ABI
    that's compatible with your C compiler, or build just build it
    yourself using whatever compiler you want. I suspect the latter
    will also require re-compiling Blender.

    --
    Grant Edwards grante Yow! Yow! Are we in the
    at perfect mood?
    visi.com
     
    Grant Edwards, Dec 20, 2004
    #16
  17. First, you might check the calldll module available here

    <http://www.nightmare.com/software.html>

    Take a look at Pyrex, it may fit your requirements to access C from
    Python.

    <http://nz.cosc.canterbury.ac.nz/~greg/python/Pyrex/>

    If you do not have these already, you may need the Win32 API extension

    <http://www.python.org/windows/win32/>


    /Jean Brouwers



    In article <cq6qi8$hk9$>, Jean-Baptiste PERIN
    <> wrote:

    > Grant Edwards a écrit :
    > > On 2004-12-20, Jean-Baptiste PERIN <> wrote:
    > >
    > >
    > >>I'm trying to make a windows dll reachable from a python script..

    > >
    > >
    > > FWIW, you can call dll's using the ctypes modules without
    > > mucking around in C. There may be performance reasons to build
    > > a "real" python module, but I haven't run across them yet.
    > >

    >
    > Not really a performance reason .. but mainly because I have a huge code
    > produced in C and no time to port it in python ..
    > Moreover .. I need the python program to interact with the C code
    > ..(perfom calls to dll-defined functions)
    >
    > I don't know ctypes module .. do you really think it can help ?
    > Do you have a link to quickly learn about it ?
    >
    > The only thing I found is :
    > http://www.python.org/workshops/1994-11/BuiltInClasses/BuiltInClasses_5.html
    >
    >
    >
    >
    >
     
    Jean Brouwers, Dec 20, 2004
    #17
  18. Jean-Baptiste PERIN

    John Machin Guest

    Jean-Baptiste PERIN wrote:
    > Hi,
    >
    > I'm trying to make a windows dll reachable from a python script ..

    and
    > I'm encountering troubles during link step ..
    >
    > I use "lcc" to compile and link
    > I use python 2.3 under win XP

    [snip]
    > Here are the errors :
    > ---------------------
    > Error c:\...\plugins\hello.c 14 undefined reference to

    __imp__Py_BuildValue
    > Error c:\...\plugins\hello.c 46 undefined reference to

    __imp__Py_InitModule4
    [snip]
    > return Py_BuildValue("i", 1);

    [snip]
    > void __declspec(dllexport) inithello()
    > {
    > (void)Py_InitModule3("hello", helloMethods, MHello__doc__);
    > Can anyone help me?


    Yes. Option 1: *Maybe* you can help yourself. Have you not noticed from
    the above that the compiler is magically prepending "__imp__" to the
    names of external functions? However the .lib that comes with Python
    won't have that "__imp__". You would have to bridge that gap, somehow.
    Have you searched the lcc documentation? Are you prepared for the next
    several problems that may arise?

    By the way, lcc also *appears* to change "...Module3" to "...Module4"
    :)

    Option 2: You have been helped already. There are two free compilers
    which work quite well for developing extensions in C for win32
    platforms; start reading here:

    http://docs.python.org/inst/tweak-flags.html#SECTION000620000000000000000
     
    John Machin, Dec 20, 2004
    #18
  19. John Machin wrote:

    > By the way, lcc also *appears* to change "...Module3" to "...Module4" :)


    $ more Include/modsupport.h

    ....

    #define Py_InitModule3(name, methods, doc) \
    Py_InitModule4(name, methods, doc, (PyObject *)NULL, \
    PYTHON_API_VERSION)

    ....

    </F>
     
    Fredrik Lundh, Dec 20, 2004
    #19

  20. >
    > That's got nothing to do with Python. You have to compile
    > extensions using a compiler that has an ABI that's compatible
    > with the ABI of the compiler used to compile Python.
    >
    > You appear to be using a binary distribution of Python that was
    > compiled with MSVS 6.0, therefore, you have to compile
    > extensions with a compiler that has an ABI compatible with MSVS
    > 6.0.
    >
    > It's no use complaining to us bout your C compiler not being
    > ABI compatible with MSVS 6.0.
    >
    > If you don't like MSVS, then your options are to either use a
    > different Python binary distro that was compiled with an ABI
    > that's compatible with your C compiler, or build just build it
    > yourself using whatever compiler you want. I suspect the latter
    > will also require re-compiling Blender.
    >


    According to what I read in this forum, MinGW is an interesting option..
    do you know if it is ABI compatible with MSVC?

    otherwise.. I'm gonna explore the solution which consist in building the
    dll with cygwin's gcc .. I've already recompiled Blender with gcc .. I
    know that, in Blender code, there are C++ function that are available
    from Blender's python ..
    I've started exploring the makefile.. apparently they are using dlltool
    in conjonction with gcc ..

    thank you very much for your help ..
     
    Jean-Baptiste PERIN, Dec 21, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Anushi
    Replies:
    5
    Views:
    8,781
  2. Steve C. Orr, MCSD
    Replies:
    1
    Views:
    563
    reaway lee
    Aug 24, 2003
  3. Lars Moastuen
    Replies:
    3
    Views:
    521
    Lars Moastuen
    Nov 26, 2004
  4. KaiWen
    Replies:
    102
    Views:
    2,738
    Jorgen Grahn
    Sep 15, 2011
  5. Marc Heiler
    Replies:
    1
    Views:
    137
    Stefan Lang
    Jan 26, 2008
Loading...

Share This Page