Undefined calling conventions in Python.h

[Jaco Naude]

Hi there,

This is my first post over here and I hope someone can give me some
guidance.

I'm trying to embed Python into a Visual C++ 2008 application and I'm
getting linker problems. I've compiled a DLL of the Python source code
using the pythoncode VC++ project in the PCbuild folder of the source
download and this works 100% without any warnings etc. I've done this
in Debug and Release mode without any problems.

When I include python_install_path\include\Python.h in my application
(with the linking setup correctly to the .lib and .dll files generated
by myself) it builds fine. However as soon as I try to call any Python
function (Py_Exit for example) I get linker errors as shown below:

1>application.obj : error LNK2031: unable to generate p/invoke for
"extern "C" void __clrcall Py_Exit(int)" (?Py_Exit@@$$J0YMXH@Z);
calling convention missing in metadata
1>frmPythonInterface.obj : error LNK2031: unable to generate p/invoke
for "extern "C" void __clrcall Py_Exit(int)" (?Py_Exit@@$$J0YMXH@Z);
calling convention missing in metadata

I'm probably missing something but I can't find any calling convention
details in Python.h or the other headers included in this file. In my
VC++ DLL project I've specified the calling convention as __stdcall. I
know the __clrcall naming convention has to do with managed code in VC+
+ but thats as far as my knowledge on that goes. Should I define the
calling conventions manually? Or is there a setting wrong somewhere in
my application project?

If anybody can give some guidance it would be greatly appreciated.

Thanks
Jaco

 

[Fredrik Lundh]

1>application.obj : error LNK2031: unable to generate p/invoke for
"extern "C" void __clrcall Py_Exit(int)" (?Py_Exit@@$$J0YMXH@Z);
calling convention missing in metadata
1>frmPythonInterface.obj : error LNK2031: unable to generate p/invoke
for "extern "C" void __clrcall Py_Exit(int)" (?Py_Exit@@$$J0YMXH@Z);
calling convention missing in metadata

I'm probably missing something but I can't find any calling convention
details in Python.h or the other headers included in this file.

the precence of name mangling indicates that your compiler doesn't
understand that Python's a C library, and therefore uses C calling
conventions. have you managed to override the defaults in some odd
way?

</F>

 

[Jaco Naude]

the precence of name mangling indicates that your compiler doesn't
understand that Python's a C library, and therefore uses C calling
conventions.  have you managed to override the defaults in some odd
way?

</F>

good point. I agree that the problem is probably due to name mangling.
I'm not sure how its possible to tell the application that the DLL is
a C dll? I've looked at the DLL using Dependency Walker and the
functions in the DLL are clean (Thus no name mangling used).

How do I tell Visual C++ that the DLL is a C dll? I thought it should
be in Python.h but this line appears at the top of the file:

/* Since this is a "meta-include" file, no #ifdef __cplusplus / extern
"C" { */

Thanks for the help
Jaco

 

[Fredrik Lundh]

good point. I agree that the problem is probably due to name mangling.
I'm not sure how its possible to tell the application that the DLL is
a C dll? I've looked at the DLL using Dependency Walker and the
functions in the DLL are clean (Thus no name mangling used).
>
How do I tell Visual C++ that the DLL is a C dll? I thought it should
be in Python.h but this line appears at the top of the file:

/* Since this is a "meta-include" file, no #ifdef __cplusplus / extern
"C" { */

All the individual includes are wrapped in the usual

#ifdef __cplusplus
extern "C" {
#endif

stuff, so the compiler should default to C bindings, without you having
to do anything. Unfortunately, I haven't used VS 2008, but I find it
hard to believe that they've messed this up completely, and a quick
googling indicates that people are using it with Python without trouble.

Maybe there's some override in your project settings caused by some
other parts of your project? (or an #undef __cplusplus somewhere, perhaps?)

</F>

 

[Jaco Naude]

All the individual includes are wrapped in the usual

     #ifdef __cplusplus
     extern "C" {
     #endif

stuff, so the compiler should default to C bindings, without you having
to do anything.  Unfortunately, I haven't used VS 2008, but I find it
hard to believe that they've messed this up completely, and a quick
googling indicates that people are using it with Python without trouble.

Maybe there's some override in your project settings caused by some
other parts of your project?  (or an #undef __cplusplus somewhere, perhaps?)

</F>

I've figured out that the names in the DLL are not mangled by looking
into the DLL using dependancy walker. I also figured out that the
problem is on the application's side and not on the dll's side.

What Visual C++ is doing is that it is looking for mangled names since
it does not know the DLL contains C functions. I've managed to work
around this by declaring the Python functions as follows before using
them in the C++ application side:

extern "C"
{
void Py_Initialize(void);
}

This seems to work and the C++ application side is not looking for
mangled names any more. Is this the right way of doing it? It seems
unnecessary to have to declare each Python function you want to use
using the extern "C" way as shown above.

It is probably more of a C++ question it turns out, but I would think
that someone in the Python group would use the Python DLL in C++. The
documentation also suggest that there is no extra work needed when
using C++ rather than C.

Thanks for the help so far.
Jaco

 

[Fredrik Lundh]

What Visual C++ is doing is that it is looking for mangled names since
it does not know the DLL contains C functions. I've managed to work
around this by declaring the Python functions as follows before using
them in the C++ application side:

extern "C"
{
void Py_Initialize(void);
}

This seems to work and the C++ application side is not looking for
mangled names any more. Is this the right way of doing it? It seems
unnecessary to have to declare each Python function you want to use
using the extern "C" way as shown above.

Eh, are you saying that you're not including the Python.h file? Because
it does exactly that, for each and every public function in the C API.

It is probably more of a C++ question it turns out, but I would think
that someone in the Python group would use the Python DLL in C++. The
documentation also suggest that there is no extra work needed when
using C++ rather than C.

Oh, but I do that all the time, without doing any extra work. Both
embedding Python in C++ programs and existing it with C++ extensions.
And I'm definitely not alone.

Here's an actual session, using the version of Visual Studio I happen to
have on this machine (2003, I think) from the command line:

> more test.cc

#include "Python.h"
#include <iostream>

main()
{
Py_Initialize();
PyRun_SimpleString("print 'hello'\n");
Py_Finalize();

std::cout << "world\n";
}

> cl cl -EHsc -MD -I \python25\include test.cc \python25\libs\python25.lib

Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077
for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

....

> test

hello
world

If you cannot get the same console program to work in your compiler
setup, something's wrong with your configuration.

</F>

 

[Fredrik Lundh]

cut and paste error there: the "cl cl" should be just one "cl", of course.

and just for the record/google, if I

1) don't include the header file, I get

test.cc(5) : error C3861: 'Py_Initialize': identifier not found, even
with argument-dependent lookup

2) attempt to undef the __cplusplus macro, I get

test.cc(1) : warning C4117: macro name '__cplusplus' is reserved,
'#undef' ignored

3) cut and paste declarations from the header files to my own file
instead of including the files, ignoring the extern "C" part, I get

test.obj : error LNK2019: unresolved external symbol "int __cdecl
Py_Finalize(void)" (?Py_Finalize@@YAHXZ) referenced in function _main

which looks pretty similar to the errors posted earlier.

</F>

 

[Jaco Naude]

Eh, are you saying that you're not including the Python.h file?  Because
it does exactly that, for each and every public function in the C API.


Oh, but I do that all the time, without doing any extra work.  Both
embedding Python in C++ programs and existing it with C++ extensions.
And I'm definitely not alone.

Here's an actual session, using the version of Visual Studio I happen to
have on this machine (2003, I think) from the command line:

 > more test.cc

#include "Python.h"
#include <iostream>

main()
{
     Py_Initialize();
     PyRun_SimpleString("print 'hello'\n");
     Py_Finalize();

     std::cout << "world\n";

}

 > cl cl -EHsc -MD -I \python25\include test.cc \python25\libs\python25..lib
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077
for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.

...

 > test
hello
world

If you cannot get the same console program to work in your compiler
setup, something's wrong with your configuration.

</F>

Ok that's probably good news, although it points out that there is
something wrong with my configuration since that does not work. I
would rather sort out the problem that having to defined each function
with a extern "C" command.

That said, let me double check something which might be causing
problems since you will be familiar with this. Which Python.h file do
you include when including the DLL in your programs? The one in the
source distribution of the one in the installation distribution? I've
been including the one in the installation distribution all along.
When I try to include the one in the source distribution it gets up to
the point where it looks for the following include:

#include "pyconfig.h"

This file is not in the same directory as the Python.h file in the
source distribution. Because of this I just used Python.h in the
installation distribution.

Thanks again,
Jaco

 

[Fredrik Lundh]

That said, let me double check something which might be causing
problems since you will be familiar with this. Which Python.h file do
you include when including the DLL in your programs? The one in the
source distribution of the one in the installation distribution? I've
been including the one in the installation distribution all along.
When I try to include the one in the source distribution it gets up to
the point where it looks for the following include:

#include "pyconfig.h"

This file is not in the same directory as the Python.h file in the
source distribution. Because of this I just used Python.h in the
installation distribution.

that's just two copies of the same file, as far as I know.

the "pyconfig.h" file contains platform-specific information; it's
generated from pyconfig.h.in on Unix-style systems, and copied from the
PC directory on Windows.

</F>

 

[Jaco Naude]

Ok that's probably good news, although it points out that there is
something wrong with my configuration since that does not work. I
would rather sort out the problem that having to defined each function
with a extern "C" command.

That said, let me double check something which might be causing
problems since you will be familiar with this. Which Python.h file do
you include when including the DLL in your programs? The one in the
source distribution of the one in the installation distribution? I've
been including the one in the installation distribution all along.
When I try to include the one in the source distribution it gets up to
the point where it looks for the following include:

#include "pyconfig.h"

This file is not in the same directory as the Python.h file in the
source distribution. Because of this I just used Python.h in the
installation distribution.

Thanks again,
Jaco

I only saw your last message after posting my previous one. Ok, so the
problem is definitely with the include file Python.h. As I said in my
last post, I might be including the wrong one. I would appreciate it
if you can tell me which one it the correct one to use: The one in the
source distribution or the one in the release distribution
(installation done from the .msi file).

Thanks
Jaco

 

[Ben Sizer]

What Visual C++ is doing is that it is looking for mangled names since
it does not know the DLL contains C functions. I've managed to work
around this by declaring the Python functions as follows before using
them in the C++ application side:

extern "C"
{
    void Py_Initialize(void);

}

You should put the extern block around the #include <python.h> call
rather than individual functions, as surely the C calling convention
should apply to everything within.

It is probably more of a C++ question it turns out, but I would think
that someone in the Python group would use the Python DLL in C++.

More of a Visual C++ question specifically, since the __clrcall prefix
is a MS specific extension (http://msdn.microsoft.com/en-us/library/
ec7sfckb(VS.80).aspx). If you're not using managed code in your app,
disable it in the project/build options. If you are, then perhaps you
just need to specify that you're not with this DLL, though I've never
had to deal with anything like that myself.

 

[Fredrik Lundh]

You should put the extern block around the #include <python.h> call
rather than individual functions, as surely the C calling convention
should apply to everything within.

Hello? Python's include files are C++ safe. I even posted a complete
compiler session to show that I'm not making that up.

</F>

 

[Jaco Naude]

You should put the extern block around the #include <python.h> call
rather than individual functions, as surely the C calling convention
should apply to everything within.


More of a Visual C++ question specifically, since the __clrcall prefix
is a MS specific extension (http://msdn.microsoft.com/en-us/library/
ec7sfckb(VS.80).aspx). If you're not using managed code in your app,
disable it in the project/build options. If you are, then perhaps you
just need to specify that you're not with this DLL, though I've never
had to deal with anything like that myself.

Fredrik, thanks for the help. I'm not sure why but it seems to work
now even if I don't include the extern "C" command. It also works with
both Python.h files (after I copied pyconfig.h from the PC folder). So
it seems like everything is working now.

Ben, Thanks for the reply. Good suggestion to place the extern "C"
around the include. I will remember that. It turns out that it works
without that as well in the end. As for the question on managed code:
My application do use managed code. I've been able to turn this off
when I created the Python DLL and it seems to work.

Thanks for all the help,
All the best
Jaco

 

[Fredrik Lundh]

Fredrik, thanks for the help. I'm not sure why but it seems to work
now even if I don't include the extern "C" command. It also works with
both Python.h files (after I copied pyconfig.h from the PC folder). So
it seems like everything is working now.

As it's supposed to do. It's times like this that you really want to
use a system that takes snapshots of every single revision of the files
you work on, so you can see what exactly it was you had done when it
didn't work (it's more often a "do'h" thing than a heisenbug ;-).

(fwiw, I'm currently trying to track down a problem at a customer site
when the entire Python application managed to remove itself during a
test run; the program crashed with "zipimport: IOError" when they tried
to use a part of the system that did some lazy imports, and when they
tried to restart the program, neither the startup script nor the ZIP
archive that contained the bulk of the application were anywhere to be
seen. Other PY files in the same directory were left intact. Fully
patched Windows XP, clean bill of health from a virus scanner. If
anyone's ever experienced anything similar, let me know.)

</F>

 

[Martin v. Löwis]

1>application.obj : error LNK2031: unable to generate p/invoke for

"extern "C" void __clrcall Py_Exit(int)" (?Py_Exit@@$$J0YMXH@Z);
calling convention missing in metadata

The main problem here is the __clrcall hint: apparently, you are
using Managed C++ resp. C++/CLI, i.e. the Microsoft .NET Framework for
C++.

Don't do that.

Instead, make sure that your project targets native Intel x86 code;
then compile your code either as C or C++ (your choice). To integrate
Python into managed C++ would require a lot of experience, and it may
well not be possible.

Regards,
Martin

 

[Ben Sizer]

Hello?  Python's include files are C++ safe.  I even posted a complete
compiler session to show that I'm not making that up.

</F>

In theory, yeah. In practice, if his compiler was somehow not
respecting that, then a quicker fix is to enclose the #include than to
do individual prototypes. Admittedly that might obscure the problem
rather than solve it.

 

[Fredrik Lundh]

In theory, yeah. In practice, if his compiler was somehow not
respecting that, then a quicker fix is to enclose the #include than to
do individual prototypes. Admittedly that might obscure the problem
rather than solve it.

Well, I'd say that the should in

You should put the extern block around the #include <python.h> call
rather than individual functions, as surely the C calling convention
should apply to everything within.

makes it look way too much like "this is how you should use python.h" to
be appropriate, given the earlier discussions in the thread, and that a
casual inspection of the include files reveals that the included files
contain exactly that thing.

C compilers can be pretty stupid, indeed, but it's usually better to
look for a solution that makes some sense (e.g. the wrong project
settings) before blaming it all on deeper magic (it will look at this
code if I put it here, but not if I put it there. hmm. must be a ley
line over there. what if I add a comment?).

</F>