Calling Python macro from ctypes

S

Steven D'Aprano

Is it possible to call a Python macro from ctypes? For example, Python
3.3 introduces some new macros for querying the internal representation
of strings:

http://www.python.org/dev/peps/pep-0393/#new-api


So I try this in 3.3:

py> import ctypes
py> ctypes.pythonapi.PyUnicode_MAX_CHAR_VALUE
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.3/ctypes/__init__.py", line 366, in
__getattr__
func = self.__getitem__(name)
File "/usr/local/lib/python3.3/ctypes/__init__.py", line 371, in
__getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python3.3: undefined symbol: PyUnicode_MAX_CHAR_VALUE
 
P

Peter Otten

Steven said:
Is it possible to call a Python macro from ctypes? For example, Python
3.3 introduces some new macros for querying the internal representation
of strings:

http://www.python.org/dev/peps/pep-0393/#new-api


So I try this in 3.3:

py> import ctypes
py> ctypes.pythonapi.PyUnicode_MAX_CHAR_VALUE
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.3/ctypes/__init__.py", line 366, in
__getattr__
func = self.__getitem__(name)
File "/usr/local/lib/python3.3/ctypes/__init__.py", line 371, in
__getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python3.3: undefined symbol: PyUnicode_MAX_CHAR_VALUE

That's not possible. It may look like a function, but a preprocessor
replaces the C macro in the C source before compilation. An example of very
bad usage of macros, just to drive the point home:

$ cat macro.c
#define IF(expr) if (expr) {
#define ENDIF ;}

main()
{
IF(1>0)
printf("It worked\n")
ENDIF
}

And here's what the compiler sees:

$ gcc -E -P macro.c



main()
{
if (1>0) {
printf("It worked\n")
;}
}
 
D

Dave Angel

Peter said:
That's not possible. It may look like a function, but a preprocessor
replaces the C macro in the C source before compilation. An example of very
bad usage of macros, just to drive the point home:

$ cat macro.c
#define IF(expr) if (expr) {
#define ENDIF ;}

main()
{
IF(1>0)
printf("It worked\n")
ENDIF
}

And here's what the compiler sees:

$ gcc -E -P macro.c



main()
{
if (1>0) {
printf("It worked\n")
;}
}

To elaborate a bit more, Python can only see those symbols that are put
into the shared library They can be functions, and they can be
"values," but they don't include macros, which are processed by the
preprocessor, before the real C compiler even starts. C Macros are
actually text-substitution rules. They can look like functions, but
those functions do not end up in the shared library.

In Windows, you can use dumpbin to examine a DLL and see what symbols it
exports. I don't remember the syntax; it's been years.

I assume there's a similar tool for Linux to examine a shared library
(typically an .so file). Perhaps "readelf" and/or "nm" is such a tool,
but I don't really know. Although I've been using Python and C++ in
Linux in recent years, I haven't used them together, and neither have
I had to examine a shared library.

The following link looks interesting, but I haven't read it yet.

http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
 
S

Steven D'Aprano

Steven said:
Is it possible to call a Python macro from ctypes? For example, Python
3.3 introduces some new macros for querying the internal representation
of strings:

http://www.python.org/dev/peps/pep-0393/#new-api
[...]

That's not possible. It may look like a function, but a preprocessor
replaces the C macro in the C source before compilation.

That's what I feared.

In that case, how would I use ctypes to access the underlying fields in
the new string implementation?
 
S

Stefan Behnel

Steven D'Aprano, 13.08.2013 08:25:
Steven said:
Is it possible to call a Python macro from ctypes? For example, Python
3.3 introduces some new macros for querying the internal representation
of strings:

http://www.python.org/dev/peps/pep-0393/#new-api
[...]

That's not possible. It may look like a function, but a preprocessor
replaces the C macro in the C source before compilation.

That's what I feared.

In that case, how would I use ctypes to access the underlying fields in
the new string implementation?

I'd personally use Cython (no surprise here), but in order to use something
like ctypes, which works at the ABI level, not the API level, and doesn't
use a C compiler to get things properly configured for the local platform,
you'd have to manually define the actual PyObject struct in order to access
its fields directly.

The macros are there to give you source code level portability for that,
but if you start defining the struct layout statically, you're pretty much
on your own when it comes to stuff like different CPython versions, debug
builds, etc.

Could you describe your use case a little deeper? Maybe there's a better
way all together to do what you want.

Stefan
 
S

Steven D'Aprano

Steven D'Aprano, 13.08.2013 08:25:
Steven D'Aprano wrote:

Is it possible to call a Python macro from ctypes? For example,
Python 3.3 introduces some new macros for querying the internal
representation of strings:

http://www.python.org/dev/peps/pep-0393/#new-api [...]

That's not possible. It may look like a function, but a preprocessor
replaces the C macro in the C source before compilation.

That's what I feared.

In that case, how would I use ctypes to access the underlying fields in
the new string implementation?

I'd personally use Cython (no surprise here), but in order to use
something like ctypes, which works at the ABI level, not the API level,
and doesn't use a C compiler to get things properly configured for the
local platform, you'd have to manually define the actual PyObject struct
in order to access its fields directly.

The macros are there to give you source code level portability for that,
but if you start defining the struct layout statically, you're pretty
much on your own when it comes to stuff like different CPython versions,
debug builds, etc.

Could you describe your use case a little deeper? Maybe there's a better
way all together to do what you want.

One concrete example of what I want to do is introspect Python 3.3
strings. E.g. to tell whether they are 1 byte, 2 byte or 4 byte strings.
Strings have a "kind" field, set to 0-3, that specifies how many bytes
per char, but it is not exposed to Python.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top