IMPORTANT 2.5 API changes for C Extension Modules and Embedders

N

Neal Norwitz

If you don't write or otherwise maintain Python Extension Modules
written in C (or C++) or embed Python in your application,
you can stop reading.

Python 2.5 alpha 1 was released April 5, 2006. The second alpha
should be released in a few weeks. There are several changes
which can cause C extension modules or embedded applications
to crash the interpreter if not fixed. Periodically, I will send out
these reminders with updated information until 2.5 is released.

* support for 64-bit sequences (eg, > 2GB strings)
* memory allocation modifications

64-bit changes
--------------
There are important changes that are in 2.5 to support 64-bit systems.
The 64-bit changes can cause Python to crash if your module is not upgraded
to support the changes. Python was changed internally to use 64-bit
values on 64-bit machines for indices. If you've got a machine with
more than 16 GB of RAM, it would be great if you can test Python with
large (> 2GB) strings and other sequences.

For more details about the Python 2.5 schedule:
http://www.python.org/dev/peps/pep-0356/
For more details about the 64-bit change:
http://www.python.org/dev/peps/pep-0353/
How to fix your module:
http://www.python.org/dev/peps/pep-0353/#conversion-guidelines

The effbot wrote a program to check your code and find potential
problems with the 64-bit APIs.
http://svn.effbot.python-hosting.com/stuff/sandbox/python/ssizecheck.py

Memory Allocation Modifications
-------------------------------
In previous versions of Python, it was possible to use different
families of APIs (PyMem_* vs. PyObject_*) to allocate and free
the same block of memory. APIs in these families include:

PyMem_*: PyMem_Malloc, PyMem_Realloc, PyMem_Free,
PyObject_*: PyObject_Malloc, PyObject_Realloc, PyObject_Free

There are a few other APIs with similar names and also the macro variants.

In 2.5, if allocate a block of memory with one family, you must reallocate
or free with the same family. That means:

If you allocate with PyMem_Malloc (or MALLOC), you must reallocate
with PyMem_Realloc (or REALLOC) and free with PyMem_Free (or FREE).
If you allocate with PyObject_Malloc (or MALLOC), you must reallocate
with PyObject_Realloc (or REALLOC) and free with PyObject_Free (or FREE).

Using inconsistent APIs can cause double frees or otherwise crash
the interpreter. It is fine to mix and match functions or macros
within the same family.

Please test and upgrade your extension modules!

Cheers,
n
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,800
Messages
2,569,657
Members
45,415
Latest member
KarolDicke
Top