Ah, ctypes

[Lawrence D'Oliveiro]

I wrote some code months ago to output 1-bit-per-pixel PNG files from
Python, doing direct calls to libpng using ctypes, because PIL didn't give
me sufficient control over colour tables and pixel depths.

I thought the code was working fine. I left it aside for some months, came
back to it a week or two ago, and found it was crashing. I was creating
CFUNCTYPE objects to do callbacks to my own I/O routines, and with GDB I was
able to narrow down the crashes to the point where libpng was trying to
invoke one of my callbacks.

What had changed? I had switched my OS from Gentoo to Debian Unstable. I
chrooted back to the Gentoo system, tried my script again--still crashed.

Went back to the doc <http://docs.python.org/library/ctypes.html> and tried
the qsort callback example. Worked fine! And then I noticed this little bit:

Important note for callback functions:

Make sure you keep references to CFUNCTYPE objects as long as they are
used from C code. ctypes doesn’t, and if you don’t, they may be garbage
collected, crashing your program when a callback is made.

Yup, that was it. I changed my installation of the callbacks from

png.png_set_write_fn \
(
write_struct,
None,
ct.CFUNCTYPE(None, ct.c_void_p, ct.c_void_p, ct.c_size_t)(write_data),
ct.CFUNCTYPE(None, ct.c_void_p)(flush_write)
)

to

cb_write_data = ct.CFUNCTYPE(None, ct.c_void_p, ct.c_void_p, ct.c_size_t)(write_data)
cb_flush_write = ct.CFUNCTYPE(None, ct.c_void_p)(flush_write)
png.png_set_write_fn \
(
write_struct,
None,
cb_write_data,
cb_flush_write
)

and it works again.

 

[David Bolen]

ctypes could potentially note that function types don't have enough
references to them when passed in as arguments to C functions? It
might slow it down microscopically but it would fix this problem.

Except that ctypes can't know the lifetime needed for the callbacks. If
the callbacks are only used while the called function is executing (say,
perhaps for a progress indicator or internal completion callback) then
it's safe to create the function wrapper just within the function call.

-- David

 

[Lawrence D'Oliveiro]

As a ctypes user I found this an interesting story - thanks for
posting it!

By the way, I hate wildcard imports. In the ctypes docs, they recommend you
do this

from ctypes import *

but I prefer this:

import ctypes as ct

which explains the "ct." prefixes in my sample code, in case you were
wondering.