extension call backs and the GIL

R

Robin Becker

I'm trying to better my understanding of multithreading. I believe a
thread that calls an extension must have the GIL. What happens if the
extension then calls back into Python? Can a function or method somehow
'belong' to a different thread? My understanding was that things should
be OK provided the extension code doesn't mess with the GIL (which it
doesn't).

I'm asking because the ReportLab tool kit contains an accelerator for a
stringWidth function that relies on a callback to fill in some global
data upon demand. A user reports thread problems when using the
accelerator.
 
A

Aahz

I'm trying to better my understanding of multithreading. I believe a
thread that calls an extension must have the GIL. What happens if the
extension then calls back into Python? Can a function or method somehow
'belong' to a different thread? My understanding was that things should
be OK provided the extension code doesn't mess with the GIL (which it
doesn't).

You need to reacquire the GIL prior to making the callback.
 
B

Bernhard Herzog

Robin Becker said:
So does calling an extension release the GIL?

No. It would be very inconvenient if it did because what most functions
do first is to call one of the PyArg_ParseTuple functions which could
execute arbitrary python code.

I think it would help if you described a bit more what the actual
problem is or what the user does.

I looked briefly at the sources and assuming tha the accelerator in
question is lib/_rl_accel.c and the callback whatever has been assigned
to _SWRecover, could it be that the problem is that your user tries to
use different callbacks in different threads? _SWRecover is a global
after all and shared by all threads.

Bernhard
 
R

Robin Becker

Bernhard Herzog said:
No. It would be very inconvenient if it did because what most functions
do first is to call one of the PyArg_ParseTuple functions which could
execute arbitrary python code.

Thank goodness for that :)
I think it would help if you described a bit more what the actual
problem is or what the user does.

I looked briefly at the sources and assuming tha the accelerator in
question is lib/_rl_accel.c and the callback whatever has been assigned
to _SWRecover, could it be that the problem is that your user tries to
use different callbacks in different threads? _SWRecover is a global
after all and shared by all threads.

I'm not certain, but yes I think that's probably what the problem comes
down to; if so I can claim there are already too many breakages in our
font system to worry about that :(
 
A

Aahz

So does calling an extension release the GIL? I believed the opposite
for some reason.

Looks like Bernhard has put you on the right track, but just for the
record: calling an extension does not automatically release the GIL; you
need to explicitly call the Py_BEGIN_ALLOW_THREADS macro. I was just
assuming that because you said "accelerator" you were already releasing
the GIL.
 
R

Robin Becker

writes
.......
Looks like Bernhard has put you on the right track, but just for the
record: calling an extension does not automatically release the GIL; you
need to explicitly call the Py_BEGIN_ALLOW_THREADS macro. I was just
assuming that because you said "accelerator" you were already releasing
the GIL.
The code is intended to run single thread so we never have done anything
about threading. It's a real pain. In a zope world we ought really to
give every request a fresh context ie each thread would see an entirely
'clean' python and then be forced to load its own fonts/resources. It
just sucks from the overall system viewpoint. How does one ensure that
each thread sees its own version of sys.modules etc etc or is that
considered a dumb question?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top