Determining if any threads are waiting for GIL

R

Rouslan Korneychuk

I'm working on a package that can compile CPython byte-code into native
machine code (so far: x86 and x86_64 are supported). I have support for
almost every byte-code instruction implemented already, but to fully
emulate the interpreter, I need an efficient way to determine two
things: when to give up the global interpreter lock (GIL) so another
thread can use it, and if there are any asynchronous calls pending (a la
Py_AddPendingCall).

The interpreter loop does this by polling a pair of boolean values
(technically there is a third value it polls first as an optimization,
but this is not important), but these values are not exposed by the API
in any way. Even if I just implement context switching between compiled
code only, I would still need a way to determine if there are pending
asynchronous calls. The whole point of this package is to speed up
Python programs, so unconditionally calling Py_MakePendingCalls every
once in a while is unacceptable.

So far the only solution I have come up with is to implement a basic
machine code interpreter that would parse PyEval_AcquireLock and
Py_AddPendingCall at run-time to determine the addresses of the values
the Python interpreter uses. I wouldn't need to fully parse the machine
code since for most instructions, I just need to figure out how long it
is so I can skip over it, but with varying compilers and build settings,
this would be opening up a can of worms.

Any suggestions are welcome.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,900
Latest member
Nell636132

Latest Threads

Top