Using Py_AddPendingCall

C

css322

I have an embedded Python program which runs in a thread in C.

When the Python interpreter switches thread context (yielding control to another thread), I'd like to be notified so I can perform certain necessary operations.

It seems that Py_AddPendingCall is exactly what I'm looking for. However, the API docs are pretty brief on this function, and I'm confused as to how Py_AddPendingCall is supposed to be used. From reading the docs, my understanding is that:

(1) A worker thread calls Py_AddPendingCall and assigns a handler function.
(2) When the Python interpreter runs, it calls the handler function whenever it yields control to another thread
(3) The handler itself is executed in the main interpreter thread, with the GIL acquired

I've googled around for example code showing how to use Py_AddPendingCall, but I can't find anything. My own attempt to use it simply doesn't work. The handler is just never called.

My worker thread code:

const char* PYTHON_CODE =
"while True:\n"
" for i in range(0,10): print(i)\n"
"\n";

int handler(void* arg)
{
printf("Pending Call invoked!\n");
abort();
}

void worker_thread()
{
PyGILState_STATE state = PyGILState_Ensure();
Py_AddPendingCall(&handler, NULL);
PyRun_SimpleString(PYTHON_CODE);
PyGILState_Release(state);
}

In this example, worker_thread is invoked in C as a pthread worker thread. This loops infinitely, but the pending call handler is never invoked.

So,obviously I'm not understanding the docs correctly. How is Py_AddPendingCall is supposed to be used?
 
A

Antoine Pitrou

css322 said:
(1) A worker thread calls Py_AddPendingCall and assigns a handler function.
(2) When the Python interpreter runs, it calls the handler function whenever
it yields control to another thread

Not exactly. As the documentation says: "If successful, func will be called
with the argument arg *at the earliest convenience*."
This is a deliberately vague wording to stress that the function will not be
called as early as you think. It *should* be called in a timely manner, but not
necessarily as soon as the thread switch happens.

Which begs the question:
In this example, worker_thread is invoked in C as a pthread worker thread. This
loops infinitely, but the pending call handler is never invoked.

What is your main Python thread doing? Is it running Python code (*)? Or do you
have a main Python thread at all? The "main Python thread" is the one from which
Py_Initialize() was called.

(*) for example, running one of the following functions:
http://docs.python.org/dev/c-api/veryhigh.html

Regards

Antoine.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top