Threading and GIL

[googler.1.webmaster]

Hi!

I have a big problem I can't solve and I hope anyone of you can help
me. I use the Python C API and in C++ I have a class which represets a
thread object, similiar to the thread class of the known Python Thread
class but with some needed additions so thats the reason why I have to
built my own. Well, this is the problem.

There is a method called Thread::End() which I can call in Python. But
now the End() Function waits until the "Main" Function of my C++ class
is done, but in this method I want to ensure the Global Interpreter
Lock but this is already locked by the Thread which executes the End()
command. So both is waiting and stops. The thread which calls the End
() method is waiting for the finish of the Main() method of the Thread
Class and the Main-Method in the other thread waits for the thread of
the End() command that this is done - welcome Deadlock.

Any suggestions what I have to do? Thank you veeery much for you help,
otherwise I become really crazy here because this is the first time I
have such a problem.


Bye

 

[Carl Banks]

Hi!

I have a big problem I can't solve and I hope anyone of you can help
me. I use the Python C API and in C++ I have a class which represets a
thread object, similiar to the thread class of the known Python Thread
class but with some needed additions so thats the reason why I have to
built my own. Well, this is the problem.

There is a method called Thread::End() which I can call in Python. But
now the End() Function waits until the "Main" Function of my C++ class
is done, but in this method I want to ensure the Global Interpreter
Lock but this is already locked by the Thread which executes the End()
command. So both is waiting and stops. The thread which calls the End
() method is waiting for the finish of the Main() method of the Thread
Class and the Main-Method in the other thread waits for the thread of
the End() command that this is done - welcome Deadlock.

Any suggestions what I have to do? Thank you veeery much for you help,
otherwise I become really crazy here because this is the first time I
have such a problem.

Bye

You can try the PyGILState_Ensure and PyGILState_Release macros (see
PEP 311, http://www.python.org/dev/peps/pep-0311/ for more
information).

Make sure you have the GIL whenever you make any calls into Python
(with a couple exceptions) and make sure you DON'T have the GIL when
creating or deleting threads. In fact, I can't think of any reason
why it wouldn't work to ensure a GIL state first thing after the
thread starts and to release it right before the thread ends.

Carl Banks

 

[googler.1.webmaster]

Hi,

thats the reason why its not working. Imagine the end() method of the
thread object is called so the C++ Function is opened where the code
for this method is in.

At a line the Code ...->End() is called which waits that the C++
Thread class
is finished. BUT here is the problem: In the Method of the C++ class
which is in threaded mode can't run because its still waiting that the
GIL
is released by the thread which executed the ->End() command.

So the app has a deadlock when it arrives at ->End() and
PyGILState_Ensure
function.

 

[Carl Banks]

thats the reason why its not working. Imagine the end() method of the
thread object is called so the C++ Function is opened where the code
for this method is in.

You're going to have to post some code if you want better help; this
description is unintelligible. I don't know what you mean by end()
method, nor whether it's a Python method or C++ method, nor what
exactly you mean by thread object (C++ or Python object?) or C++
Function.

At a line the Code ...->End() is called which waits that the C++
Thread class
is finished. BUT here is the problem: In the Method of the C++ class
which is in threaded mode can't run because its still waiting that the
GIL
is released by the thread which executed the ->End() command.

So the app has a deadlock when it arrives at ->End() and
PyGILState_Ensure
function.

So you have to release the GIL before calling End(). Just surround End
() by Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS. (Presumably
you know the thread state when you call End(); if you don't then
surround the calls to Py_{BEGIN,END}_ALLOW_THREADS with calls to
PyGILState_{Ensure,Release}. Yes, that means you acquire the GIL just
so you can be sure that you've released it. I know of no other way to
be sure you don't have the GIL.)

If you are trying to kill a different thread, and I can't quite tell
if that's what you're doing, then be aware that it is very tricky to
do right and most think it to be a bad idea. If you don't allow the
thread you're killing to clean up it can deadlock, and even if you do,
you have to be careful to clean up properly and you have to be
constantly on guard for what might happen in a thread is killed in the
middle of something.


Carl Banks

 

[googler.1.webmaster]

hey, thanks, that works fine. I wrapped it around, done a lot of tests
and it works fine.
Just had done a few other things to make it stable.

cheers.