I want to release the GIL

[Piotr Sobolewski]

Hello,
I have such program:

import time
import thread
def f():
    global lock
    while True:
        lock.acquire()
        print thread.get_ident()
        time.sleep(1)
        lock.release()
lock=thread.allocate_lock()
thread.start_new_thread(f,())
thread.start_new_thread(f,())
time.sleep(60)

As you can see, I start two threads. Each one works in an infinite
loop.
Inside that loop it acquires lock, prints its own id, sleeps a bit and
then
releases lock.

When I run it, I notice that only one thread works and the other one
never
has a chance to run. I guess it is because the thread don't have a
chance
to release the GIL - after it releases the lock, it almost immediately
(in
the very next bytecode command) reacquires it. I know I can
put "time.sleep(0.01)" command after between release and reacquire,
but it
doesn't seem elegant - why should I make my program sleep instead of
work?

Is there any simple way to release the GIL? Like:
lock.release()
thread.release_gil()
?

Thanks in advance!

 

[Chris Rebert]

Hello,
I have such program:

import time
import thread
def f():
global lock
while True:
lock.acquire()
print thread.get_ident()
time.sleep(1)
lock.release()
lock=thread.allocate_lock()
thread.start_new_thread(f,())
thread.start_new_thread(f,())
time.sleep(60)

As you can see, I start two threads. Each one works in an infinite
loop.
Inside that loop it acquires lock, prints its own id, sleeps a bit and
then
releases lock.

When I run it, I notice that only one thread works and the other one
never
has a chance to run. I guess it is because the thread don't have a
chance
to release the GIL - after it releases the lock, it almost immediately
(in
the very next bytecode command) reacquires it. I know I can
put "time.sleep(0.01)" command after between release and reacquire,
but it
doesn't seem elegant - why should I make my program sleep instead of
work?

You let *one thread* sleep so that the *other thread* can wake, grab
the lock, and *work*.

Cheers,
Chris

 

[Carl Banks]

Hello,
I have such program:

import time
import thread
def f():
    global lock
    while True:
        lock.acquire()
        print thread.get_ident()
        time.sleep(1)
        lock.release()
lock=thread.allocate_lock()
thread.start_new_thread(f,())
thread.start_new_thread(f,())
time.sleep(60)

Let me state first of all that you are going about it all wrong.
Simple locks don't handle the above siutation very well because,
frankly, there's no benefit to using threads in this way at all.
Since neither thread can run at the same time as the other, you might
as well have just used ordinary looping.

Normally, when using threads, you only want to use locking for brief
periods when accessing data shared between threads, like so:

def f():
do_some_stuff
lock.acquire()
access_shared_data
lock.release()
do_some_more_stuff

Starvation is much less likely to occur in this situation. (The
interpreter periodically releases the GIL, so if there's stuff between
a release and the next acquire, it'll have a window in which to
release the GIL--unlike in your example where the window was very
tiny.)

As you can see, I start two threads. Each one works in an infinite
loop.
Inside that loop it acquires lock, prints its own id, sleeps a bit and
then
releases lock.

When I run it, I notice that only one thread works and the other one
never
has a chance to run. I guess it is because the thread don't have a
chance
to release the GIL - after it releases the lock, it almost immediately
(in
the very next bytecode command) reacquires it.

Not really the GIL's fault. Or rather, fiddling with the GIL is the
wrong way to get what you want here.

What you want is some kind of thread balancing, to make sure all
threads have a chance to run. Let me stress that, in this case, you
only need balancing because you're using threads incorrectly. But
I'll mention one way to do it anyway.

The easiest way I can think of is to use a counter that is protected
by a threading.Condition. One thread only runs when the counter is
even, the other only when it's odd. It might be implemented like
this:


counter = 0
c = threading.Condition()

def f():
global counter
c.acquire()
while counter % 1 == 0: # == 1 for the other thread
c.wait()
do_something()
counter += 1
c.notify()
c.release()


The reason threading.Condition is required and not a simple lock is
that simply acquiring the lock is not enough; the counter must be in
the right state as well.


Carl Banks

 

[Martin v. Löwis]

When I run it, I notice that only one thread works and the other one

never has a chance to run. I guess it is because the thread don't
have a chance to release the GIL

This guess is wrong. A call to lock.acquire *does* release the GIL,
(as does the call to thread.sleep, as somebody else pointed out).

The real problem is that thread.lock is not guaranteed to be fair
(and isn't fair on most operating systems).

What happens is this (slight variations possible)

thread 1 thread 2
acquire GIL (wait for GIL)

release GIL (OS might schedule thread 2
here, but typically won't)
acquire lock
acquire GIL

print ident

release GIL
sleep acquire GIL
release GIL
(wait for lock) (still hold by thread 1)
(wait for GIL)
acquire GIL

release lock

release GIL
(wait for lock) (both threads waiting for
lock here. OS choses thread 1)
acquire lock
acquire GIL

(and so on)

To make that work the way you want it, you need a fair wake-up order.
Here is an example, using a list of Event objects:

import time
import threading

events = []

def f(e):
while True:
events.append(e)
token = e.wait()
e.clear()
print threading.currentThread()
time.sleep(1)
events.remove(e)
events[0].set()

for i in range(3):
t = threading.Thread(target=f, args=(threading.Event(),))
t.setDaemon(1)
t.start()
while not events:
print events
time.sleep(0.1)
events[0].set()
time.sleep(10)

HTH,
Martin