How to check for threads being finished?

[Steven D'Aprano]

I have a pool of worker threads, created like this:

threads = [MyThread(*args) for i in range(numthreads)]
for t in threads:
t.start()


I then block until the threads are all done:

while any(t.isAlive() for t in threads):
pass


Is that the right way to wait for the threads to be done? Should I stick
a call to time.sleep() inside the while loop? If so, how long should I
sleep? That's probably an unanswerable question, but some guidelines on
choosing the sleep time will be appreciated.

 

[Chris Angelico]

I then block until the threads are all done:

while any(t.isAlive() for t in threads):
pass

Using the threading module, I assume. Is there any reason you can't
simply join() each thread in succession?

ChrisA

 

[Irmen de Jong]

I then block until the threads are all done:

while any(t.isAlive() for t in threads):
pass


Is that the right way to wait for the threads to be done? Should I stick
a call to time.sleep() inside the while loop? If so, how long should I
sleep? That's probably an unanswerable question, but some guidelines on
choosing the sleep time will be appreciated.

I think your while loop busy-waits until the threads are completed.
Do this instead:

for t in threads: t.join() # optionally pass a timeout to join


-Irmen

 

[Ian Kelly]

I have a pool of worker threads, created like this:

threads = [MyThread(*args) for i in range(numthreads)]
for t in threads:
t.start()


I then block until the threads are all done:

while any(t.isAlive() for t in threads):
pass


Is that the right way to wait for the threads to be done? Should I stick
a call to time.sleep() inside the while loop? If so, how long should I
sleep? That's probably an unanswerable question, but some guidelines on
choosing the sleep time will be appreciated.

for thread in threads:
thread.join()

 

[Dennis Lee Bieber]

I have a pool of worker threads, created like this:

threads = [MyThread(*args) for i in range(numthreads)]
for t in threads:
t.start()


I then block until the threads are all done:

while any(t.isAlive() for t in threads):
pass


Is that the right way to wait for the threads to be done? Should I stick
a call to time.sleep() inside the while loop? If so, how long should I
sleep? That's probably an unanswerable question, but some guidelines on
choosing the sleep time will be appreciated.

If you are just going to wait for them to finish, and never use them
again...

for t in threads:
t.join()

Doesn't matter which thread ends first, this operation will either
block if the thread hasn't finished or cycle to the next thread if it has.

 

[Cameron Simpson]

| I have a pool of worker threads, created like this:
|
| threads = [MyThread(*args) for i in range(numthreads)]
| for t in threads:
| t.start()
|
| I then block until the threads are all done:
|
| while any(t.isAlive() for t in threads):
| pass

This spins.

How about:

for t in threads:
t.join()

Blocks instead of polls.

Cheers,

 

[Steven D'Aprano]

I think your while loop busy-waits until the threads are completed. Do
this instead:

for t in threads: t.join() # optionally pass a timeout to join

Yes, the busy-wait was what I was concerned about.

Thanks to everyone who suggested the same solution.

 

[Stefan Behnel]

Irmen de Jong, 05.07.2013 19:12:

I think your while loop busy-waits until the threads are completed.
Do this instead:

for t in threads: t.join() # optionally pass a timeout to join

A related stdlib tool that many people don't seem to know is the thread
pool in concurrent.futures. It supports both waiting for all threads to
finish as well as iterating over results as they come in. It also comes
with a parallel map() implementation.

http://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example

http://docs.python.org/3/library/concurrent.futures.html#module-functions

New in Py3.2, but there's also a backport AFAIR.

Stefan