lots of futex_wait calls

[skunkwerk]

I've got a python program written for the django web framework that
starts about 100 threads. When I start the server, it sometimes eats
up 100% of the CPU for a good minute or so... though none of the
threads are CPU-intensive

doing a strace on the program, i found lots of calls like this:

select(5, [4], [], [], {1, 0}) = 0 (Timeout)
futex(0x86a3ce0, FUTEX_WAIT, 0, NULL) = 0

i've read the man page for futex... but is this normal?

thanks

 

[André Malo]

I've got a python program written for the django web framework that
starts about 100 threads. When I start the server, it sometimes eats
up 100% of the CPU for a good minute or so... though none of the
threads are CPU-intensive

doing a strace on the program, i found lots of calls like this:

select(5, [4], [], [], {1, 0}) = 0 (Timeout)
futex(0x86a3ce0, FUTEX_WAIT, 0, NULL) = 0

i've read the man page for futex... but is this normal?

More or less. Most of the futex calls (if not all) are grabbing or releasing
the global interpreter lock (GIL).

It's usually helpful to increase the thread-schedule-checkinterval in order
to lessen the system load (especially the number of context switches). See
sys.setcheckinterval.

nd

 

[skunkwerk]

skunkwerkwrote:

I've got a python program written for the django web framework that
starts about 100 threads. When I start the server, it sometimes eats
up 100% of the CPU for a good minute or so... though none of the
threads are CPU-intensive

doing a strace on the program, i found lots of calls like this:

select(5, [4], [], [], {1, 0}) = 0 (Timeout)
futex(0x86a3ce0, FUTEX_WAIT, 0, NULL) = 0

i've read the man page for futex... but is this normal?

More or less. Most of the futex calls (if not all) are grabbing or releasing
the global interpreter lock (GIL).

It's usually helpful to increase the thread-schedule-checkinterval in order
to lessen the system load (especially the number of context switches). See
sys.setcheckinterval.

nd

I've set the checkinterval to 200, and it seems to be ok... but after
one or two days, the python processes will start hogging 100% of the
CPU and bring the system to a crawl. I ran strace again, and all of
the calls are:

select(5, [4],[],[],{1,0}) = 0 (Timeout)
futex(0x877d0c8, FUTEX_WAIT, 0 NULL) = 0
futex(0x877d0c8, FUTEX_WAKE,1) = 0

is there any way to find out what's causing this? would you need to
look at my threading code?

thanks,
imran