python thread scheduler?

P

project2501

i'm doing some benchmarking and python is certainly fast enough (the
time.time resolution is more than good enough).

however, i am using the threading module to implement worker trheads to
hit a server with varying levels of workload.

hwoever, my result graphs (of server response time) are very flat and
these flat graphs are higher with the number of threads.

this suggests to me that the barrier i'm hitting is the python thread
schedular, not the server software being tested.

i've verified this with 2 different machines, each runnning the python
client, to hit a server and the graphs are still flat.

any ideas about how i can verify my suspicions and hwo to overcome them?
is there a switch in python that will allow me to have "very independent"
threads? forking takes too much memory and the clients machines get bogged
down beofre the server software under test.
 
P

project2501

to clarifgy, i'm seeing responec times as low as 0.3 seconds when the
client has 5 worker threads... rising to an average of about 8 seconds
with 50 threads... and more ith 100 threads.

here is what the python threads do (simplified):
while (work units remaining):
take time
send request to server
get responce from server
take time
record time difference
sleep for a small interval (to yield, and to vary the request rate)
reduce work unit by 1

i get flat graphs of responce time against request rate (which
doubles a specified number of times, after 10 work units.

i would expect the graphs to start to vary non-linearly ... especially at
higher numbers of threads (say 50 or more) and request rates of 256 per
second or more ...
 
A

Aahz

to clarifgy, i'm seeing responec times as low as 0.3 seconds when the
client has 5 worker threads... rising to an average of about 8 seconds
with 50 threads... and more ith 100 threads.

here is what the python threads do (simplified):
while (work units remaining):
take time
send request to server
get responce from server
take time
record time difference
sleep for a small interval (to yield, and to vary the request rate)
reduce work unit by 1

Since this is almost entirely I/O bound, it *should* scale well with
threads. Question: is your server on the same machine as the client?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top