python thread scheduler?

Discussion in 'Python' started by project2501, Apr 7, 2004.

  1. project2501

    project2501 Guest

    i'm doing some benchmarking and python is certainly fast enough (the
    time.time resolution is more than good enough).

    however, i am using the threading module to implement worker trheads to
    hit a server with varying levels of workload.

    hwoever, my result graphs (of server response time) are very flat and
    these flat graphs are higher with the number of threads.

    this suggests to me that the barrier i'm hitting is the python thread
    schedular, not the server software being tested.

    i've verified this with 2 different machines, each runnning the python
    client, to hit a server and the graphs are still flat.

    any ideas about how i can verify my suspicions and hwo to overcome them?
    is there a switch in python that will allow me to have "very independent"
    threads? forking takes too much memory and the clients machines get bogged
    down beofre the server software under test.
     
    project2501, Apr 7, 2004
    #1
    1. Advertising

  2. project2501

    project2501 Guest

    to clarifgy, i'm seeing responec times as low as 0.3 seconds when the
    client has 5 worker threads... rising to an average of about 8 seconds
    with 50 threads... and more ith 100 threads.

    here is what the python threads do (simplified):
    while (work units remaining):
    take time
    send request to server
    get responce from server
    take time
    record time difference
    sleep for a small interval (to yield, and to vary the request rate)
    reduce work unit by 1

    i get flat graphs of responce time against request rate (which
    doubles a specified number of times, after 10 work units.

    i would expect the graphs to start to vary non-linearly ... especially at
    higher numbers of threads (say 50 or more) and request rates of 256 per
    second or more ...
     
    project2501, Apr 7, 2004
    #2
    1. Advertising

  3. project2501

    Aahz Guest

    In article <>,
    project2501 <> wrote:
    >
    >to clarifgy, i'm seeing responec times as low as 0.3 seconds when the
    >client has 5 worker threads... rising to an average of about 8 seconds
    >with 50 threads... and more ith 100 threads.
    >
    >here is what the python threads do (simplified):
    > while (work units remaining):
    > take time
    > send request to server
    > get responce from server
    > take time
    > record time difference
    > sleep for a small interval (to yield, and to vary the request rate)
    > reduce work unit by 1


    Since this is almost entirely I/O bound, it *should* scale well with
    threads. Question: is your server on the same machine as the client?
    --
    Aahz () <*> http://www.pythoncraft.com/

    Why is this newsgroup different from all other newsgroups?
     
    Aahz, Apr 9, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Codemonkey
    Replies:
    8
    Views:
    9,572
  2. Sakharam
    Replies:
    2
    Views:
    8,635
    Sakharam
    Sep 1, 2005
  3. Guest

    Re: python thread scheduler?

    Guest, Apr 7, 2004, in forum: Python
    Replies:
    1
    Views:
    436
    project2501
    Apr 8, 2004
  4. Eric @ Zomething

    Windows XP - cron or scheduler for Python?

    Eric @ Zomething, Jun 20, 2004, in forum: Python
    Replies:
    19
    Views:
    4,658
    Esmail Bonakdarian
    Dec 27, 2004
  5. Vellingiri Arul

    what is mean by thread scheduler in Ruby?

    Vellingiri Arul, Sep 28, 2007, in forum: Ruby
    Replies:
    2
    Views:
    109
    Charles Oliver Nutter
    Sep 28, 2007
Loading...

Share This Page