[ANN] RWB 0.2.0 (now with extra RubyForge goodness)

Discussion in 'Ruby' started by pat eyler, Nov 9, 2005.

  1. pat eyler

    pat eyler Guest

    RWB 0.1.1 was released last night, but before I put together an
    announcement, I ended up putting together a 0.2.0 release too.

    This combined release features three new warmup methods:
    warmup(num_runs)
    rand_warmup(num_requests)
    spec_warmup(urls, num_runs)
    These methods will exercise your website prior to the recorded tests.

    RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
    behind a web proxy.

    You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    of examples are hiding at www.red-bean.com/~pate as well).

    Don't forget to send me you comments, suggestions, requests, patches,
    etc.

    --
    thanks,
    -pate
    -------------------------
    pat eyler, Nov 9, 2005
    #1
    1. Advertising

  2. pat eyler

    Jeff Wood Guest

    ------=_Part_8937_16703898.1132762138086
    Content-Type: text/plain; charset=ISO-8859-1
    Content-Transfer-Encoding: quoted-printable
    Content-Disposition: inline

    have you released this as a gem yet?

    j.

    On 11/9/05, pat eyler <> wrote:
    >
    > RWB 0.1.1 was released last night, but before I put together an
    > announcement, I ended up putting together a 0.2.0 release too.
    >
    > This combined release features three new warmup methods:
    > warmup(num_runs)
    > rand_warmup(num_requests)
    > spec_warmup(urls, num_runs)
    > These methods will exercise your website prior to the recorded tests.
    >
    > RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
    > behind a web proxy.
    >
    > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    > of examples are hiding at www.red-bean.com/~pate as well).
    >
    > Don't forget to send me you comments, suggestions, requests, patches,
    > etc.
    >
    > --
    > thanks,
    > -pate
    > -------------------------
    >
    >



    --
    "Remember. Understand. Believe. Yield! -> http://ruby-lang.org"

    Jeff Wood

    ------=_Part_8937_16703898.1132762138086--
    Jeff Wood, Nov 23, 2005
    #2
    1. Advertising

  3. pat eyler

    pat eyler Guest

    I haven't ... I've been caught up in work related goo. I'm part way
    through implementing quartile graphs and will release it as a gem
    when I finish that.

    -pate

    On 11/23/05, Jeff Wood <> wrote:
    > have you released this as a gem yet?
    >
    > j.
    >
    > On 11/9/05, pat eyler <> wrote:
    > >
    > > RWB 0.1.1 was released last night, but before I put together an
    > > announcement, I ended up putting together a 0.2.0 release too.
    > >
    > > This combined release features three new warmup methods:
    > > warmup(num_runs)
    > > rand_warmup(num_requests)
    > > spec_warmup(urls, num_runs)
    > > These methods will exercise your website prior to the recorded tests.
    > >
    > > RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
    > > behind a web proxy.
    > >
    > > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    > > of examples are hiding at www.red-bean.com/~pate as well).
    > >
    > > Don't forget to send me you comments, suggestions, requests, patches,
    > > etc.
    > >
    > > --
    > > thanks,
    > > -pate
    > > -------------------------
    > >
    > >

    >
    >
    > --
    > "Remember. Understand. Believe. Yield! -> http://ruby-lang.org"
    >
    > Jeff Wood
    >
    >



    --
    thanks,
    -pate
    -------------------------
    pat eyler, Nov 23, 2005
    #3
  4. you guys ever think of explaining wtf you talking about? rwb?? what is it?
    :)
    --r


    2005/11/9, pat eyler <>:
    > RWB 0.1.1 was released last night, but before I put together an
    > announcement, I ended up putting together a 0.2.0 release too.
    >
    > This combined release features three new warmup methods:
    > warmup(num_runs)
    > rand_warmup(num_requests)
    > spec_warmup(urls, num_runs)
    > These methods will exercise your website prior to the recorded tests.
    >
    > RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
    > behind a web proxy.
    >
    > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    > of examples are hiding at www.red-bean.com/~pate as well).
    >
    > Don't forget to send me you comments, suggestions, requests, patches,
    > etc.
    >
    > --
    > thanks,
    > -pate
    > -------------------------
    >
    >
    Robert Wagner, Nov 24, 2005
    #4
  5. pat eyler

    pat eyler Guest

    On 11/23/05, Robert Wagner <> wrote:
    > you guys ever think of explaining wtf you talking about? rwb?? what is it=

    ?
    > :)



    rwb is the Ruby Web Bench, a Ruby library designed to let you run
    performance/load tests against a webserver/website. Using it you
    can write scripts like this:

    require 'rwb'

    urls =3D RWB::Builder.new()
    urls.add_url(50,
    "http://www.example.com/welcome")
    urls.add_url(25,
    "http://www.example.com/expensive_dynamic_page")
    search_terms =3D ['foo+bar', 'bar+baz', 'quux']
    urls.add_url_group(10, "http://www.examples.com/search?",
    search_terms)

    tests =3D RWB::Runner.new(urls, 50_000, 500)
    tests.run

    tests.sla_levels =3D [0.5, 0.9, 0.99]
    tests.report_header
    tests.report_overall([0.25, 0.5, 0.75])
    tests.report_urls
    tests.report_by_time


    Which would create a report something like this:

    Concurrency Level: 500
    Total Requests: 50000
    Total time for testing: 248.407943 secs
    Requests per second: 201.281808448452
    Mean time per request: 49 msecs
    Standard deviation: 44
    Overall results:
    Shortest time: 23 msecs
    25.0%ile time: 28 msecs
    50.0%ile time: 29 msecs
    75.0%ile time: 46 msecs
    Longest time: 276 msecs
    Results for http://www.example.com/welcome:
    Shortest time: 23 msecs
    50.0%ile time: 29 msecs
    90.0%ile time: 123 msecs
    99.0%ile time: 210 msecs
    Longest time: 276 msecs
    Results for http://www.example.com/expensive_dynamic_page:
    Shortest time: 23 msecs
    50.0%ile time: 29 msecs
    90.0%ile time: 122 msecs
    99.0%ile time: 210 msecs
    Longest time: 255 msecs
    Results for http://www.example.com/search?:
    Shortest time: 24 msecs
    50.0%ile time: 29 msecs
    90.0%ile time: 120 msecs
    99.0%ile time: 209 msecs
    Longest time: 239 msecs
    Results by time:
    results for requests 0 - 10000
    Shortest time: 23 msecs
    50.0%ile time: 30 msecs
    90.0%ile time: 61 msecs
    99.0%ile time: 73 msecs
    Longest time: 104 msecs
    results for requests 10000 - 20000
    Shortest time: 24 msecs
    50.0%ile time: 29 msecs
    90.0%ile time: 93 msecs
    99.0%ile time: 114 msecs
    Longest time: 140 msecs
    results for requests 20000 - 30000
    Shortest time: 23 msecs
    50.0%ile time: 29 msecs
    90.0%ile time: 129 msecs
    99.0%ile time: 153 msecs
    Longest time: 232 msecs
    results for requests 30000 - 40000
    Shortest time: 24 msecs
    50.0%ile time: 29 msecs
    90.0%ile time: 159 msecs
    99.0%ile time: 183 msecs
    Longest time: 203 msecs
    results for requests 40000 - 50000
    Shortest time: 23 msecs
    50.0%ile time: 28 msecs
    90.0%ile time: 198 msecs
    99.0%ile time: 231 msecs
    Longest time: 276 msecs



    There's also some info in the email you quoted below:

    > --r
    >
    >
    > 2005/11/9, pat eyler <>:


    > > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    > > of examples are hiding at www.red-bean.com/~pate as well).
    > >
    > > Don't forget to send me you comments, suggestions, requests, patches,
    > > etc.


    --
    hth,
    -pate
    -------------------------
    pat eyler, Nov 24, 2005
    #5
  6. I took a brief look at the code a week or so ago, and I like it.
    However (and this might just be my lack of deep understanding of the
    way Ruby deals with sockets), it seems to me that the 'concurrency'
    feature doesn't really do much.
    Isn't the bulk of the work performed in the concurrent threads being
    serialized by the host process? When I look at the timestamps on the
    web server, the requests seem to arrive more or less in order.
    Am I using a crappy web server log, or is this really how it behaves
    in the absence of native threads? I'm probably just expecting
    parallelism, and getting concurrency instead.
    I slapped together a lame little script that starts 100 instances of
    the Ruby interpreter, and that managed to actually make numerous
    requests arrive simultaneously at the server. Unfortunately, it used
    a truly shocking amount of memory on the client system.

    Stated another way; how different are:
    tests =3D RWB::Runner.new(urls, 1000, 5)
    and:
    tests =3D RWB::Runner.new(urls, 1000, 1000)
    ?

    Thanks for reading my incoherent babble.
    --Wilson.


    On 11/9/05, pat eyler <> wrote:
    > RWB 0.1.1 was released last night, but before I put together an
    > announcement, I ended up putting together a 0.2.0 release too.
    >
    > This combined release features three new warmup methods:
    > warmup(num_runs)
    > rand_warmup(num_requests)
    > spec_warmup(urls, num_runs)
    > These methods will exercise your website prior to the recorded tests.
    >
    > RWB 0.2.0 also adds the Runner#add_proxy method, so RWB will now work
    > behind a web proxy.
    >
    > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    > of examples are hiding at www.red-bean.com/~pate as well).
    >
    > Don't forget to send me you comments, suggestions, requests, patches,
    > etc.
    Wilson Bilkovich, Nov 24, 2005
    #6
  7. pat eyler

    pat eyler Guest

    On 11/23/05, Wilson Bilkovich <> wrote:
    > I took a brief look at the code a week or so ago, and I like it.
    > However (and this might just be my lack of deep understanding of the
    > way Ruby deals with sockets), it seems to me that the 'concurrency'
    > feature doesn't really do much.


    Concurrency may be too strong a word, but I'm not sure what else
    to call it. So far RWB has worked well enough for me, but I'd be
    happy to work with people who find it's not up to their tasks. My
    big aims to this point are to get a working request engine and a
    nice, featureful reporting system.

    > Isn't the bulk of the work performed in the concurrent threads being
    > serialized by the host process? When I look at the timestamps on the
    > web server, the requests seem to arrive more or less in order.
    > Am I using a crappy web server log, or is this really how it behaves
    > in the absence of native threads? I'm probably just expecting
    > parallelism, and getting concurrency instead.


    Unless someone wants to point out an improvement in my thread
    handling, I think that wysiwyg applies.

    > I slapped together a lame little script that starts 100 instances of
    > the Ruby interpreter, and that managed to actually make numerous
    > requests arrive simultaneously at the server. Unfortunately, it used
    > a truly shocking amount of memory on the client system.


    Wow, I'll bet that was huge. I'd be interested in seeing what
    native threads do for performance. Could you share your results?

    >
    > Stated another way; how different are:
    > tests =3D RWB::Runner.new(urls, 1000, 5)
    > and:
    > tests =3D RWB::Runner.new(urls, 1000, 1000)
    > ?


    The big difference is going to be against a slower webserver
    (or against slower dynamic pages). In that case the second
    invocation will spawn 1000 threads without weighting for any
    of them to finish, while the first will end up waiting at least
    some of the time. (At work I've had a couple of requests go
    in excess of 3 seconds, so I can really see the difference
    with higher levels of (pseudo) concurrancy.

    The other difference you'll see (even on a fast site) is in load
    generation/testing. Here are two runs for comparison:

    Concurrency Level: 5
    Total Requests: 1000
    Total time for testing: 16.329556 secs
    Requests per second: 61.2386521715594
    Mean time per request: 81 msecs
    Standard deviation: 77
    Overall results:
    Shortest time: 48 msecs
    25.0%ile time: 56 msecs
    50.0%ile time: 57 msecs
    75.0%ile time: 59 msecs
    Longest time: 703 msecs


    Concurrency Level: 1000
    Total Requests: 1000
    Total time for testing: 8.444824 secs
    Requests per second: 118.415730156129
    Mean time per request: 84 msecs
    Standard deviation: 186
    Overall results:
    Shortest time: 21 msecs
    25.0%ile time: 26 msecs
    50.0%ile time: 34 msecs
    75.0%ile time: 37 msecs
    Longest time: 1103 msecs

    You can see that we were able to throw almost
    twice as many requests per second against the
    server this way, which can be an important measure
    when you're talking about how to scale/optimize
    a website for a large traffic load.

    >
    > Thanks for reading my incoherent babble.


    Thanks for sharing.

    > --Wilson.
    >




    --
    thanks,
    -pate
    -------------------------
    pat eyler, Nov 24, 2005
    #7
  8. 2005/11/24, pat eyler <>:
    > On 11/23/05, Robert Wagner <> wrote:
    > > you guys ever think of explaining wtf you talking about? rwb?? what is =

    it?
    > > :)

    >
    >
    > rwb is the Ruby Web Bench, a Ruby library designed to let you run
    > performance/load tests against a webserver/website. Using it you
    > can write scripts like this:
    >


    web performance tests... this is very nice.

    > There's also some info in the email you quoted below:
    >


    me thinks: a short "about" should be the first sentence in an announcement

    ciao,
    --r


    > > --r
    > >
    > >
    > > 2005/11/9, pat eyler <>:

    >
    > > > You can grab your copy from rubyforge.org/projects/rwb/ (a couple
    > > > of examples are hiding at www.red-bean.com/~pate as well).
    > > >
    > > > Don't forget to send me you comments, suggestions, requests, patches,
    > > > etc.

    >
    > --
    > hth,
    > -pate
    > -------------------------
    >
    >
    Robert Wagner, Nov 24, 2005
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tom Copeland
    Replies:
    0
    Views:
    121
    Tom Copeland
    Sep 6, 2003
  2. pat eyler

    [ANN] Ruby Web Bench 0.0.0 (RWB)

    pat eyler, Nov 7, 2005, in forum: Ruby
    Replies:
    5
    Views:
    94
    Bob Hutchison
    Nov 8, 2005
  3. pat eyler

    RWB, not quite an announcement

    pat eyler, Dec 1, 2005, in forum: Ruby
    Replies:
    2
    Views:
    92
    pat eyler
    Dec 2, 2005
  4. pat eyler

    [ANN] RWB 0.2.1

    pat eyler, Dec 2, 2005, in forum: Ruby
    Replies:
    0
    Views:
    94
    pat eyler
    Dec 2, 2005
  5. Rob Sanheim
    Replies:
    0
    Views:
    96
    Rob Sanheim
    Jul 6, 2007
Loading...

Share This Page