Bandwidth constraint causing IIS/ASP.NET problem

Discussion in 'ASP .Net' started by Jack Fox, Sep 11, 2005.

  1. Jack Fox

    Jack Fox Guest

    While stress-testing our ASP.NET app, we discovered a flaw in our test
    set-up, which also reveals a flaw in our ASP.NET configuration that we do
    not know how to address.



    We simulated too many clients, making requests faster than the bandwidth on
    that part of the network could consume the responses. It seems that the
    inability of the requestors to receive their responses causes IIS, and/or
    aspnet_wp.exe, to eventually max-out its threads and come to its knees.



    We changed our unrealistic virtual client distribution and distributed our
    load generation to other parts of the network. Our app then did fine.



    But.it seems that configuring more response load on the network than the
    bandwidth can handle is a type of denial of service attack against
    IIS/ASP.NET. How can we configure IIS and/or ASP.NET to prevent this?
     
    Jack Fox, Sep 11, 2005
    #1
    1. Advertising

  2. Jack Fox wrote:
    > We simulated too many clients, making requests faster than the
    > bandwidth on that part of the network could consume the responses. It
    > seems that the inability of the requestors to receive their responses
    > causes IIS, and/or aspnet_wp.exe, to eventually max-out its threads
    > and come to its knees.
    > But.it seems that configuring more response load on the network than
    > the bandwidth can handle is a type of denial of service attack against
    > IIS/ASP.NET. How can we configure IIS and/or ASP.NET to prevent this?


    It's hard to believe that you could have exceeded bandwidth limitations in
    this scenario. It seems more likely that you stressed the box. However,
    without some perfmon data, it's hard to say where you hosed yourself.

    There are so many places where a performance problem can happen. If you're
    in GC too much, you will see this kind of behavior. When we're in GC (or
    when CPU reaches a certain threshold), new worker threads are not spun up to
    handle requests. It's also possible that you might not have enough
    completion port threads available. You also might not have enough worker
    threads or too many worker threads.

    Without details, it's impossible to say. However, there is a test you can
    run. When you're bogged down, do HTML/ASP pages serve okay? If so, it's
    likely something from paragraph 2. If not, something unrelated to ASP.NET
    is going on.

    --
    Jim Cheshire
    JIMCO Software
    http://www.jimcosoftware.com

    FrontPage add-ins for FrontPage 2000 - 2003
     
    JIMCO Software, Sep 11, 2005
    #2
    1. Advertising

  3. Jack Fox

    Jack Fox Guest

    I should have mentioned the reason I know we exceeded the bandwidth is
    because we were simulating 1,200 clients with 3 - 6 seconds between each
    client request, requesting pages that are up to 100kb. If you do some back
    of the envelope arithmetic you'll probably agree that all of that request
    and response traffic to one machine connected through one 100mbs connection
    will exceed the bandwidth.

    When we distributed the client load over several machines so that we didn't
    have the 100mbs bandwidth constraint, everything worked fine.

    The point I'm trying to make is that by making requests that will generate
    too much response traffic for the client network node, you are in effect
    performing a denial of service attack on the IIS/ASP.NET server. We've
    boosted our thread count before, and your suggestion of looking at the
    completion port threads is probably worthwhile, but it seems that a
    malicious client could still overwhelm the server because there is alway a
    thread limit. Is there any way to short-circuit this kind of attack?

    "JIMCO Software" <> wrote in message
    news:%...
    > Jack Fox wrote:
    >> We simulated too many clients, making requests faster than the
    >> bandwidth on that part of the network could consume the responses. It
    >> seems that the inability of the requestors to receive their responses
    >> causes IIS, and/or aspnet_wp.exe, to eventually max-out its threads
    >> and come to its knees.
    >> But.it seems that configuring more response load on the network than
    >> the bandwidth can handle is a type of denial of service attack against
    >> IIS/ASP.NET. How can we configure IIS and/or ASP.NET to prevent this?

    >
    > It's hard to believe that you could have exceeded bandwidth limitations in
    > this scenario. It seems more likely that you stressed the box. However,
    > without some perfmon data, it's hard to say where you hosed yourself.
    >
    > There are so many places where a performance problem can happen. If
    > you're in GC too much, you will see this kind of behavior. When we're in
    > GC (or when CPU reaches a certain threshold), new worker threads are not
    > spun up to handle requests. It's also possible that you might not have
    > enough completion port threads available. You also might not have enough
    > worker threads or too many worker threads.
    >
    > Without details, it's impossible to say. However, there is a test you can
    > run. When you're bogged down, do HTML/ASP pages serve okay? If so, it's
    > likely something from paragraph 2. If not, something unrelated to ASP.NET
    > is going on.
    >
    > --
    > Jim Cheshire
    > JIMCO Software
    > http://www.jimcosoftware.com
    >
    > FrontPage add-ins for FrontPage 2000 - 2003
    >
    >
    >
    >
     
    Jack Fox, Sep 11, 2005
    #3
  4. Jack Fox wrote:
    > I should have mentioned the reason I know we exceeded the bandwidth is
    > because we were simulating 1,200 clients with 3 - 6 seconds between
    > each client request, requesting pages that are up to 100kb. If you do
    > some back of the envelope arithmetic you'll probably agree that all
    > of that request and response traffic to one machine connected through
    > one 100mbs connection will exceed the bandwidth.
    >
    > When we distributed the client load over several machines so that we
    > didn't have the 100mbs bandwidth constraint, everything worked fine.
    >


    Jack,

    What you are describing here would not cause a DoS.

    As for increasing worker threads, I didn't suggest that you do that. You
    would only do that if you have available CPU to process them and currently
    have no requests queued. What I was suggesting is that you need to use
    performance data (via Perfmon logs) to determine where your perf problem is.
    It doesn't seem to me that you've done that.

    --
    Jim Cheshire
    JIMCO Software
    http://www.jimcosoftware.com

    FrontPage add-ins for FrontPage 2000 - 2003
     
    JIMCO Software, Sep 11, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ibiza
    Replies:
    1
    Views:
    2,632
    Erik Funkenbusch
    Mar 10, 2006
  2. bootman
    Replies:
    0
    Views:
    567
    bootman
    Aug 11, 2003
  3. =?Utf-8?B?UmljaGFyZA==?=

    Bandwidth Testing in Asp.Net 2.0

    =?Utf-8?B?UmljaGFyZA==?=, May 9, 2006, in forum: ASP .Net
    Replies:
    3
    Views:
    657
    Spam Catcher
    May 9, 2006
  4. =?Utf-8?B?QW5kcmV3IEhheWVz?=

    C#/Java Souce Code For ASP.NET Bandwidth/Speed Test Page Needed

    =?Utf-8?B?QW5kcmV3IEhheWVz?=, Mar 5, 2007, in forum: ASP .Net
    Replies:
    5
    Views:
    1,911
    =?Utf-8?B?QW5kcmV3IEhheWVz?=
    Mar 7, 2007
  5. puvit82
    Replies:
    4
    Views:
    762
    puvit82
    Feb 1, 2008
Loading...

Share This Page