AppPool Recycle Failure

Discussion in 'ASP .Net' started by =?Utf-8?B?UGlnZ2xldDE4Nw==?=, Jun 8, 2004.

  1. I'm running a Server Farm of 5 presentation layer machines
    There are 55 different apppools, each is dedicated to a single .net applicatio
    Only approximately 10 are running at any time and 8 of those get enough traffic to never shut down
    1 in an unknown number of app pool restarts, from both timed recycle as well as timed shut down, the application is never able to answer a request
    It appears to attempt the JITC and never completes it to give a response on the application
    Have looked at this through the IIS discussion group, and they are pretty sure this is a code issue on our side, but since we have been unable to reproduce the problem in a development environment it is very difficult to troublshoot

    The applications use the same basic code base with modifications in presentation
    All uncached data comes from a Round Robined set of Middle tier servers, which switch machine after each request. Round Robin is provided by a list and custom code calls each machine in order

    The only information we really have is that this occurs on recycles/restarts it does not matter if the server is being pounded, or is under light load
    In looking at the output of IISState we found one of these errors for each thread that was being orphaned by IIS and showing up as Current Anonymous Users under Web Service Performance counters, as well as the Requests Current ASP.Net Counters

    Thread ID: 1
    System Thread ID: f0
    Kernel Time: 0:0:0.
    User Time: 0:0:0.
    Other information: Thread is waiting for a lock to be released. Looking for lock owner
    Owning thread System ID: 1f7
    Thread Type: AS
    Executing Page: ASP.dll symbols not found. Unable to locate ASP page
    Continuing with other analysis

    # ChildEBP RetAddr
    00 009ffc3c 77f43741 SharedUserData!SystemCallStub+0x
    01 009ffc40 77f5d64e ntdll!ZwWaitForSingleObject+0x
    02 009ffc7c 77f42044 ntdll!RtlpWaitForCriticalSection+0x12
    03 009ffc84 79e8d946 ntdll!RtlEnterCriticalSection+0x4
    04 009ffccc 79e75739 aspnet_isapi!GetAppDomain+0x9
    05 009fff1c 79e755e9 aspnet_isapi!HttpCompletion::processRequestInManagedCode+0x13
    06 009fff24 79e8cb80 aspnet_isapi!HttpCompletion::processCompletion+0x2
    07 009fff68 792e9155 aspnet_isapi!CorThreadPoolWorkitemCallback+0x2
    08 009fff98 792e92cf mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0x2
    09 009fffb8 77e4a990 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x12
    0a 009fffec 00000000 kernel32!BaseThreadStart+0x3

    When the site fails to restore, each new connection creates a Concurrent request that is never answered. We have allowed some of these to go past 3 hours by removing the machine they were running on from the Load Balance

    The fun part is when this happens on a site that has been set to use a Web Garden, this will happen to just one of the threads, so the Automated checking software won't pick it up as a failure, because only every 3rd request is failing

    Where can we go from here?
    =?Utf-8?B?UGlnZ2xldDE4Nw==?=, Jun 8, 2004
    1. Advertisements

  2. =?Utf-8?B?UGlnZ2xldDE4Nw==?=

    Natty Gur Guest


    mmm it wont be easy but certainly will be solved.

    I suggest you to use post mortem debug tolls such as adplus and windows
    debug tools. I add a link that will guide you through this process :

    Those tools will help you to track the problem.

    Natty Gur[MVP]

    blog :
    Mobile: +972-(0)52-8888377

    *** Sent via Devdex ***
    Don't just participate in USENET...get rewarded for it!
    Natty Gur, Jun 9, 2004
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Adam Clark

    ASPDOTNET AppPool Crashing

    Adam Clark, Oct 27, 2005, in forum: ASP .Net
    Alvin Bruney - ASP.NET MVP
    Oct 27, 2005
  2. Ben Dewey

    MS Powerpoint AppPool Identity

    Ben Dewey, May 11, 2006, in forum: ASP .Net
    May 12, 2006
  3. =?Utf-8?B?QWxoYW1icmEgRWlkb3MgS2lxdWVuZXQ=?=

    restart AppPool by code or script

    =?Utf-8?B?QWxoYW1icmEgRWlkb3MgS2lxdWVuZXQ=?=, Oct 3, 2007, in forum: ASP .Net
    Oct 3, 2007
  4. JM_newsgroups
    Dec 12, 2008
  5. Kevin C
    Hernan de Lahitte
    Nov 19, 2004
  6. 401 if AppPool is not Network Service

    , Mar 23, 2006, in forum: ASP .Net Security
    Mar 23, 2006
  7. Navin Mishra

    How to detect if hosting IIS AppPool is recycled or stopped ?

    Navin Mishra, May 2, 2005, in forum: ASP .Net Web Services
    Navin Mishra
    May 2, 2005
  8. Navin Mishra

    Monitor IIS AppPool from ASP.NET web service

    Navin Mishra, Jul 28, 2005, in forum: ASP .Net Web Services
    Navin Mishra
    Aug 6, 2005