More mod_wsgi weirdness: process restarts on redirect

Discussion in 'Python' started by Ron Garret, Jan 29, 2009.

  1. Ron Garret

    Ron Garret Guest

    I'm running mod_wsgi under apache (on Debian etch so it's a somewhat out
    of date version, though I doubt that has anything to do with this issue).

    I have a little test page that displays the process ID under which my
    app is running, and some global state, so I can tell when the wsgi app
    gets reloaded. This mostly only happens when I restart apache, but it
    also seems to happen when my WSGI app does an HTTP 302 redirect. (I'm
    actually using Yaro and calling req.redirect, but that only does a
    straightforward HTTP 302 redirect as far as I can tell.) It not only
    resets the global state, but changes process ID, so it seems to be doing
    a complete restart of mod_wsgi, which seems a little excessive.

    My question is: is this supposed to be happening? Or is this an
    indication that something is wrong, and if so, what?

    Thanks,
    rg
    Ron Garret, Jan 29, 2009
    #1
    1. Advertising

  2. Ron Garret wrote:
    > My question is: is this supposed to be happening? Or is this an
    > indication that something is wrong, and if so, what?


    You are probably just hitting a different instance of Apache, thus the
    different process ID.

    j
    Joshua Kugler, Jan 29, 2009
    #2
    1. Advertising

  3. Ron Garret

    Ron Garret Guest

    In article <>,
    Ron Garret <> wrote:

    > I'm running mod_wsgi under apache (on Debian etch so it's a somewhat out
    > of date version, though I doubt that has anything to do with this issue).
    >
    > I have a little test page that displays the process ID under which my
    > app is running, and some global state, so I can tell when the wsgi app
    > gets reloaded. This mostly only happens when I restart apache, but it
    > also seems to happen when my WSGI app does an HTTP 302 redirect. (I'm
    > actually using Yaro and calling req.redirect, but that only does a
    > straightforward HTTP 302 redirect as far as I can tell.) It not only
    > resets the global state, but changes process ID, so it seems to be doing
    > a complete restart of mod_wsgi, which seems a little excessive.
    >
    > My question is: is this supposed to be happening? Or is this an
    > indication that something is wrong, and if so, what?
    >
    > Thanks,
    > rg


    Here's a standalone WSGI app demonstrating the phenomenon:

    def redirect_test(env, start):
    if env['PATH_INFO']:
    start('302 Found', [('Location', '/')])
    return ['Redirecting']
    else:
    start('200 OK', [('Content-type', 'text/plain')])
    return ['PID', str(os.getpid())]
    pass

    application = redirect_test


    Fire this up under mod_wsgi and observe that the process ID stays the
    same when you reload the app. Now add a path component to trigger the
    redirect and observe that process ID changes.

    rg
    Ron Garret, Jan 29, 2009
    #3
  4. Ron Garret

    Ron Garret Guest

    In article <>,
    Joshua Kugler <> wrote:

    > Ron Garret wrote:
    > > My question is: is this supposed to be happening? Or is this an
    > > indication that something is wrong, and if so, what?

    >
    > You are probably just hitting a different instance of Apache, thus the
    > different process ID.


    Yep, that's what it turned out to be. I thought I had a
    WSGIDaemonProcess processes=1 directive in my config, but I had it in
    the wrong place (a different vhost) so it wasn't actually active.

    But that leaves me wondering why the redirect would reliably trigger
    switching processes. The reason I thought that I had the correct
    configuration and only had one process is that when I reloaded the
    non-redirected page I *always* got the same process ID. How does
    mod_wsgi decide which process (and which thread for that matter) to use?

    rg
    Ron Garret, Jan 30, 2009
    #4
  5. On Jan 30, 11:01 am, Ron Garret <> wrote:
    > In article <>,
    >  Joshua Kugler <> wrote:
    >
    > > Ron Garret wrote:
    > > > My question is: is this supposed to be happening?  Or is this an
    > > > indication that something is wrong, and if so, what?

    >
    > > You are probably just hitting a different instance of Apache, thus the
    > > different process ID.

    >
    > Yep, that's what it turned out to be.  I thought I had a
    > WSGIDaemonProcess processes=1 directive in my config, but I had it in
    > the wrong place (a different vhost) so it wasn't actually active.
    > http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
    > But that leaves me wondering why the redirect would reliably trigger
    > switching processes.  The reason I thought that I had the correct
    > configuration and only had one process is that when I reloaded the
    > non-redirected page I *always* got the same process ID.  How doesmod_wsgidecide which process  (and which thread for that matter) to use?


    Details on process/threading in mod_wsgi available at:

    http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading

    When using WSGIDaemonProcess directive, if you want a single process
    it is better to allow it to default to a single process and not have
    'processes=1'. As soon as you say 'processes=1' it will trigger
    wsgi.multiprocess to be True rather than default of False. This may
    sound counter intuitive, but is a little back door to allow
    wsgi.multiprocess to be set to True somehow when distributing an
    application across a cluster of machines where it does need to be True
    even if each machine only has a single process for that application.
    Tthat wsgi.multiprocess is True will not usually matter unless you are
    trying to use debugging middleware that require that there only be a
    single process.

    As to why you were getting a different process, because you were
    actually running in embedded mode due to WSGIDaemonProcess/
    WSGIProcessGroup being in wrong context, then what process was used
    was really up to Apache and how it works. Specifically it can have
    multiple processes that can listen on the HTTP port (80). Because only
    one should be listening at a time it uses a cross process mutex lock
    to mediate access. When a process handles a request, it gives up the
    lock. If using worker MPM then another thread in same process may get
    lock, or for either worker MPM or prefork MPM, then another process
    could get it. Which actually gets it is a bit indeterminate as simply
    depends on which process the operating system lets have the lock next.
    So, there is no strict rule one can say as to who would get it next.

    Graham
    Graham
    Graham Dumpleton, Jan 30, 2009
    #5
  6. Ron Garret

    Ron Garret Guest

    In article
    <>,
    Graham Dumpleton <> wrote:

    > On Jan 30, 11:01 am, Ron Garret <> wrote:
    > > In article <>,
    > >  Joshua Kugler <> wrote:
    > >
    > > > Ron Garret wrote:
    > > > > My question is: is this supposed to be happening?  Or is this an
    > > > > indication that something is wrong, and if so, what?

    > >
    > > > You are probably just hitting a different instance of Apache, thus the
    > > > different process ID.

    > >
    > > Yep, that's what it turned out to be.  I thought I had a
    > > WSGIDaemonProcess processes=1 directive in my config, but I had it in
    > > the wrong place (a different vhost) so it wasn't actually active.
    > > http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
    > > But that leaves me wondering why the redirect would reliably trigger
    > > switching processes.  The reason I thought that I had the correct
    > > configuration and only had one process is that when I reloaded the
    > > non-redirected page I *always* got the same process ID.  How
    > > doesmod_wsgidecide which process  (and which thread for that matter) to
    > > use?

    >
    > Details on process/threading in mod_wsgi available at:
    >
    > http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
    >
    > When using WSGIDaemonProcess directive, if you want a single process
    > it is better to allow it to default to a single process and not have
    > 'processes=1'. As soon as you say 'processes=1' it will trigger
    > wsgi.multiprocess to be True rather than default of False. This may
    > sound counter intuitive, but is a little back door to allow
    > wsgi.multiprocess to be set to True somehow when distributing an
    > application across a cluster of machines where it does need to be True
    > even if each machine only has a single process for that application.
    > Tthat wsgi.multiprocess is True will not usually matter unless you are
    > trying to use debugging middleware that require that there only be a
    > single process.
    >
    > As to why you were getting a different process, because you were
    > actually running in embedded mode due to WSGIDaemonProcess/
    > WSGIProcessGroup being in wrong context, then what process was used
    > was really up to Apache and how it works. Specifically it can have
    > multiple processes that can listen on the HTTP port (80). Because only
    > one should be listening at a time it uses a cross process mutex lock
    > to mediate access. When a process handles a request, it gives up the
    > lock. If using worker MPM then another thread in same process may get
    > lock, or for either worker MPM or prefork MPM, then another process
    > could get it. Which actually gets it is a bit indeterminate as simply
    > depends on which process the operating system lets have the lock next.
    > So, there is no strict rule one can say as to who would get it next.
    >
    > Graham
    > Graham


    Thanks!

    rg
    Ron Garret, Jan 30, 2009
    #6
  7. Ron Garret

    Guest

    Quoth Ron Garret <>:
    > In article <>,
    > Joshua Kugler <> wrote:
    >
    > > Ron Garret wrote:
    > > > My question is: is this supposed to be happening? Or is this an
    > > > indication that something is wrong, and if so, what?

    > >
    > > You are probably just hitting a different instance of Apache, thus the
    > > different process ID.

    >
    > Yep, that's what it turned out to be. I thought I had a
    > WSGIDaemonProcess processes=1 directive in my config, but I had it in
    > the wrong place (a different vhost) so it wasn't actually active.
    >
    > But that leaves me wondering why the redirect would reliably trigger
    > switching processes. The reason I thought that I had the correct
    > configuration and only had one process is that when I reloaded the
    > non-redirected page I *always* got the same process ID. How does
    > mod_wsgi decide which process (and which thread for that matter) to use?


    My WAG would be that when doing a refresh your client used a persistent
    http connection, and thus talked again to the same Apache child, but when
    it got a redirect it dropped the old connection and opened up a new one,
    thus having a high probability of hitting a new child. But that is,
    as I say, a WAG.

    --RDM
    , Jan 31, 2009
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. steve
    Replies:
    4
    Views:
    528
    Brian van den Broek
    Mar 13, 2005
  2. Scott SA

    apache/mod_wsgi daemon mode

    Scott SA, Feb 3, 2008, in forum: Python
    Replies:
    0
    Views:
    534
    Scott SA
    Feb 3, 2008
  3. Brian Smith

    FW: apache/mod_wsgi daemon mode

    Brian Smith, Feb 3, 2008, in forum: Python
    Replies:
    0
    Views:
    354
    Brian Smith
    Feb 3, 2008
  4. Scott SA

    RE: apache/mod_wsgi daemon mode

    Scott SA, Feb 3, 2008, in forum: Python
    Replies:
    1
    Views:
    1,070
    Graham Dumpleton
    Feb 3, 2008
  5. Navin Mishra

    Worker process restarts ?

    Navin Mishra, Jan 30, 2006, in forum: ASP .Net Web Services
    Replies:
    3
    Views:
    191
    Yunus Emre ALPÖZEN [MVP]
    Jan 31, 2006
Loading...

Share This Page