Question about long-running web scripts

Discussion in 'Python' started by Gilles, Oct 25, 2012.

  1. Gilles

    Gilles Guest

    Hello

    I'd like to check something about running Python web applications.

    Generally speaking, the reason scripts run faster when called through
    FastCGI or the mod_* modules, is because the interpreter is already up
    and running.
    But when running PHP scripts, this does nothing about fetching the
    file from disk, recompiling, rerunning it, and usually reconnecting to
    the database.

    OTOH, Python web scripts can be written as long-running scripts: In
    this case, what is the added-value of using FastCGI? Why can't the web
    server simply call the Python script directly, just like CGI?

    Thank you.
    Gilles, Oct 25, 2012
    #1
    1. Advertising

  2. Gilles

    Tim Golden Guest

    On 25/10/2012 12:45, Gilles wrote:
    > I'd like to check something about running Python web applications.
    >
    > Generally speaking, the reason scripts run faster when called
    > through FastCGI or the mod_* modules, is because the interpreter is
    > already up and running. But when running PHP scripts, this does
    > nothing about fetching the file from disk, recompiling, rerunning it,
    > and usually reconnecting to the database.
    >
    > OTOH, Python web scripts can be written as long-running scripts: In
    > this case, what is the added-value of using FastCGI? Why can't the
    > web server simply call the Python script directly, just like CGI?


    (Your question is a little confused at the end. I'm choosing to
    understand: why can't we just run Python one-shot, like CGI? The likely
    alternative meaning is: why can't the incoming request be routed to an
    already-running Python program -- which is not, of course, what CGI
    generally does. Hence my confusion).

    The answer is: it can. CGI is a protocol rather than anything else. In
    front of a CGI exchange is the browser (or some other web client).
    Behind it is some program which is capable of producing a valid HTTP
    response, including a Python program.

    It's perfectly possible to run a usable website against Python running
    one-shot. You won't get terrific performance out of it, but for a
    website which doesn't expect humungous amounts of traffic, it'll work fine.

    The amount of time it takes a half-decent, even shared, server to start
    up a Python process, connect to a database, pull stuff together, and
    send a response will likely not impact on an average user's experience.
    As long as too many of them don't try to do that at the same time.
    Exactly where the line is drawn will depend on your particular hosting
    solution, your assumed traffic, and your users' expectations as to
    responsiveness.

    TJG
    Tim Golden, Oct 25, 2012
    #2
    1. Advertising

  3. Gilles

    Gilles Guest

    On Thu, 25 Oct 2012 13:03:14 +0100, Tim Golden <>
    wrote:
    >(Your question is a little confused at the end. I'm choosing to
    >understand: why can't we just run Python one-shot, like CGI? The likely
    >alternative meaning is: why can't the incoming request be routed to an
    >already-running Python program -- which is not, of course, what CGI
    >generally does. Hence my confusion).


    Yes indeed. Sorry about the confusion.

    But actually, I didn't mean one-shot scripts, where the Python
    interpreter + script must be loaded each time, but rather: If I leave
    a Python running in an endless loop, why not just use either CGI or
    some other basic way to call the script instead of FastCGI?

    In the case of PHP, FastCGI makes sense, but I don't see the benefit
    for long-running Python scripts.
    Gilles, Oct 25, 2012
    #3
  4. Gilles

    David Hutto Guest

    On Thu, Oct 25, 2012 at 8:03 AM, Tim Golden <> wrote:
    > On 25/10/2012 12:45, Gilles wrote:
    >> I'd like to check something about running Python web applications.
    >>
    >> Generally speaking, the reason scripts run faster when called
    >> through FastCGI or the mod_* modules, is because the interpreter is
    >> already up and running. But when running PHP scripts, this does
    >> nothing about fetching the file from disk, recompiling, rerunning it,
    >> and usually reconnecting to the database.
    >>


    I'd say that is the same as py, unless it's a cron job to limit script
    iterations

    >> OTOH, Python web scripts can be written as long-running scripts: In
    >> this case, what is the added-value of using FastCGI? Why can't the
    >> web server simply call the Python script directly, just like CGI?


    The server should call a the script, or script.sleep()

    There are also server options to setup when a script is run, other
    than a cron jo for php.

    >
    > (Your question is a little confused at the end. I'm choosing to
    > understand: why can't we just run Python one-shot, like CGI? The likely
    > alternative meaning is: why can't the incoming request be routed to an
    > already-running Python program -- which is not, of course, what CGI
    > generally does. Hence my confusion).
    >
    > The answer is: it can. CGI is a protocol rather than anything else. In
    > front of a CGI exchange is the browser (or some other web client).
    > Behind it is some program which is capable of producing a valid HTTP
    > response, including a Python program.
    >
    > It's perfectly possible to run a usable website against Python running
    > one-shot. You won't get terrific performance out of it, but for a
    > website which doesn't expect humungous amounts of traffic, it'll work fine.
    >
    > The amount of time it takes a half-decent, even shared, server to start
    > up a Python process, connect to a database, pull stuff together, and
    > send a response will likely not impact on an average user's experience.
    > As long as too many of them don't try to do that at the same time.
    > Exactly where the line is drawn will depend on your particular hosting
    > solution, your assumed traffic, and your users' expectations as to
    > responsiveness.
    >
    > TJG
    > --
    > http://mail.python.org/mailman/listinfo/python-list




    --
    Best Regards,
    David Hutto
    CEO: http://www.hitwebdevelopment.com
    David Hutto, Oct 25, 2012
    #4
  5. Gilles

    Tim Golden Guest

    On 25/10/2012 13:40, Gilles wrote:
    > On Thu, 25 Oct 2012 13:03:14 +0100, Tim Golden <>
    > wrote:
    >> (Your question is a little confused at the end. I'm choosing to
    >> understand: why can't we just run Python one-shot, like CGI? The likely
    >> alternative meaning is: why can't the incoming request be routed to an
    >> already-running Python program -- which is not, of course, what CGI
    >> generally does. Hence my confusion).

    >
    > Yes indeed. Sorry about the confusion.
    >
    > But actually, I didn't mean one-shot scripts, where the Python
    > interpreter + script must be loaded each time, but rather: If I leave
    > a Python running in an endless loop, why not just use either CGI or
    > some other basic way to call the script instead of FastCGI?


    In essence, you're describing FastCGI. A Python program (or, indeed, any
    program) which uses FastCGI runs continuously and waits for the incoming
    request on a TCP socket (instead of as a sys.stdin stream + env vars
    immediately after process startup).

    The key description is here:

    http://www.fastcgi.com/drupal/node/6?q=node/15

    (The sections have no anchors; you're looking for the section titled "2.
    FastCGI Interface")

    TJG
    Tim Golden, Oct 25, 2012
    #5
  6. Gilles

    Gilles Guest

    On Thu, 25 Oct 2012 08:53:11 -0400, David Hutto
    <> wrote:
    >>> OTOH, Python web scripts can be written as long-running scripts: In
    >>> this case, what is the added-value of using FastCGI? Why can't the
    >>> web server simply call the Python script directly, just like CGI?

    >
    >The server should call a the script, or script.sleep()
    >
    >There are also server options to setup when a script is run, other
    >than a cron jo for php.


    Thanks. Does it mean that Python scripts that rely on either fcgi.py
    or WSGI really have some endless loop somewhere and will keep running
    once they're launched by FastCGI?
    Gilles, Oct 26, 2012
    #6
  7. Gilles

    Gilles Guest

    On Thu, 25 Oct 2012 14:24:16 +0100, Tim Golden <>
    wrote:
    >> But actually, I didn't mean one-shot scripts, where the Python
    >> interpreter + script must be loaded each time, but rather: If I leave
    >> a Python running in an endless loop, why not just use either CGI or
    >> some other basic way to call the script instead of FastCGI?

    >
    >In essence, you're describing FastCGI. A Python program (or, indeed, any
    >program) which uses FastCGI runs continuously and waits for the incoming
    >request on a TCP socket (instead of as a sys.stdin stream + env vars
    >immediately after process startup).


    Thanks for the clarification.

    Since, unlike PHP, the Python interpreter is not available in a
    FastCGI-capable version, this explains why the www server must be told
    which specific Python script to run through FastCGI.

    The reason I ask for all this, is that I want to understand how things
    work under the hood before relying on a Python framework to handle the
    nitty-gritty.
    Gilles, Oct 26, 2012
    #7
  8. Gilles

    Tim Golden Guest

    On 26/10/2012 10:58, Gilles wrote:
    > On Thu, 25 Oct 2012 14:24:16 +0100, Tim Golden <>
    > wrote:
    >>> But actually, I didn't mean one-shot scripts, where the Python
    >>> interpreter + script must be loaded each time, but rather: If I leave
    >>> a Python running in an endless loop, why not just use either CGI or
    >>> some other basic way to call the script instead of FastCGI?

    >>
    >> In essence, you're describing FastCGI. A Python program (or, indeed, any
    >> program) which uses FastCGI runs continuously and waits for the incoming
    >> request on a TCP socket (instead of as a sys.stdin stream + env vars
    >> immediately after process startup).

    >
    > Thanks for the clarification.
    >
    > Since, unlike PHP, the Python interpreter is not available in a
    > FastCGI-capable version, this explains why the www server must be told
    > which specific Python script to run through FastCGI.


    I think that this is the distinction you're making:

    PHP: mod_php (fastcgi mode) runs myscript.php

    Python: <some python fcgi frontend>.py runs myscript.py

    which is, essentially, true, not least because PHP and web apps are
    pretty much synonymous in many people's minds. Both ways: the only thing
    PHP does is web; the simplest route to a web app is PHP.

    Certainly there are Python equivalents (mod_python, mod_wsgi, etc.)
    which can run in effectively the same way as mod_php, and they could be
    configured to run an fcgi frontend script, I presume. There's always a
    certain confusion here because you can often one mechanisms (say,
    mod_wsgi) to act as another (say legacy one-shot CGI) and because some
    things are both mechanism and protocol and it's not always easy to tease
    the two apart.


    >
    > The reason I ask for all this, is that I want to understand how things
    > work under the hood before relying on a Python framework to handle the
    > nitty-gritty.


    Good scheme.

    TJG
    Tim Golden, Oct 26, 2012
    #8
  9. Gilles

    Gilles Guest

    On Fri, 26 Oct 2012 12:00:17 +0100, Tim Golden <>
    wrote:
    >Certainly there are Python equivalents (mod_python, mod_wsgi, etc.)
    >which can run in effectively the same way as mod_php, and they could be
    >configured to run an fcgi frontend script, I presume. There's always a
    >certain confusion here because you can often one mechanisms (say,
    >mod_wsgi) to act as another (say legacy one-shot CGI) and because some
    >things are both mechanism and protocol and it's not always easy to tease
    >the two apart.


    Thanks again.
    Gilles, Oct 26, 2012
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. George Marsaglia

    Assigning unsigned long to unsigned long long

    George Marsaglia, Jul 8, 2003, in forum: C Programming
    Replies:
    1
    Views:
    657
    Eric Sosman
    Jul 8, 2003
  2. Daniel Rudy

    unsigned long long int to long double

    Daniel Rudy, Sep 19, 2005, in forum: C Programming
    Replies:
    5
    Views:
    1,174
    Peter Shaggy Haywood
    Sep 20, 2005
  3. Mathieu Dutour

    long long and long

    Mathieu Dutour, Jul 17, 2007, in forum: C Programming
    Replies:
    4
    Views:
    457
    santosh
    Jul 24, 2007
  4. Bart C

    Use of Long and Long Long

    Bart C, Jan 9, 2008, in forum: C Programming
    Replies:
    27
    Views:
    778
    Peter Nilsson
    Jan 15, 2008
  5. Replies:
    4
    Views:
    238
    Dave A.
    Jun 24, 2005
Loading...

Share This Page