Simple discussion of python cgi approaches?

Discussion in 'Python' started by Kylotan, Apr 13, 2004.

  1. Kylotan

    Kylotan Guest

    [Apologies if anyone sees this twice - at first I attempted to post it
    via my ISP's news server, but my ISP is notorious for poor news
    handling and I don't see it on Google Groups as having got through

    I've been writing a CGI app with Python 2.3 and Apache. It's ok, but a
    bit slow, and I assume a significant part of this is the process
    start-up time, among other things. So I'm interested in alternatives
    or optimisations. The problem is, there doesn't seem to be the kind of
    information out there that I want. There seem to be plenty of places
    that throw a list of URLs at you (eg.,, and a few
    sites that will give the absolute basics of CGI programming itself
    (which I don't need).

    I'm particularly confused by a few people advocating things like
    Xitami + LRWP for being very simple, yet it can't possibly be simpler
    than just dropping a cgi script into a directory, as I do with Apache,
    and the docs on LRWP
    ( make it look
    like I'd have to do significant rewriting of the scripts. It describes
    itself as being like FastCGI which, if true, doesn't exactly endear
    FastCGI to me either.

    All I want is a short description on the technologies that very
    closely resemble standard CGI, what they have to offer me, and what
    sort of alterations I would have to make to my installation or code in
    order to use these. Does such a comparison exist already? If not, is
    anybody able to comment on their own experiences with Python CGI apps?
    Kylotan, Apr 13, 2004
    1. Advertisements

  2. Am Mon, 12 Apr 2004 18:27:58 -0700 schrieb Kylotan:


    I started with Zope in 2001. This is very different
    than CGI. Then I switched to quixote. It is very much
    like CGI. Every HTTP-Request results in an method call
    which gets an request object as argument.

    In my situation it is enough to run quixote with plain cgi
    (no SCGI, mod_python, ...): About three request per minute.
    Loading the interpreter for every request is not the

    Thomas Guettler, Apr 13, 2004
    1. Advertisements

  3. Kylotan

    Steve Holden Guest

    There are many differeing opinions on this subject, so maybe you'll just
    have to accept that responses reflect the prejudices of their
    authors. They will at least usually be better-informed than random web
    sites, one might hope.
    Well, you said you are looking for "alternatives or optimaizations", so
    it would appear that you feel you are approaching the limits of your
    simple approach. It's a very unsual solution that will be simpler,
    faster and cheaper than the "obvious" one, though such solutions aren't
    unheard of, even in the web world.
    Well, let me briefly summaries the advantages of Xitami/LRWP and
    mod_python, two approaches with which I am familiar, as compared with
    good 'ole vanilla CGI.

    Both of these systems have the advantage that once your web
    functionality is stable (i.e. the programs aren't changing each time you
    want the run) you avoid the overhead of reloading both the interpreter
    and the source code of your page generators (CGI scripts) for each page.

    mod_python effectively works by integrating the interpreter into Apache
    in such a way that each directory can, if necessary, have a separate
    interpreter instance (to avoid namespace clashes between different
    applications). Modules are loaded on-demand, and stay resident in the
    Apache process unless changed. Sometimes there are problems if an
    indirectly-imported module changes, since mod_python only checks those
    it imports itself.

    The intergration is very tight, and you can write Apache handlers of all
    kinds in Python. I have experimented myself with writing request
    handlers, with results briefly reported at PyCON this year. It's a good
    general-purpose approach for uni-processor solutions.

    Xitami is slightly different: the server is deliberately written as a
    lightweight asynchronous request handler which is capable of queueing
    requests until a service routine becomes available. The LRWP linkage
    isn't actually as complicated as it might seem, and the only major
    difference is that a service process must register with the server to
    handle a class of URLs before it receives requests.

    This gives you two immediate benefits: firstly, since network sockets
    are used to communicate between the web server and the service process,
    the web server can act as a conduit to external functionality very
    easily - it's trivial to build a remote LRWP, since the only difference
    between a local and a remote LRWP is the fact that the remote one will
    use an external IP address for the web server rather than "localhost".
    Second, since the web server is perfectly happy to accept multiple
    registrations for a given class of URLs, and since each registration can
    come from a different host, the asynchronous nature of the Xitami system
    allows you to achieve potentially great scalability.

    I described this approach in "Python Web Programming", since it seemed
    to me to offer the easiest way to build web-based Python applications
    which could potentially scale to hundreds of processors. The really nice
    part is that you can add more LRWPs to any URL class that needs more
    power, dynamically and without trauma.

    I even built a system on Windows that had a URL to start up LRWPs on
    demand, and to shut them down, under client control. It's very flexible.

    Clearly anything that has performance benefits over CGI is likely to be
    a little more difficult to get running but believe me, the extra effort
    isn't that great when you consider the rewards.


    Steve Holden, Apr 13, 2004
  4. It should be noted that Quixote's CGI publisher can also be run under
    FastCGI; it should automatically detect when FastCGI is being used and
    service multiple requests in a single process. Most FastCGI implementations
    work like that (because the spec suggests it), so maybe just using FastCGI
    with the existing CGI scripts will be sufficient for the original poster.

    A.M. Kuchling, Apr 13, 2004
  5. Kylotan

    Kylotan Guest

    I know. I guess I am just spoiled by the simplicity of using things
    like Psyco
    that I sometimes forget that not everything comes for free!
    Yet I'm guessing it requires a significant amount of special-case code
    use its own API? Looking at the manual it seems non-trivial to get it
    to work with scripts that were originally written to consume stdin and
    emit to stdout.
    The main problem I face is that I don't know what platform my scripts
    eventually run on. I may not have the ability to install extra
    software so I
    need to keep my scripts close to the CGI standard. That is, one file
    per script,
    using the cgi module for form processing and print for output. I don't
    having to redirect stdout or call these scripts indirectly instead of
    their __main__ functions, but writing to specific APIs (beyond a small
    layer, of course) is not an option at the moment.

    So I'm hoping for simple solutions that will involve no more than a
    few changes
    to each script, whereas most of the solutions that seem to be
    available require
    me to use their own request objects, or to treat pages as callable
    functions within an overall page as mod_python seems to require.
    Kylotan, Apr 14, 2004
  6. Kylotan

    Steve Holden Guest

    Kylotan wrote:
    Well, what you've just said probably rules out both the approaches I
    chose to write about, but you've also defined your problem much more
    closely and so someone else migth be able to offer more helpful advice.

    Steve Holden, Apr 15, 2004
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.