cgi concurrency approaches?

Discussion in 'Python' started by Paul Rubin, Jan 23, 2004.

  1. Paul Rubin

    Paul Rubin Guest

    I'm wondering if folks here have favorite lightweight ways of dealing
    with concurrency in cgi's. Take a simple case:

    You want to write a cgi that implements a simple counter. The first
    time it's called, it prints "1". The next time, "2", etc. The naive
    implementation looks something like:

    fd = open("counter.num", "rw")
    n = int(fd.read())
    fd.seek(0, 0)
    fd.write("%d"% n+1)
    print "Content-type: text/html\n\n"
    print n

    but of course there's the obvious race condition if two people hit the
    cgi at the same time.

    Fancier solutions include running an transactional database in another
    process and connecting to it, setting up a daemon that remembers the
    counter value in memory and serializes access through a socket that
    the cgi opens, using a file-system specific hack like linking to
    counter file to lock it, having a timeout/retry if the counter is
    locked, with a possible hangup if a lock file gets left around by
    accident, etc. Each is a big pain in the neck.

    Anyone have some simpler approaches?
     
    Paul Rubin, Jan 23, 2004
    #1
    1. Advertising

  2. Paul Rubin wrote:

    > I'm wondering if folks here have favorite lightweight ways of dealing
    > with concurrency in cgi's. Take a simple case:
    >
    > You want to write a cgi that implements a simple counter. The first
    > time it's called, it prints "1". The next time, "2", etc. The naive
    > implementation looks something like:
    >
    > fd = open("counter.num", "rw")
    > n = int(fd.read())
    > fd.seek(0, 0)
    > fd.write("%d"% n+1)
    > print "Content-type: text/html\n\n"
    > print n
    >
    > but of course there's the obvious race condition if two people hit the
    > cgi at the same time.
    >
    > Fancier solutions include running an transactional database in another
    > process and connecting to it, setting up a daemon that remembers the
    > counter value in memory and serializes access through a socket that
    > the cgi opens, using a file-system specific hack like linking to
    > counter file to lock it, having a timeout/retry if the counter is
    > locked, with a possible hangup if a lock file gets left around by
    > accident, etc. Each is a big pain in the neck.
    >
    > Anyone have some simpler approaches?


    I'd just run Xitami (www.xitami.org) and connect a LRWP (Long-Running
    Web Process). I'm not familliar with Apache, so I don't know if
    mod_python is equivalent.

    - Josiah
     
    Josiah Carlson, Jan 23, 2004
    #2
    1. Advertising

  3. Paul Rubin

    Rene Pijlman Guest

    Paul Rubin <http://>:
    >I'm wondering if folks here have favorite lightweight ways of dealing
    >with concurrency in cgi's. Take a simple case:
    >
    >You want to write a cgi that implements a simple counter. The first
    >time it's called, it prints "1". The next time, "2", etc.

    [...]
    >Anyone have some simpler approaches?


    Use mod_python and keep the counter in a Python variable (if the value
    needs not be persistent).
    http://www.modpython.org/

    --
    René Pijlman
     
    Rene Pijlman, Jan 23, 2004
    #3
  4. Paul Rubin

    Matt Goodall Guest

    On Fri, 2004-01-23 at 10:04, Rene Pijlman wrote:
    > Paul Rubin <http://>:
    > >I'm wondering if folks here have favorite lightweight ways of dealing
    > >with concurrency in cgi's. Take a simple case:
    > >
    > >You want to write a cgi that implements a simple counter. The first
    > >time it's called, it prints "1". The next time, "2", etc.

    > [...]
    > >Anyone have some simpler approaches?

    >
    > Use mod_python and keep the counter in a Python variable (if the value
    > needs not be persistent).
    > http://www.modpython.org/


    That will only work if you use Apache 2 in threaded mode. Apache 1 and
    Apache 2 in forking mode (multiple processes) will have a copy of the
    counter object per-process.

    Cheers, Matt

    --
    Matt Goodall, Pollenation Internet Ltd
    w: http://www.pollenation.net
    e:

    Any views expressed are my own and do not necessarily reflect the
    views of my employer.
     
    Matt Goodall, Jan 23, 2004
    #4
  5. Paul Rubin

    Rene Pijlman Guest

    Matt Goodall:
    >Rene Pijlman:
    >> Use mod_python and keep the counter in a Python variable

    >
    >That will only work if you use Apache 2 in threaded mode.


    Of course, good point.

    --
    René Pijlman
     
    Rene Pijlman, Jan 23, 2004
    #5
  6. Paul Rubin

    Alan Kennedy Guest

    [replying to someone else's reply because I missed the original]

    [Paul Rubin]
    > I'm wondering if folks here have favorite lightweight ways of dealing
    > with concurrency in cgi's,

    [snip]
    > but of course there's the obvious race
    > condition if two people hit the
    > cgi at the same time.
    >
    > Fancier solutions include running an transactional database in another
    > process and connecting to it, setting up a daemon that remembers the
    > counter value in memory and serializes access through a socket that
    > the cgi opens, using a file-system specific hack like linking to
    > counter file to lock it, having a timeout/retry if the counter is
    > locked, with a possible hangup if a lock file gets left around by
    > accident, etc. Each is a big pain in the neck.
    > Anyone have some simpler approaches?


    I don't know about a platform independent solution, but if you were
    willing to limit yourself to certain varieties of Unix, e.g. Linux, it
    would probably be relatively easy to implement a persistent counter
    using something like System V message queues, or shared memory +
    semaphores. This obviously will work only on systems which support
    System V IPC.

    Here is a brief page about System V IPC

    http://www.science.unitn.it/~fiorella/guidelinux/tlk/node56.html

    And here's a module, which I've never used, which purports to provide
    a python interface to System V IPC facilities, if available.

    http://www.theory.org/~mac4/software/

    Given the above facilities, there are two ways I would approach the
    problem of a persistent counter.

    1. Use a dedicated Queue, and hold the counter value inside a single
    message which "lives" on that queue. Those incrementing the counter
    read the single message from the queue, increment it and put it back
    on the queue. Any processes awaiting access to the "counter message"
    would then do a blocking read on the queue. Since all puts and gets of
    messages are atomic, you are guaranteed only atomic updates to your
    counter. However, you could lock your system up if one of your
    accessing CGI processes did not put the counter back again! You can
    use timeouts as well, if my memory serves (it's been a long time since
    I used System V IPC). You can also get fancy, with priorities, etc.

    2. Store the value in pre-created shared memory partition, protected
    by a semaphore. Since there are shared memory python modules for
    several platforms, you might have a better chance with this approach
    on non-unix platforms.

    The Python Object Sharing (POSH) module might provide wrappers to some
    useful functionality, although the module itself might be a little
    heavyweight for providing a simple persistent counter.

    http://poshmodule.sourceforge.net/posh/html/posh.html

    HTH,

    --
    alan kennedy
    ------------------------------------------------------
    check http headers here: http://xhaus.com/headers
    email alan: http://xhaus.com/contact/alan
     
    Alan Kennedy, Jan 23, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Calvin
    Replies:
    2
    Views:
    681
    Mike Treseler
    Jul 13, 2005
  2. Eli Bendersky
    Replies:
    1
    Views:
    1,204
    Mike Treseler
    Mar 1, 2006
  3. Vicky

    better approaches

    Vicky, Jan 31, 2004, in forum: ASP .Net
    Replies:
    6
    Views:
    400
    Vicky
    Feb 2, 2004
  4. Jason
    Replies:
    1
    Views:
    3,520
    Chris Smith
    Jan 12, 2004
  5. Kylotan
    Replies:
    5
    Views:
    470
    Steve Holden
    Apr 15, 2004
Loading...

Share This Page