Distributed locking

Discussion in 'Python' started by James, May 14, 2009.

  1. James

    James Guest

    Hey all, I'm looking for suggestions on how to tackle distributed
    locking across several Python programs on several different machines.

    - the objects to be locked are uniquely identified by an integer
    - I need "one at a time" semantics for the lock: zero or one read-
    writer at any point
    - the operations to be performed on the objects are strictly limited
    in duration (no more than 10 seconds)

    As the operations have a maximum duration, my ideal solution would be
    to lock the objects, and have them automatically unlock after some
    time period even if the original lock-holder doesn't release them.

    I'm currently playing with memcached as a locking mechanism as it's
    simplicity and timeout fit my requirements well.

    There is a check-and-set operation which atomically writes new data
    iff the existing value hasn't been changed since we last looked.
    However, their CAS operation doesn't handle the case of a non-existent
    key..

    Does anyone have suggestions on how I can do distributed locking to
    take advantage of the time limit on operations?

    Many thanks,
    James
     
    James, May 14, 2009
    #1
    1. Advertising

  2. James schrieb:
    > Hey all, I'm looking for suggestions on how to tackle distributed
    > locking across several Python programs on several different machines.
    >
    > - the objects to be locked are uniquely identified by an integer
    > - I need "one at a time" semantics for the lock: zero or one read-
    > writer at any point
    > - the operations to be performed on the objects are strictly limited
    > in duration (no more than 10 seconds)
    >
    > As the operations have a maximum duration, my ideal solution would be
    > to lock the objects, and have them automatically unlock after some
    > time period even if the original lock-holder doesn't release them.
    >
    > I'm currently playing with memcached as a locking mechanism as it's
    > simplicity and timeout fit my requirements well.
    >
    > There is a check-and-set operation which atomically writes new data
    > iff the existing value hasn't been changed since we last looked.
    > However, their CAS operation doesn't handle the case of a non-existent
    > key..
    >
    > Does anyone have suggestions on how I can do distributed locking to
    > take advantage of the time limit on operations?


    I'd use a database. Maybe the time limit could be modeled with a
    transaction-timeout, but even if not, you should be able to model a
    solution with timestamps and distinct transactions to modify the locked row.

    Also, for the case of the non-existing lock, go with something like this:


    if not lock_exists(my_lock):
    lock_locks_table()
    if not lock_exists(my_lock):
    create_lock(my_lock)

    with my_lock:
    ...


    In the __enter__ and __exit__-methods of my_lock, you can perfom the
    necessary database-ops.


    Diez
     
    Diez B. Roggisch, May 14, 2009
    #2
    1. Advertising

  3. >>>>> James <> (J) wrote:

    >J> Hey all, I'm looking for suggestions on how to tackle distributed
    >J> locking across several Python programs on several different machines.


    Have you looked at the multiprocessing package? It has distributed
    Locks's with timeouts which might well fit your requirements.
    --
    Piet van Oostrum <>
    URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
    Private email:
     
    Piet van Oostrum, May 14, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. MovnOn
    Replies:
    0
    Views:
    406
    MovnOn
    Oct 14, 2003
  2. =?Utf-8?B?dHN1aQ==?=

    Client/Server v.s. Distributed application

    =?Utf-8?B?dHN1aQ==?=, Jun 4, 2004, in forum: ASP .Net
    Replies:
    4
    Views:
    3,028
    Jiho Han
    Jun 4, 2004
  3. =?Utf-8?B?SnVsaWFuIFNhbno=?=

    Distributed Transaction Error

    =?Utf-8?B?SnVsaWFuIFNhbno=?=, Nov 22, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    453
    =?Utf-8?B?SnVsaWFuIFNhbno=?=
    Nov 22, 2004
  4. =?Utf-8?B?SmltIEhlYXZleQ==?=

    Oracle - cannot ROLLBACK in a distributed tran

    =?Utf-8?B?SmltIEhlYXZleQ==?=, Feb 3, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    956
    =?Utf-8?B?SmltIEhlYXZleQ==?=
    Feb 3, 2005
  5. Timasmith
    Replies:
    4
    Views:
    458
    Bjorn Borud
    Nov 1, 2006
Loading...

Share This Page