File locking, portably?

Discussion in 'Ruby' started by Hugh Sasse Staff Elec Eng, Apr 15, 2004.

  1. Searching the web and books for information on this, I can't seem to
    find a definitive yes or no to my question:

    "Is there a portable way to do file locking?"

    Some of the problems that are mentioned are that NFS systems make
    almost everything non-atomic, locking methods often depend on fcntl
    which is not available on all systems, and the dreaded race
    condition when test and set are non-atomic.

    Ruby can be used on the Mac, PC, and Unix, so I'm really after
    something that portable. I can't use a Mutex because I need this to
    be exclusive across process boundaries (several invokations of the
    program).

    My searching suggests this is a common problem, but the answer to it
    is rare!

    Thank you
    Hugh
    Hugh Sasse Staff Elec Eng, Apr 15, 2004
    #1
    1. Advertising

  2. Hugh Sasse Staff Elec Eng

    Ara.T.Howard Guest

    On Thu, 15 Apr 2004, Hugh Sasse Staff Elec Eng wrote:

    > Searching the web and books for information on this, I can't seem to
    > find a definitive yes or no to my question:
    >
    > "Is there a portable way to do file locking?"
    >
    > Some of the problems that are mentioned are that NFS systems make
    > almost everything non-atomic, locking methods often depend on fcntl
    > which is not available on all systems, and the dreaded race
    > condition when test and set are non-atomic.
    >
    > Ruby can be used on the Mac, PC, and Unix, so I'm really after
    > something that portable. I can't use a Mutex because I need this to
    > be exclusive across process boundaries (several invokations of the
    > program).
    >
    > My searching suggests this is a common problem, but the answer to it
    > is rare!
    >
    > Thank you
    > Hugh


    i been doing alot of experiments with locking myself, mainly on nfs systems
    for some designs for a distributed work queue i'm working on, and have come to
    largely the same conclusions. however, you defintely want fcntl based locking
    for NFS systems. as far as i know any posix compliant sytem has fcntl but i'm
    a windows dummy (windows people insert correction)...

    you might want to check out a few things i've done - most of them were done
    __very__ quickly and further testing is in order but:

    * c ext to replace File.flock with fcntl based impl

    http://www.codeforpeople.com/lib/ruby/posixlock/

    * a simpler, but less portable?, pure ruby solution provided by matz

    http://www.codeforpeople.com/lib/ruby/nfslock/

    * interface to liblockfile (man 1 lockfile)

    http://www.codeforpeople.com/lib/ruby/lockfile/


    the tests i've been running (day at a time) consist of multiple processes on
    multiple hosts competing to update a queue in an ordered fashion... if the
    queue is ever out of order, or a marshall error is thrown, the test 'fails'.
    i also mark the times each node aquires the lock and gather stats on the
    min/max/avg time required to obtain the lock. i've run using all three
    methods above, plus system calls to lockfile, for my locking mechanism and
    have the following observation

    * they all work on nfs - i get a core dump every now and again in the
    liblockfile impl which is almost certainly a bug in my own code

    * lockd sucks at giving at sort of 'even' distribution to the processes,
    what i generally see is one node hogging the lock for a while, then
    eventually lockd seems to realize this and give it another node for a
    while. for my uses this is not a big deal since the competition in
    production would not actually be that fierce... it DOES work though with
    a sufficiently new lockd impl or a rather expensive netap...

    * the max time between locks for 6 or so process competing for a fcntl based
    lock on our systems is around 30 seconds

    * lockfile seems to work really well - given max/min/avg of about 1 sec for
    all nodes. this really suprised me.

    * the big drawback to lockfiles is potential hangs and inability to grant
    read-locks. there is serious locking package on CPAN which claims to do
    this (read/write nfs safe lockfiles) at

    http://search.cpan.org/~bbb/File-NFSLock-1.20/lib/File/NFSLock.pm

    the idea of this seems quite sketchy. i have not tested it.


    if you are interested in my test code drop me a line - it's one script that
    you run on all the node, and a monitoring script that goes with it.... nice
    a terrible like my testing code tends to be...

    in any case - i would think implementing the algorithim used by liblockfile in
    ruby might be a good solution. the hard work at making things portable has
    been done for you by matz and co. i made a stab at that (it's in the lockfile
    package) but it is NOT finished... i should probably take it out of there...

    i'm very interested in any findings you have along these lines. please keep
    us informed.

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
    Ara.T.Howard, Apr 15, 2004
    #2
    1. Advertising

  3. As a side note on implementation, is there a particular reason you chose
    to use filelocking when something like drb would allow you to do thread
    atomic gets across a network without necessarily having shared disk
    space? Let alone a shared tuplespace which does even more fun things
    for you?

    Charlie
    Charles Comstock, Apr 15, 2004
    #3
  4. Hugh Sasse Staff Elec Eng

    Ara.T.Howard Guest

    On Wed, 14 Apr 2004, Charles Comstock wrote:

    > As a side note on implementation, is there a particular reason you chose
    > to use filelocking when something like drb would allow you to do thread
    > atomic gets across a network without necessarily having shared disk
    > space? Let alone a shared tuplespace which does even more fun things
    > for you?
    >
    > Charlie
    >


    that's a good idea... i've thought of it myself, but our sysads are nazis
    about opening any ports...

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
    Ara.T.Howard, Apr 15, 2004
    #4
  5. On Thu, 15 Apr 2004, Charles Comstock wrote:

    > As a side note on implementation, is there a particular reason you chose
    > to use filelocking when something like drb would allow you to do thread
    > atomic gets across a network without necessarily having shared disk
    > space? Let alone a shared tuplespace which does even more fun things
    > for you?


    Because I using this for (reading and) writing the config file which
    tells other processes where to find my DRb server, not wanting to
    open things up so widely by having a Ring server (which I don't
    fully understand yet anyway)...
    >
    > Charlie
    >


    Hugh
    Hugh Sasse Staff Elec Eng, Apr 15, 2004
    #5
  6. Hugh Sasse Staff Elec Eng

    John Platte Guest

    On 2004 Apr 15, at 1:49, Ara.T.Howard wrote:

    >> something like drb...

    >
    > that's a good idea... i've thought of it myself, but our sysads are
    > nazis
    > about opening any ports...


    SSH tunneling?

    --
    Ryan "John" Platte
    Custom services, NIKA Consulting
    http://nikaconsulting.com/
    John Platte, Apr 15, 2004
    #6
  7. On Thu, 15 Apr 2004, John Platte wrote:

    > On 2004 Apr 15, at 1:49, Ara.T.Howard wrote:
    >
    > >> something like drb...

    > >
    > > that's a good idea... i've thought of it myself, but our sysads are
    > > nazis
    > > about opening any ports...

    >
    > SSH tunneling?


    This is getting somewhat less portable..... :)

    Hugh
    Hugh Sasse Staff Elec Eng, Apr 15, 2004
    #7
  8. Hugh Sasse Staff Elec Eng

    Ara.T.Howard Guest

    On Thu, 15 Apr 2004, Hugh Sasse Staff Elec Eng wrote:

    > On Thu, 15 Apr 2004, John Platte wrote:
    >
    > > On 2004 Apr 15, at 1:49, Ara.T.Howard wrote:
    > >
    > > >> something like drb...
    > > >
    > > > that's a good idea... i've thought of it myself, but our sysads are
    > > > nazis
    > > > about opening any ports...

    > >
    > > SSH tunneling?

    >
    > This is getting somewhat less portable..... :)


    for me it's o.k. - we are all linux. see my other thread about pty/expect and
    you'll see where i'm headed there ;-) something like:

    * nfs mounted home dir for all processing node's user

    * systems starts and generates a random passhprase
    * feeds passphrase into ssh-keygen/ssh-agent/ssh-add
    * copies id_rsa.pub to authorized_keys (remember nfs home dir)
    * now all nodes can tunnel drb through ssh

    why generate the passphrase? because these machine routinues go down/come up
    in the middle of the night and ssh-add would hang, waiting for a passphrase so
    i could either

    a) embed the passphrase in the source
    b) randomly generate on startup so it is know only to the program

    i realize this is terrible but what else can i do when you won't open up ports
    and can't see to keep lockd running?

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
    Ara.T.Howard, Apr 15, 2004
    #8
  9. Hugh Sasse Staff Elec Eng

    John Platte Guest

    On 2004 Apr 15, at 9:54, Ara.T.Howard wrote:

    >>> SSH tunneling?

    >>
    >> This is getting somewhat less portable..... :)

    >
    > for me it's o.k. - we are all linux.


    Y'all are probably aware, but I should mention that PuTTY has quite a
    complete SSH feature set for Windows (including ssh-agent), and is
    MIT-licensed.

    --
    Ryan "John" Platte
    Custom services, NIKA Consulting
    http://nikaconsulting.com/
    John Platte, Apr 15, 2004
    #9
  10. On Fri, 16 Apr 2004, John Platte wrote:

    > On 2004 Apr 15, at 9:54, Ara.T.Howard wrote:
    >
    > >>> SSH tunneling?
    > >>
    > >> This is getting somewhat less portable..... :)

    > >
    > > for me it's o.k. - we are all linux.

    >
    > Y'all are probably aware, but I should mention that PuTTY has quite a
    > complete SSH feature set for Windows (including ssh-agent), and is
    > MIT-licensed.


    Yes, I'm using that, but have not done much with it
    programmatically. But this seems a long way round, with dependencies,
    to get file locking... Is there anything just using ruby that is
    reliable?

    Hugh
    Hugh Sasse Staff Elec Eng, Apr 15, 2004
    #10
  11. Hugh Sasse Staff Elec Eng

    Ara.T.Howard Guest

    On Fri, 16 Apr 2004, Hugh Sasse Staff Elec Eng wrote:

    > On Fri, 16 Apr 2004, John Platte wrote:
    >
    > > On 2004 Apr 15, at 9:54, Ara.T.Howard wrote:
    > >
    > > >>> SSH tunneling?
    > > >>
    > > >> This is getting somewhat less portable..... :)
    > > >
    > > > for me it's o.k. - we are all linux.

    > >
    > > Y'all are probably aware, but I should mention that PuTTY has quite a
    > > complete SSH feature set for Windows (including ssh-agent), and is
    > > MIT-licensed.

    >
    > Yes, I'm using that, but have not done much with it
    > programmatically. But this seems a long way round, with dependencies,
    > to get file locking... Is there anything just using ruby that is
    > reliable?



    File.flock should work - execpt for NFS. if you are on a hetrogeneous NFS
    systems then i'd imagine you have much bigger problems though...

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
    Ara.T.Howard, Apr 15, 2004
    #11
  12. Hugh Sasse Staff Elec Eng wrote:

    > On Thu, 15 Apr 2004, Charles Comstock wrote:
    >
    >
    >>As a side note on implementation, is there a particular reason you chose
    >>to use filelocking when something like drb would allow you to do thread
    >>atomic gets across a network without necessarily having shared disk
    >>space? Let alone a shared tuplespace which does even more fun things
    >>for you?

    >
    >
    > Because I using this for (reading and) writing the config file which
    > tells other processes where to find my DRb server, not wanting to
    > open things up so widely by having a Ring server (which I don't
    > fully understand yet anyway)...
    >
    >>Charlie
    >>

    >
    >
    > Hugh
    >
    >


    Yea I don't understand Ring servers either, I wish someone would explain
    them. Is it doing a portscan or something, I just don't follow how it
    magically finds the Ringer server.

    You could of course have a single config/reader writer server of course,
    but I understand why that might be excessive.

    My most recent DRb app does use shared home dirs for reading the server
    location, but other than that I try to avoid that and send more over the
    wire. I guess it justs feels more portable that way. It's a fun little
    project though, a client/agent/server for running a programming contest,
    but ensuring that each contestee is not sharing cpu power with other
    contest members by running there code in my userspace on there machine.
    Works pretty slick, and is only about 250 lines of code total. It's a
    project I wouldn't even undertake in another language, but in ruby it's
    a snap.

    Charlie
    Charles Comstock, Apr 16, 2004
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. news
    Replies:
    0
    Views:
    352
  2. Simon G Best
    Replies:
    3
    Views:
    379
    Simon G Best
    Jul 17, 2003
  3. Tero Saarni
    Replies:
    2
    Views:
    275
    Tero Saarni
    Aug 7, 2003
  4. John J. Lee

    beeping portably

    John J. Lee, Mar 20, 2005, in forum: Python
    Replies:
    4
    Views:
    362
    nicke
    Mar 22, 2005
  5. Timasmith
    Replies:
    4
    Views:
    438
    Bjorn Borud
    Nov 1, 2006
Loading...

Share This Page