auto-deleted explcitily named temp files - possible?

Discussion in 'Perl Misc' started by odigity@gmail.com, Apr 4, 2005.

  1. Guest

    OK, so I want to have my cake and eat it to.

    I want to be able to control access to resources (in this case, CPUs)
    by using named temp files. Imagine you have twenty dual-cpu hosts with
    shared mount points, and several applications that each run in a
    distributed fashion. I don't want two apps both running on host01 and
    ignoring the fact that host02 is free.

    So I came up with the idea that there would be a directory - say
    /resource - where files would be created indicating that the cpu in
    question is being used. For example, if an app starts and launches two
    processes on host01, saturating both CPUs, it would create
    /resource/host01.1 and /resource/host01.2, and then delete them when
    finished. That way apps can efficiently use processor resources with
    tempfiles as a super-simple communication mechanism.

    One requirement is that the lockfile automatically get deleted no
    matter what happened to the process. This is imperative, since if it
    is possible for a process to exit and leave the lockfile behind, other
    apps will continue to think that resource is in use and not use it.
    Now, the usual trick for this is creating the file then immediately
    unlinking it, but this technique leaves the file nameless, which
    defeats the purpose of having the file open.

    I can't find any other way to open a named file and guarantee that it
    is deleted when the process exits. Any ideas?
    , Apr 4, 2005
    #1
    1. Advertising

  2. Guest

    wrote:
    ....
    > So I came up with the idea that there would be a directory - say
    > /resource - where files would be created indicating that the cpu in
    > question is being used. For example, if an app starts and launches two
    > processes on host01, saturating both CPUs, it would create
    > /resource/host01.1 and /resource/host01.2, and then delete them when
    > finished. That way apps can efficiently use processor resources with
    > tempfiles as a super-simple communication mechanism.
    >
    > One requirement is that the lockfile automatically get deleted no
    > matter what happened to the process. This is imperative, since if it
    > is possible for a process to exit and leave the lockfile behind, other
    > apps will continue to think that resource is in use and not use it.


    Each process should not only create the file, but also lock the file
    using flock. Then whatever process is looking for a free CPU, should, if
    it doesn't find one, go through all the files making sure they are
    actually locked (by trying to flock them), and if it finds ones that aren't
    already locked it knows that they have been abandoned by dead process.

    Of course, this depends on flock working properly across the network.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , Apr 4, 2005
    #2
    1. Advertising

  3. Anno Siegel Guest

    <> wrote in comp.lang.perl.misc:
    > wrote:
    > ...
    > > So I came up with the idea that there would be a directory - say
    > > /resource - where files would be created indicating that the cpu in
    > > question is being used. For example, if an app starts and launches two
    > > processes on host01, saturating both CPUs, it would create
    > > /resource/host01.1 and /resource/host01.2, and then delete them when
    > > finished. That way apps can efficiently use processor resources with
    > > tempfiles as a super-simple communication mechanism.
    > >
    > > One requirement is that the lockfile automatically get deleted no
    > > matter what happened to the process. This is imperative, since if it
    > > is possible for a process to exit and leave the lockfile behind, other
    > > apps will continue to think that resource is in use and not use it.

    >
    > Each process should not only create the file, but also lock the file
    > using flock. Then whatever process is looking for a free CPU, should, if
    > it doesn't find one, go through all the files making sure they are
    > actually locked (by trying to flock them), and if it finds ones that aren't
    > already locked it knows that they have been abandoned by dead process.


    I was going to suggest something similar, only I wouldn't bother with
    deleting the lock files at all. Just try to create them (using
    sysopen with O_EXCL) in case it isn't there, then rely on the existence
    of locks alone.

    Anno
    Anno Siegel, Apr 5, 2005
    #3
  4. Anno Siegel Guest

    <> wrote in comp.lang.perl.misc:
    > wrote:
    > ...
    > > So I came up with the idea that there would be a directory - say
    > > /resource - where files would be created indicating that the cpu in
    > > question is being used. For example, if an app starts and launches two
    > > processes on host01, saturating both CPUs, it would create
    > > /resource/host01.1 and /resource/host01.2, and then delete them when
    > > finished. That way apps can efficiently use processor resources with
    > > tempfiles as a super-simple communication mechanism.
    > >
    > > One requirement is that the lockfile automatically get deleted no
    > > matter what happened to the process. This is imperative, since if it
    > > is possible for a process to exit and leave the lockfile behind, other
    > > apps will continue to think that resource is in use and not use it.

    >
    > Each process should not only create the file, but also lock the file
    > using flock. Then whatever process is looking for a free CPU, should, if
    > it doesn't find one, go through all the files making sure they are
    > actually locked (by trying to flock them), and if it finds ones that aren't
    > already locked it knows that they have been abandoned by dead process.


    I was going to suggest something similar, only I wouldn't bother with
    deleting the lock file at all. Just try to create it (using
    sysopen with O_EXCL) in case it isn't there, then rely on the existence
    of locks alone.

    Anno
    Anno Siegel, Apr 5, 2005
    #4
  5. Guest

    Both good ideas. I believe flock does work properly over NFS. I'll
    give it a shot!

    -ofer
    , Apr 5, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    2
    Views:
    570
    Mark P
    May 9, 2005
  2. Replies:
    0
    Views:
    436
  3. Chumley the Walrus
    Replies:
    2
    Views:
    252
    Tom Gosselin
    Aug 10, 2004
  4. A. Farber
    Replies:
    3
    Views:
    255
    Ben Morrow
    Mar 3, 2004
  5. PerlFAQ Server
    Replies:
    0
    Views:
    258
    PerlFAQ Server
    Apr 26, 2011
Loading...

Share This Page