Re: How to safely maintain a status file

Discussion in 'Python' started by Laszlo Nagy, Jul 8, 2012.

  1. Laszlo Nagy

    Laszlo Nagy Guest

    On Sun, 8 Jul 2012 21:29:41 +1000, Richard Baron Penman
    <> declaimed the following in gmane.comp.python.general:
    >> and then on startup read from tmp_file if status_file does not exist.
    >> But this seems awkward.
    >>

    > It also violates your requirement -- since the "crash" could take
    > place with a partial "temp file".
    >
    > I'd suggest that, rather than deleting the old status file, you
    > rename IT -- and only delete it IF you successfully rename the temp
    > file.

    Yes, this is much better. Almost perfect. Don't forget to consult your
    system documentation, and check if the rename operation is atomic or
    not. (Most probably it will only be atomic if the original and the
    renamed file are on the same physical partition and/or mount point).

    But even if the rename operation is atomic, there is still a race
    condition. Your program can be terminated after the original status file
    has been deleted, and before the temp file was renamed. In this case,
    you will be missing the status file (although your program already did
    something just it could not write out the new status).

    Here is an algorithm that can always write and read a status (but it
    might not be the latest one). You can keep the last two status files.

    Writer:
    * create temp file, write new status info
    * create lock file if needed
    * flock it
    * try:
    * delete older status file
    * rename temp file to new status file
    * finally: unlock the lock file

    Reader:

    * flock the lock file
    * try:
    * select the newer status file
    * read status info
    * finally: unlock the lock file

    It is guaranteed that you will always have a status to read, and in most
    cases this will be the last one (because the writer only locks for a
    short time). However, it is still questionable, because your writer may
    be waiting for the reader to unlock, so the new status info may not be
    written immediatelly.

    It would really help if you could tell us what are you trying to do that
    needs status.

    Best,

    Laszlo
     
    Laszlo Nagy, Jul 8, 2012
    #1
    1. Advertising

  2. Laszlo Nagy

    Nobody Guest

    On Sun, 08 Jul 2012 22:57:56 +0200, Laszlo Nagy wrote:

    > Yes, this is much better. Almost perfect. Don't forget to consult your
    > system documentation, and check if the rename operation is atomic or not.
    > (Most probably it will only be atomic if the original and the renamed file
    > are on the same physical partition and/or mount point).


    On Unix, rename() is always atomic, and requires that source and
    destination are on the same partition (if you want to "move" a file across
    partitions, you have to copy it then delete the original).

    > But even if the rename operation is atomic, there is still a race
    > condition. Your program can be terminated after the original status file
    > has been deleted, and before the temp file was renamed. In this case, you
    > will be missing the status file (although your program already did
    > something just it could not write out the new status).


    In the event of abnormal termination, losing some data is to be expected.
    The idea is to only lose the most recent data while keeping the old copy,
    rather than losing everything. Writing to a temp file then rename()ing
    achieves that.
     
    Nobody, Jul 9, 2012
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Richard Baron Penman

    How to safely maintain a status file

    Richard Baron Penman, Jul 8, 2012, in forum: Python
    Replies:
    0
    Views:
    205
    Richard Baron Penman
    Jul 8, 2012
  2. Dennis Lee Bieber

    Re: How to safely maintain a status file

    Dennis Lee Bieber, Jul 8, 2012, in forum: Python
    Replies:
    1
    Views:
    190
    Plumo
    Jul 9, 2012
  3. Plumo
    Replies:
    1
    Views:
    218
    Michael Hrivnak
    Jul 9, 2012
  4. Plumo
    Replies:
    1
    Views:
    216
    Laszlo Nagy
    Jul 12, 2012
  5. John Nagle
    Replies:
    2
    Views:
    323
    Laszlo Nagy
    Jul 12, 2012
Loading...

Share This Page