Re: 2 daemons write to a single file /w python file IO

Discussion in 'Python' started by Steven W. Orr, Sep 14, 2007.

  1. On Tuesday, Sep 11th 2007 at 21:17 -0700, quoth Andrey:

    =>i have a newbie question about the file() function.
    =>I have 2 daemons running on my linux box.
    =>1 will record the IDs to a file - logs.txt
    =>other 1 will open this file, read the IDs, and then "Clean up the
    =>file" -logs.txt
    =>Since these 2 daemons will run every 2-5mins, I think this will crash, isn't
    =>it? When both daemons try to write to the file at the same time.
    =>I am wondering if this won't crash, OR if there is some simple high-level
    =>functions can lock the file while writing...
    =>I also wonder if one side locked the file, what happens if the other side
    =>try to open this locked file? raise error? so i also need to write a loop to
    =>wait for the file to release locking?
    =>seems a basic easy thing but i just cannot find an simple answer.
    =>I always advoid this issue by putting them in mysql (well, fast and hassle
    =>free for locking)

    This is more of a unix question as opposed to a python question, but I'll
    'splain it anyways :)

    If two processes each have a channel opened to the same file then it is
    very possible for the writes to collide. One way to mitigate the damage
    would be to make sure that the two processes open the file for append
    access instead of simple write access. This will ensure that the if
    ProcessA writes data that ProcessB's next write won't overwrite the data
    that A just wrote. But! It still won't work! You will still get
    interleaving of data in your output file. They will all get out there, but
    the data will on occasion look garbled because the interleaving won't look
    like it was partitioned as the different processes intended.

    So what's the solution? It turns out that data that is written to a pipe
    (as opposed to data that is written to a file) is guaranteed to maintain
    its partitioned integrity.

    P1 >> fn
    P2 >> fn

    If you are the guy who starts P1 and P2 from a parent process(PP) then you
    could structure your pipes so you end up with

    PP > fn
    / \
    / \
    P1 P2

    and just make PP be a multiplexor. :)

    Time flies like the wind. Fruit flies like a banana. Stranger things have .0.
    happened but none stranger than this. Does your driver's license say Organ ..0
    Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
    individuals! What if this weren't a hypothetical question?
    steveo at
    Steven W. Orr, Sep 14, 2007
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    Sep 13, 2006
  2. Andrey
    Sep 12, 2007
  3. PurpleServerMonkey

    Python, Daemons and D-Bus

    PurpleServerMonkey, May 24, 2008, in forum: Python
    Diez B. Roggisch
    May 25, 2008
  4. Jeffrey Barish

    Single-instance daemons

    Jeffrey Barish, Nov 12, 2008, in forum: Python
    Дамјан ГеоргиевÑки
    Nov 17, 2008
  5. Junkone
    Kevin Williams
    Feb 16, 2008