Design Question: Managing multiple writes to flat file

Discussion in 'Java' started by Rhino, Mar 24, 2005.

  1. Rhino

    Rhino Guest

    I have a variety of applications that need to be able to write diagnostics
    to the same flat file. While there won't be that many situations where any
    of these applications are running concurrently, it could happen. What is the
    best way to write my applications to handle the case where Program A wants
    to write to the flat file while Program B is already writing to it?

    Am I safe in assuming that there is no safe way to have multiple Java
    programs simultaneously writing to the same flat file? If I am, then am I
    correct in assuming that each program that wants to write to the flat file
    needs to first verify that the file is not already in use? If so, what is
    the correct way to determine if some other program is writing to a file? I
    don't see anything in the File class that indicates if a file is in use,
    unless perhaps canRead() does that; the API is very vague on that so I'm not
    sure if canRead() is actually reporting on whether a file is already in use.

    Now, assuming that I'm right about having to verify that no other program is
    using the file, what should my program do if it finds that the file is in
    use when it wants to write to it? Should the program that wants the file be
    forced to exit? Or should it go on about its work and simply not try to
    write to the file after all?

    How do loggers handle this situation? What happens if two concurrent
    applications want to write to the same log file at the same time?

    --
    Rhino
    ---
    rhino1 AT sympatico DOT ca
    "There are two ways of constructing a software design. One way is to make it
    so simple that there are obviously no deficiencies. And the other way is to
    make it so complicated that there are no obvious deficiencies." - C.A.R.
    Hoare
    Rhino, Mar 24, 2005
    #1
    1. Advertising

  2. Rhino wrote:
    > I have a variety of applications that need to be able to write diagnostics
    > to the same flat file. While there won't be that many situations where any
    > of these applications are running concurrently, it could happen. What is the
    > best way to write my applications to handle the case where Program A wants
    > to write to the flat file while Program B is already writing to it?
    >
    > Am I safe in assuming that there is no safe way to have multiple Java
    > programs simultaneously writing to the same flat file?


    Hello,

    Suppose in this case you have a multithreaded logging application, which
    has a single synchronized log(String s); alike function, why shouldn't
    this work ? The multithreading part enables it to handle multiple
    applications that want to log but the synchronized logging method
    ensures that only one file is logging at the same time.

    So both program A and B use a logger program C. You could even do remote
    logging by letting program C listen on the network or use other
    mechanisms to connect to program C including divine intervention ;-)

    hth
    Elie De Brauwer, Mar 24, 2005
    #2
    1. Advertising

  3. "Rhino" <> wrote in message
    news:9AD0e.9843$...
    > I have a variety of applications that need to be able to write diagnostics
    > to the same flat file. While there won't be that many situations where any
    > of these applications are running concurrently, it could happen. What is

    the
    > best way to write my applications to handle the case where Program A wants
    > to write to the flat file while Program B is already writing to it?


    Acquire a lock for the file, write to it, release the lock. See the bottom
    of the message for the details on how to do this using a ServerSocket as the
    lock.

    >
    > Am I safe in assuming that there is no safe way to have multiple Java
    > programs simultaneously writing to the same flat file?


    Whether or not simultaneous writing is possible (e.g. implicit locks,
    atomicity, flushing, etc) depends on the OS rather than Java.

    > If I am, then am I
    > correct in assuming that each program that wants to write to the flat file
    > needs to first verify that the file is not already in use? If so, what is
    > the correct way to determine if some other program is writing to a file? I
    > don't see anything in the File class that indicates if a file is in use,
    > unless perhaps canRead() does that; the API is very vague on that so I'm

    not
    > sure if canRead() is actually reporting on whether a file is already in

    use.

    You have to arrange the lock yourself--see the details below

    > Now, assuming that I'm right about having to verify that no other program

    is
    > using the file, what should my program do if it finds that the file is in
    > use when it wants to write to it?


    Clearly, it would have to wait. You want writing to the file to be quick so
    that you acquire the lock, open the file, write to the file, close the file
    and release the lock. As an added bonus, you want a lock that is
    automatically discarded if your program fails.

    > Should the program that wants the file be
    > forced to exit? Or should it go on about its work and simply not try to
    > write to the file after all?


    It should wait and try again.

    >
    > How do loggers handle this situation? What happens if two concurrent
    > applications want to write to the same log file at the same time?


    Some heavy-duty logging systems use a logging server. Applications write to
    the server (where concurrency is ok) and the server writes to the file. For
    most ordinary systems simple locking is ok.

    You can arrange for two or more independent applications (Java or not) to
    write to the same flat file in a controlled way. First, pick a TCP port
    number to be your lock. Any rarely-used number (e.g. 7112) will be fine
    provided that all your applications know about. When any of the
    applications needs to write to the file it first tries to allocate a server
    socket for that port (e.g. new ServerSocket (LOCK_PORT...). If the socket
    gets allocated, the program can open the file, write to it and close the
    file. It should also then close the socket. If it cannot allocate the
    socket it should wait a short time and try again.

    TCP ports are global to the machine and issued uniquely and atomically--only
    one program can have it at any time. Having the port means you have the
    lock. If any program can't allocate the port it means it's already in use.
    The program should wait a bit and try again. If the writes are reasonably
    quick it will get in.

    Note that you don't actually have to do anything with the socket. Just
    allocate it and release it.

    Cheers,
    Matt Humphrey http://www.iviz.com/
    Matt Humphrey, Mar 24, 2005
    #3
  4. Rhino

    Rhino Guest

    "Elie De Brauwer" <> wrote in message
    news:LnE0e.45054$-ops.be...
    > Rhino wrote:
    > > I have a variety of applications that need to be able to write

    diagnostics
    > > to the same flat file. While there won't be that many situations where

    any
    > > of these applications are running concurrently, it could happen. What is

    the
    > > best way to write my applications to handle the case where Program A

    wants
    > > to write to the flat file while Program B is already writing to it?
    > >
    > > Am I safe in assuming that there is no safe way to have multiple Java
    > > programs simultaneously writing to the same flat file?

    >
    > Hello,
    >
    > Suppose in this case you have a multithreaded logging application, which
    > has a single synchronized log(String s); alike function, why shouldn't
    > this work ? The multithreading part enables it to handle multiple
    > applications that want to log but the synchronized logging method
    > ensures that only one file is logging at the same time.
    >
    > So both program A and B use a logger program C. You could even do remote
    > logging by letting program C listen on the network or use other
    > mechanisms to connect to program C including divine intervention ;-)
    >

    I'm going to have to think about this a bit.

    Thanks for your reply!

    Rhino
    Rhino, Mar 24, 2005
    #4
  5. Rhino

    Rhino Guest

    "Matt Humphrey" <> wrote in message
    news:...
    >
    > "Rhino" <> wrote in message
    > news:9AD0e.9843$...
    > > I have a variety of applications that need to be able to write

    diagnostics
    > > to the same flat file. While there won't be that many situations where

    any
    > > of these applications are running concurrently, it could happen. What is

    > the
    > > best way to write my applications to handle the case where Program A

    wants
    > > to write to the flat file while Program B is already writing to it?

    >
    > Acquire a lock for the file, write to it, release the lock. See the

    bottom
    > of the message for the details on how to do this using a ServerSocket as

    the
    > lock.
    >
    > >
    > > Am I safe in assuming that there is no safe way to have multiple Java
    > > programs simultaneously writing to the same flat file?

    >
    > Whether or not simultaneous writing is possible (e.g. implicit locks,
    > atomicity, flushing, etc) depends on the OS rather than Java.
    >
    > > If I am, then am I
    > > correct in assuming that each program that wants to write to the flat

    file
    > > needs to first verify that the file is not already in use? If so, what

    is
    > > the correct way to determine if some other program is writing to a file?

    I
    > > don't see anything in the File class that indicates if a file is in use,
    > > unless perhaps canRead() does that; the API is very vague on that so I'm

    > not
    > > sure if canRead() is actually reporting on whether a file is already in

    > use.
    >
    > You have to arrange the lock yourself--see the details below
    >
    > > Now, assuming that I'm right about having to verify that no other

    program
    > is
    > > using the file, what should my program do if it finds that the file is

    in
    > > use when it wants to write to it?

    >
    > Clearly, it would have to wait. You want writing to the file to be quick

    so
    > that you acquire the lock, open the file, write to the file, close the

    file
    > and release the lock. As an added bonus, you want a lock that is
    > automatically discarded if your program fails.
    >
    > > Should the program that wants the file be
    > > forced to exit? Or should it go on about its work and simply not try to
    > > write to the file after all?

    >
    > It should wait and try again.
    >
    > >
    > > How do loggers handle this situation? What happens if two concurrent
    > > applications want to write to the same log file at the same time?

    >
    > Some heavy-duty logging systems use a logging server. Applications write

    to
    > the server (where concurrency is ok) and the server writes to the file.

    For
    > most ordinary systems simple locking is ok.
    >
    > You can arrange for two or more independent applications (Java or not) to
    > write to the same flat file in a controlled way. First, pick a TCP port
    > number to be your lock. Any rarely-used number (e.g. 7112) will be fine
    > provided that all your applications know about. When any of the
    > applications needs to write to the file it first tries to allocate a

    server
    > socket for that port (e.g. new ServerSocket (LOCK_PORT...). If the socket
    > gets allocated, the program can open the file, write to it and close the
    > file. It should also then close the socket. If it cannot allocate the
    > socket it should wait a short time and try again.
    >
    > TCP ports are global to the machine and issued uniquely and

    atomically--only
    > one program can have it at any time. Having the port means you have the
    > lock. If any program can't allocate the port it means it's already in

    use.
    > The program should wait a bit and try again. If the writes are reasonably
    > quick it will get in.
    >
    > Note that you don't actually have to do anything with the socket. Just
    > allocate it and release it.
    >

    Thanks very much for your suggestions! They are clear and sound like they
    should work well for my purposes.

    Rhino
    Rhino, Mar 24, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mohammad S Khan
    Replies:
    3
    Views:
    8,653
    Chris Uppal
    Aug 31, 2004
  2. Daniel Draper

    Multiple writes to client socket

    Daniel Draper, Apr 2, 2005, in forum: C Programming
    Replies:
    3
    Views:
    437
    Peter Nilsson
    Apr 4, 2005
  3. davidcalvin

    Multiple reads and writes with popen?

    davidcalvin, Jul 9, 2008, in forum: C Programming
    Replies:
    0
    Views:
    1,084
    davidcalvin
    Jul 9, 2008
  4. vani
    Replies:
    2
    Views:
    483
  5. Replies:
    3
    Views:
    146
    J. Gleixner
    Oct 22, 2007
Loading...

Share This Page