Design Question: Managing multiple writes to flat file

[Rhino]

I have a variety of applications that need to be able to write diagnostics
to the same flat file. While there won't be that many situations where any
of these applications are running concurrently, it could happen. What is the
best way to write my applications to handle the case where Program A wants
to write to the flat file while Program B is already writing to it?

Am I safe in assuming that there is no safe way to have multiple Java
programs simultaneously writing to the same flat file? If I am, then am I
correct in assuming that each program that wants to write to the flat file
needs to first verify that the file is not already in use? If so, what is
the correct way to determine if some other program is writing to a file? I
don't see anything in the File class that indicates if a file is in use,
unless perhaps canRead() does that; the API is very vague on that so I'm not
sure if canRead() is actually reporting on whether a file is already in use.

Now, assuming that I'm right about having to verify that no other program is
using the file, what should my program do if it finds that the file is in
use when it wants to write to it? Should the program that wants the file be
forced to exit? Or should it go on about its work and simply not try to
write to the file after all?

How do loggers handle this situation? What happens if two concurrent
applications want to write to the same log file at the same time?

--
Rhino
---
rhino1 AT sympatico DOT ca
"There are two ways of constructing a software design. One way is to make it
so simple that there are obviously no deficiencies. And the other way is to
make it so complicated that there are no obvious deficiencies." - C.A.R.
Hoare

 

[Elie De Brauwer]

I have a variety of applications that need to be able to write diagnostics
to the same flat file. While there won't be that many situations where any
of these applications are running concurrently, it could happen. What is the
best way to write my applications to handle the case where Program A wants
to write to the flat file while Program B is already writing to it?

Am I safe in assuming that there is no safe way to have multiple Java
programs simultaneously writing to the same flat file?

Hello,

Suppose in this case you have a multithreaded logging application, which
has a single synchronized log(String s); alike function, why shouldn't
this work ? The multithreading part enables it to handle multiple
applications that want to log but the synchronized logging method
ensures that only one file is logging at the same time.

So both program A and B use a logger program C. You could even do remote
logging by letting program C listen on the network or use other
mechanisms to connect to program C including divine intervention ;-)

hth

 

[Matt Humphrey]

I have a variety of applications that need to be able to write diagnostics
to the same flat file. While there won't be that many situations where any
of these applications are running concurrently, it could happen. What is the
best way to write my applications to handle the case where Program A wants
to write to the flat file while Program B is already writing to it?

Acquire a lock for the file, write to it, release the lock. See the bottom
of the message for the details on how to do this using a ServerSocket as the
lock.

Am I safe in assuming that there is no safe way to have multiple Java
programs simultaneously writing to the same flat file?

Whether or not simultaneous writing is possible (e.g. implicit locks,
atomicity, flushing, etc) depends on the OS rather than Java.

If I am, then am I
correct in assuming that each program that wants to write to the flat file
needs to first verify that the file is not already in use? If so, what is
the correct way to determine if some other program is writing to a file? I
don't see anything in the File class that indicates if a file is in use,
unless perhaps canRead() does that; the API is very vague on that so I'm not
sure if canRead() is actually reporting on whether a file is already in

use.

You have to arrange the lock yourself--see the details below

Now, assuming that I'm right about having to verify that no other program is
using the file, what should my program do if it finds that the file is in
use when it wants to write to it?

Clearly, it would have to wait. You want writing to the file to be quick so
that you acquire the lock, open the file, write to the file, close the file
and release the lock. As an added bonus, you want a lock that is
automatically discarded if your program fails.

Should the program that wants the file be
forced to exit? Or should it go on about its work and simply not try to
write to the file after all?

It should wait and try again.

How do loggers handle this situation? What happens if two concurrent
applications want to write to the same log file at the same time?

Some heavy-duty logging systems use a logging server. Applications write to
the server (where concurrency is ok) and the server writes to the file. For
most ordinary systems simple locking is ok.

You can arrange for two or more independent applications (Java or not) to
write to the same flat file in a controlled way. First, pick a TCP port
number to be your lock. Any rarely-used number (e.g. 7112) will be fine
provided that all your applications know about. When any of the
applications needs to write to the file it first tries to allocate a server
socket for that port (e.g. new ServerSocket (LOCK_PORT...). If the socket
gets allocated, the program can open the file, write to it and close the
file. It should also then close the socket. If it cannot allocate the
socket it should wait a short time and try again.

TCP ports are global to the machine and issued uniquely and atomically--only
one program can have it at any time. Having the port means you have the
lock. If any program can't allocate the port it means it's already in use.
The program should wait a bit and try again. If the writes are reasonably
quick it will get in.

Note that you don't actually have to do anything with the socket. Just
allocate it and release it.

Cheers,
Matt Humphrey (e-mail address removed) http://www.iviz.com/

 

[Rhino]

Hello,

Suppose in this case you have a multithreaded logging application, which
has a single synchronized log(String s); alike function, why shouldn't
this work ? The multithreading part enables it to handle multiple
applications that want to log but the synchronized logging method
ensures that only one file is logging at the same time.

So both program A and B use a logger program C. You could even do remote
logging by letting program C listen on the network or use other
mechanisms to connect to program C including divine intervention ;-)

I'm going to have to think about this a bit.

Thanks for your reply!

Rhino

 

[Rhino]

Acquire a lock for the file, write to it, release the lock. See the bottom
of the message for the details on how to do this using a ServerSocket as the

Whether or not simultaneous writing is possible (e.g. implicit locks,
atomicity, flushing, etc) depends on the OS rather than Java.

use.

You have to arrange the lock yourself--see the details below
program

Clearly, it would have to wait. You want writing to the file to be quick so
that you acquire the lock, open the file, write to the file, close the file
and release the lock. As an added bonus, you want a lock that is
automatically discarded if your program fails.


It should wait and try again.


Some heavy-duty logging systems use a logging server. Applications write to
the server (where concurrency is ok) and the server writes to the file. For
most ordinary systems simple locking is ok.

You can arrange for two or more independent applications (Java or not) to
write to the same flat file in a controlled way. First, pick a TCP port
number to be your lock. Any rarely-used number (e.g. 7112) will be fine
provided that all your applications know about. When any of the
applications needs to write to the file it first tries to allocate a server
socket for that port (e.g. new ServerSocket (LOCK_PORT...). If the socket
gets allocated, the program can open the file, write to it and close the
file. It should also then close the socket. If it cannot allocate the
socket it should wait a short time and try again.

TCP ports are global to the machine and issued uniquely and atomically--only
one program can have it at any time. Having the port means you have the
lock. If any program can't allocate the port it means it's already in use.
The program should wait a bit and try again. If the writes are reasonably
quick it will get in.

Note that you don't actually have to do anything with the socket. Just
allocate it and release it.

Thanks very much for your suggestions! They are clear and sound like they
should work well for my purposes.

Rhino