File Read in 2 JVM

[moleskyca1]

I am going to read file in one JVM that someone create in another JVM.
Writer JVM is older class, but I can ask them to modify it for me.

This is in linux.

I am worried about dirty reads. Reading the file when writer JVM is not
done writing. What is good way to handle this? The file is huge (like
40-60 gigs) and will not fit in memory.

 

[kingpin+nntp]

I am going to read file in one JVM that someone create in another JVM.
Writer JVM is older class, but I can ask them to modify it for me.

This is in linux.

I am worried about dirty reads. Reading the file when writer JVM is not
done writing. What is good way to handle this? The file is huge (like
40-60 gigs) and will not fit in memory.

Usually only one process can open a file for writing at one time, but
this depends on which API was used and the implementation of the
underlying OS, so it's not a very reliable approach.

Three possible solutions:

0. Have the reader lock the file so that no other process can write to
it. If you can't get the lock, then it means another process is
writing to it.

1. Have the writer create a secondary status file, which is deleted
once the process has been completed. The java.io.File.deleteOnExit()
method can make this easier.

2. Have the writer create the file with a different name/extension, and
then rename it after closing it.

 

[Chris Uppal]

Three possible solutions:

0. Have the reader lock the file so that no other process can write to
it. If you can't get the lock, then it means another process is
writing to it.

1. Have the writer create a secondary status file, which is deleted
once the process has been completed. The java.io.File.deleteOnExit()
method can make this easier.

2. Have the writer create the file with a different name/extension, and
then rename it after closing it.

....also...

3. Have the writer write a unique end-marker to the output file.

-- chris

 

[kingpin+nntp]

...also...

3. Have the writer write a unique end-marker to the output file.

That's a very good suggestion, as long as there's absolutely no
potential for data being written to the file to be identical to the
end-marker character sequence.