I/O file operations efficiency

[hristo]

I have some questions regarding the I/O file operations efficiency.

Consider I/O operations involving the disk. Assume that I write
periodically one Byte in the file after executing one processing block
of code. Does this mean that at each of these periods, that only Byte
will be stored in the disk despite its slow access time, or instead it
will be stored in the memory buffer (of what size?) first then moved
into the disk once the buffer is full?

Should the programmer force the second option (by using
BufferOutputStream) or is it done automatically by the JVM or OS?

I have read also that writing and reading should be done on chunks of
256 (512,1024) bytes since the disk sector is of this size. Should I
specify explicitly the buffer size or will it be handled automatically
by the JVM or OS

Say I have a stream of X bytes, should write it into the disk per
once, or instead repetitively on X/256 bytes size.

Thanks for your help

 

[Eric Sosman]

I have some questions regarding the I/O file operations efficiency.

Consider I/O operations involving the disk. Assume that I write
periodically one Byte in the file after executing one processing block
of code. Does this mean that at each of these periods, that only Byte
will be stored in the disk despite its slow access time, or instead it
will be stored in the memory buffer (of what size?) first then moved
into the disk once the buffer is full?

Yes. No. Both. Maybe.

Most O/Ses will hold disk output in a buffer for a while,
either until the buffer fills or an explicit buffer-flush is
requested (possibly by closing the file) or until some time
period elapses. Most O/Ses also provide ways to modify this
behavior, changing a disk's or file's buffering parameters or
defeating buffering altogether. The details (including just
what adjustments are possible, and under what circumstances)
are O/S-specific, and not directly controllable from Java.

Should the programmer force the second option (by using
BufferOutputStream) or is it done automatically by the JVM or OS?

BufferedOutputStream provides a buffer that operates at
a different level: it's in the JVM's memory, not in the O/S.
The output data sits in the JVM's buffer until it's time for
it to be flushed, then it's written to the O/S. At that point
the O/S may decide to write it directly or may choose to
buffer it again. Presumably, the folks who wrote the JVM for
your system put some thought into making BufferedOutputStream
"play nicely" with the O/S' own buffering schemes, but the
two are logically independent.

I have read also that writing and reading should be done on chunks of
256 (512,1024) bytes since the disk sector is of this size. Should I
specify explicitly the buffer size or will it be handled automatically
by the JVM or OS

No; you should use 8192 bytes because the O/S performs its
I/O in sixteen-sector "pages." Or maybe you should use 4096
bytes because the O/S writes eight "pages" at a time, or 65536
because it writes 128 pages. Or maybe you should write ~1450
bytes at a time because the disk is mounted remotely and the
data must cross an Ethernet cable with a 1500-byte MTU, or ...

You should probably not worry about any of this unless
you have measured the I/O performance and found it too slow
or unless you have an a priori reason to believe that you'll
need exceptionally high speeds (e.g., for streaming high-
speed and high-volume telemetry).

Say I have a stream of X bytes, should write it into the disk per
once, or instead repetitively on X/256 bytes size.

Write it in whatever chunks are most natural for the
program that's generating the data: a character at a time,
a line at a time, or whatever. Trust the JVM and the O/S to
do something reasonable. What you get will almost certainly
be slower than the very utmost the hardware could possibly be
driven to deliver, but is very likely to be "good enough."

Then, *if* the data rate turns out to be inadequate,
start measuring and tweaking -- but only then, because the
more of this you do, the more you tie your program to the
peculiarities of a particular platform.