NIO enigma

[NRM]

I am seeing anomalous behavior on Windows XP Pro in my NIO code. The
code in question is as follows and it works as expected on Linux.

import java.io.ByteArrayInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;

public class FileAppender {
public static void main(String[] args) throws IOException {
FileChannel fs = new FileOutputStream("output",
true).getChannel();
ByteArrayInputStream bs = new ByteArrayInputStream(new byte[]
{50, 51});
ReadableByteChannel byteChannel = Channels.newChannel(bs);
fs.transferFrom(byteChannel, 0, 1024);
fs.close();
}
}

As you can see, I am using NIO to append to an existing file. In
Windows, the code always overwrites the first two bytes, whereas in
Linux two bytes are added by each run. Although the code here deals
with a byte array, the same problem happens with any input stream. The
problem is as follows:

On Linux this produces "2323" when run twice. On Windows it produces
"23". Note that I am using Sun's JDK 1.5.0_07 in both cases.

Does anyone have experience with OS specific bugs for NIO? Is there a
known issue around this behavior? I noticed an open bug with Sun on NIO
(http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4950302) where
random write produces correct results on Windows but not on Linux.

Thanks for your help!

 

[Mark Thornton]

I am seeing anomalous behavior on Windows XP Pro in my NIO code. The
code in question is as follows and it works as expected on Linux.

import java.io.ByteArrayInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;

public class FileAppender {
public static void main(String[] args) throws IOException {
FileChannel fs = new FileOutputStream("output",
true).getChannel();
ByteArrayInputStream bs = new ByteArrayInputStream(new byte[]
{50, 51});
ReadableByteChannel byteChannel = Channels.newChannel(bs);
fs.transferFrom(byteChannel, 0, 1024);
fs.close();
}
}

As you can see, I am using NIO to append to an existing file. In
Windows, the code always overwrites the first two bytes, whereas in
Linux two bytes are added by each run. Although the code here deals
with a byte array, the same problem happens with any input stream. The
problem is as follows:

On Linux this produces "2323" when run twice. On Windows it produces
"23". Note that I am using Sun's JDK 1.5.0_07 in both cases.

Does anyone have experience with OS specific bugs for NIO? Is there a
known issue around this behavior? I noticed an open bug with Sun on NIO
(http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4950302) where
random write produces correct results on Windows but not on Linux.

Thanks for your help!

This is the bit you need to examine more carefully:

fs.transferFrom(byteChannel, 0, 1024);

This writes to fs starting at position '0', which on Windows is the
beginning of the file. On Windows append mode is implemented by opening
the file as normal and then just setting the current position to the end
of the file (the OS does not have a native append mode like *nix).

Mark Thornton

 

[Mark Thornton]

You consider forgetting about append mode altogether. Quite a bit of its
useful behaviour on *nix does not work on Windows. For example, on *nix
it is common to have two or more processes all appending to the same
file. The operating system guarantees that the writes are atomic,
resulting in correctly interleaved records. This does not work on Windows.

This should not be taken to imply that Windows is a bad operating
system, it is merely very different. Java is not always able to hide
these differences.

Mark Thornton

 

[NRM]

This is the bit you need to examine more carefully:

fs.transferFrom(byteChannel, 0, 1024);

This writes to fs starting at position '0', which on Windows is the
beginning of the file. On Windows append mode is implemented by opening
the file as normal and then just setting the current position to the end
of the file (the OS does not have a native append mode like *nix).

Mark Thornton

Thanks Mark!

 

[Chris]

As you can see, I am using NIO to append to an existing file. In
Windows, the code always overwrites the first two bytes, whereas in
Linux two bytes are added by each run. Although the code here deals
with a byte array, the same problem happens with any input stream. The
problem is as follows:

On Linux this produces "2323" when run twice. On Windows it produces
"23". Note that I am using Sun's JDK 1.5.0_07 in both cases.

Does anyone have experience with OS specific bugs for NIO? Is there a
known issue around this behavior? I noticed an open bug with Sun on NIO
(http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4950302) where
random write produces correct results on Windows but not on Linux.

Here is the code that I use. Works on Windows, has not been tested on
Linux. Can you tell me if it works correctly for you?

/**
* Append the contents of extra file to main.
*/
public static void appendFile(RandomAccessFile main, RandomAccessFile
extra) throws IOException {

// this method uses NIO direct transfer. It delegates the task
// to the operating system. Only works under 1.4
extra.getChannel().transferTo(0, Long.MAX_VALUE, main.getChannel());
extra.close();
}