RandomAccessFile and java.nio.*

[Andreas Leitgeb]

As far as I understood the tutorials and documentation, then the java.io.*
classes have been redesigned (already back in the Java 1.4 days) to work
on the nio-API as its lower level.

From this, it's not surprising, that RandomAccessFile has a method to
get access to the nio FileChannel and is mostly some convenience class
that (among other things) allows reading single lines, and then have the
current fileposition right after the newline of that read line.

Unfortunately, however, the RandomAccessFile still has only constructors
taking the open-mode as a String (with only "r","rw","rws","rwd" being the
legal values).

If instead I open a FileChannel using the nio API, then I can specify POSIX
flags, like TRUNCATE or APPEND.

What I'm missing is a way to wrap a RandomAccessFile over an existing
FileChannel, such that I can use the more flexible open modes, but
still get a convenient readLine().

Btw., I'd really like a readLine() for reading lines also from Channels
that are not FileChannel, but perhaps Pipe.SourceChannel or sockets.

And just one more thing: I've seen that I can wrap streams like System.out
in a Channel, using Channels.newChannel(System.out), but I don't know how
I could undo this wrapping, eventually getting rid of the wrapping Channel
without having System.out closed as a side effect. I'm a bit scared that
merely "forgetting" the Channel might eventually cause it to be close()d
from its finalizer.

 

[Jeff Higgins]

As far as I understood the tutorials and documentation, then the java.io.*
classes have been redesigned (already back in the Java 1.4 days) to work
on the nio-API as its lower level.

From this, it's not surprising, that RandomAccessFile has a method to
get access to the nio FileChannel and is mostly some convenience class
that (among other things) allows reading single lines, and then have the
current fileposition right after the newline of that read line.

Unfortunately, however, the RandomAccessFile still has only constructors
taking the open-mode as a String (with only "r","rw","rws","rwd" being the
legal values).

If instead I open a FileChannel using the nio API, then I can specify POSIX
flags, like TRUNCATE or APPEND.

What I'm missing is a way to wrap a RandomAccessFile over an existing
FileChannel, such that I can use the more flexible open modes, but
still get a convenient readLine().

Btw., I'd really like a readLine() for reading lines also from Channels
that are not FileChannel, but perhaps Pipe.SourceChannel or sockets.

And just one more thing: I've seen that I can wrap streams like System.out
in a Channel, using Channels.newChannel(System.out), but I don't know how
I could undo this wrapping, eventually getting rid of the wrapping Channel
without having System.out closed as a side effect. I'm a bit scared that
merely "forgetting" the Channel might eventually cause it to be close()d
from its finalizer.

?
LineNumberReader takes a Reader
Channels.newReader takes a ReadableByteChannel (FileChannel)

 

[markspace]

LineNumberReader takes a Reader
Channels.newReader takes a ReadableByteChannel (FileChannel)

Once you have a Reader, using a BufferReader would probably be more
clear to a maintainer, assuming line number counting isn't required.

But I agree with Andreas that RandomAccessFile seems to have been left
an orphan when the NIO stuff came out. I can't figure out a way to get
there.

 

[Jeff Higgins]

Once you have a Reader, using a BufferReader would probably be more
clear to a maintainer, assuming line number counting isn't required.

But I agree with Andreas that RandomAccessFile seems to have been left
an orphan when the NIO stuff came out. I can't figure out a way to get
there.

Huh.
java.nio.file.Files.newRandomAccess(...)
<http://openjdk.java.net/jeps/1>

 

[markspace]

Huh.
java.nio.file.Files.newRandomAccess(...)
<http://openjdk.java.net/jeps/1>

I don't see that method there. Are you proposing it be suggested for
the JEPS 1 project?

 

[Jeff Higgins]

I don't see that method there. Are you proposing it be suggested for
the JEPS 1 project?

No, I was suggesting the proposal process described in JEP 1.

From the "Mechanics" section:

"To submit a JEP for posting send a completed template in e-mail, as a
text/plain attachment, to jep dash submit at openjdk dot java dot net.
An actual human will sanity-check your JEP, allocate a number for it,
and post it to the archive."

Alternatively:
for discussion
<http://mail.openjdk.java.net/mailman/listinfo/nio-discuss>
for contributions
<http://mail.openjdk.java.net/mailman/listinfo/nio-dev>

 

[Andreas Leitgeb]

java.util.scanner has been updated with NIO2. It has the nextLine method
you want.

Thanks! I really missed that.

(So far, my tests fail - all of Scanner's .hasNext* methods return
false, despite the testfile containing appropriate data), but I guess
that's probably a bug in my test-environment, likely some code reading
all the data from the channel before I even create the Scanner instance...
- I'll find out)

Even if I get this to work, I fear, it still lacks a way to tell me
the current position within the file, after having read some item.
So, I'll probably need to roll my own Scanner with file position
housekeeping, anyway :-(

 

[Andreas Leitgeb]

Thanks Jeff & Mark for this subthread.

It's not very likely that I'll create a JEP(roposal) for this ;-)

BufferedReader has the same problem as Scanner (as suggested by Wayne):
it doesn't give me a clue as to the exact file position corresponding
to the point where the line (and nothing beyond) has been read.
That's not really surprising as such, given the stack of Reader/Stream-
wrappers involved, but RandomAccessFile *was* able to give me that
information.

What I really need is saving a particular position (like after having
read first line), in order to be able to jump back to that position at
a later time and (physically) re-read the second line (data might have
changed by the time I re-read it, but the position will be same. Yeah,
odd).

(This is just for files, of course. There is no seek'ing back on pipes
and sockets, of course, but I'd still like to use the same readLine()/
nextLine()-API for all channels regardless if I'll later need to seek(),
or not.)

It's still good to know, that I'm not re-inventing the wheel, when
I roll my own for those needs.

 

[Jeff Higgins]

Thanks Jeff & Mark for this subthread.

It's not very likely that I'll create a JEP(roposal) for this ;-)

Durn. I was hoping to see the results of a sanity check.

BufferedReader has the same problem as Scanner (as suggested by Wayne):
it doesn't give me a clue as to the exact file position corresponding
to the point where the line (and nothing beyond) has been read.
That's not really surprising as such, given the stack of Reader/Stream-
wrappers involved, but RandomAccessFile *was* able to give me that
information.

What I really need is saving a particular position (like after having
read first line), in order to be able to jump back to that position at
a later time and (physically) re-read the second line (data might have
changed by the time I re-read it, but the position will be same. Yeah,
odd).

(This is just for files, of course. There is no seek'ing back on pipes
and sockets, of course, but I'd still like to use the same readLine()/
nextLine()-API for all channels regardless if I'll later need to seek(),
or not.)

It's not real clear to me what you want to accomplish.
It you read a line such as:
First Name ___________________
do you hope to replace it with:
First Name Andreas

It's still good to know, that I'm not re-inventing the wheel, when
I roll my own for those needs.

I'm hoping for a real world excellent example of a
Spliterator implementation, please keep us updated.

 

[Jeff Higgins]

Durn. I was hoping to see the results of a sanity check.

Of my own non-proposal of a
java.nio.file.Files.newRandomAccess(...) method, that is.

 

[markspace]

It's still good to know, that I'm not re-inventing the wheel, when
I roll my own for those needs.

I took a quick look at RandomAccessFile, and it seems to use a lot of
native methods. I would guess that porting it to the NIO FileChannels
was more work than Sun/Oracle wanted to get into.

That said, a drop-in replacement would be nice. Factor out an interface
RandomAccessIO, make RandomAccessFile extend that interface, and then
add a new class RandomAccessChannel that extends RandomAccessIO and
wraps a FileChannel. That I think would be my suggestion, if I were
going to make one.

As it is, you're probably OK by re-inventing the wheel here, I don't see
any other way. RandomAccessFile is not final, you could extend it if
you absolutely needed a drop-in replacement, but that seems like a very
difficult solution given the way it's implemented. Rolling your own
from scratch is probably better.

 

[markspace]

It's still good to know, that I'm not re-inventing the wheel, when
I roll my own for those needs.

You might want to be suspicious of RandomAccessFile anyway. It's a 1.0
class and seems to have some weird corners. For example, readLine() is
basically this:

StringBuffer input = new StringBuffer();
// loop until done:
int c;
c = read();
input.append((char)c);

That isn't going to work for anything that's not plain old ASCII. I can
see why BufferedReader, which uses a CharsetDecoder, is preferred.

 

[Andreas Leitgeb]

You might want to be suspicious of RandomAccessFile anyway. It's a 1.0
class and seems to have some weird corners. For example, readLine() is
basically this:
StringBuffer input = new StringBuffer();
// loop until done:
int c;
c = read();
input.append((char)c);
That isn't going to work for anything that's not plain old ASCII. I can
see why BufferedReader, which uses a CharsetDecoder, is preferred.

I haven't yet had a closer look at RandomAccessFile, but I'd likely have
written it just like that for the first draft, that is: repeatedly read
into a ByteBuffer of size 1, until I stumble over a '\n'.

I'm in the lucky situation of not needing to support more than iso8859-1
and utf-8, and a single '\r' will never be a line-ending so I can rely
on single-byte '\n' (eventually ignoring an immediately preceding '\r').

Later, I'll probably wrap the nio-Channel and some Buffer into some
custom class, and make some provisions that position() will correct the
OS' position() by the number of unused bytes in the Buffer, and in case
a write happens, when read-buffer is not empty, and if the Channel is
tied to a plain file, then just call setPosition(...) first.

Now, that I posted it, all left to do for me is actually coding it ;-)

 

[Andreas Leitgeb]

RandomAccessFile is not final, you could extend it if
you absolutely needed a drop-in replacement, ...

The main blocking point with that approach would be RAF's cons-
tructors. I can't subclass it without going through one of those.

Fortunately, I do *not* "absolutely need a drop-in replacement."
The only reason I even looked at it, was the readLine() method
with its convenient position()-consistency.

 

[Andreas Leitgeb]

Durn. I was hoping to see the results of a sanity check.
Of my own non-proposal of a
java.nio.file.Files.newRandomAccess(...) method, that is.


It's not real clear to me what you want to accomplish.
It you read a line such as:
First Name ___________________
do you hope to replace it with:
First Name Andreas

Essentially yes. Except, that the _____ is in the next line
after "First Name", and it's not necessarily *me* who replaces
the data. And "First Name" is not in a compile-time fixed position
within the file, but it will be fixed for the relevant timespan
at runtime.

Now you know, why I don't think that my needs justify a JEP.

I'm hoping for a real world excellent example of a
Spliterator implementation, please keep us updated.

Spliterator lol
I hope it won't turn into splatter...

My road ahead is sketched in a followup to one of markspace's posts.