Linking data model to a File

[Ron]

I want to create a Swing application that allows a user to view a text
file. The application has to allow for a very large file size ( >
100M ), and the user should be able to see changes to the text file in
realtime...changes that are being introduced to the text file by
another process. You can think of this application as a graphical
verison of the unix command "tail -f", and it is also safe to assume
that the external changes to the text file are always additions to the
end of the file (e.g. a growing log file).

The elements I am having trouble with:
a) Linking the data model to the file such "paging" is performed (i.e.
For large files, the bulk of the data is only on disk. Data in memory
is the currently visible data plus whatever is optimal for good GUI
responsiveness when scrolling up or down.)
b) Detecting changes to the file such that the GUI is responsive to
those changes whether the file is growing slowly or very, very
quickly.
c) Handling very large files. I'm hoping the answer to a) will also
solve this, but maybe there is more to consider.

I've tried hard to find answers by searching through the
comp.lang.java.* groups, the gui faq, and countless online resources,
but I still haven't found an example of an app like this, or a
Document subclass that has a File for the underlying data (let alone
paging functionality) or even a basic outline of how to link data
models to files.

I've found enough bits and pieces that I can start trying to put it
all together myself from scratch. But before I start, I'm hoping for
advice from readers of this group. Do you know of any example code
that would be helpful? Any advanced tutorials for meeting the
requirements I've stated above? Or, in your opinion, will I need to
do this from scratch?

Although I didn't find any answers in the news groups, I did find
posts (sometimes many, many years old) asking a very similar
question. Since there are no responses to those posts, I'm worried
what I want to do might be quite difficult. So I would also be
interested in hearing from people who have had similar requirements
and were unable to find a solution that didn't require weeks of
development.

Thanks!

 

[Jeff Higgins]

I want to create a Swing application that allows a user to view a text
file. The application has to allow for a very large file size ( >
100M ), and the user should be able to see changes to the text file in
realtime...changes that are being introduced to the text file by
another process. You can think of this application as a graphical
verison of the unix command "tail -f", and it is also safe to assume
that the external changes to the text file are always additions to the
end of the file (e.g. a growing log file).

The elements I am having trouble with:
a) Linking the data model to the file such "paging" is performed (i.e.
For large files, the bulk of the data is only on disk. Data in memory
is the currently visible data plus whatever is optimal for good GUI
responsiveness when scrolling up or down.)
b) Detecting changes to the file such that the GUI is responsive to
those changes whether the file is growing slowly or very, very
quickly.
c) Handling very large files. I'm hoping the answer to a) will also
solve this, but maybe there is more to consider.

I've tried hard to find answers by searching through the
comp.lang.java.* groups, the gui faq, and countless online resources,
but I still haven't found an example of an app like this, or a
Document subclass that has a File for the underlying data (let alone
paging functionality) or even a basic outline of how to link data
models to files.

Some things that helped me a long time ago.
Unfortunately not Java oriented.
<http://www.cs.unm.edu/~crowley/papers/sds/sds.html>

 

[Roedy Green]

a) Linking the data model to the file such "paging" is performed (i.e.
For large files, the bulk of the data is only on disk. Data in memory
is the currently visible data plus whatever is optimal for good GUI
responsiveness when scrolling up or down.)

You might create an in-ram index to offsets in the file where the text
begins. You can then decide on where the chunk you want begins and
ends on disk and read in it in with a single RandomAccess I/O. You
read bytes then convert to a String.

Your index could be a simple array of ints
..
If you process in pages, your index could point the first byte of each
page, shrinking the index considerably.

 

[Roedy Green]

You might create an in-ram index to offsets in the file where the text
begins. You can then decide on where the chunk you want begins and
ends on disk and read in it in with a single RandomAccess I/O. You
read bytes then convert to a String.

The index would point to the start of each line. You can get the
length by looking at where the N+1th link starts.