Re: Buffered RandomAccessFile

Discussion in 'Java' started by Arne Vajhøj, Mar 25, 2010.

  1. Arne Vajhøj

    Arne Vajhøj Guest

    On 25-03-2010 12:26, Spud wrote:
    > Can someone suggest a good approach for caching parts of a large data file?
    >
    > I've got a number of very large (multi-gigabyte) files. Each file is
    > organized into pages of variable length. Pages can be small (< 1k) or
    > much larger (> 1mb). The organization of the file is similar to a btree,
    > and the access pattern is similar: you start at the root, skip to pages
    > scattered in different places in the file, and then when you get to a
    > leaf node you read sequentially, possibly over many megabytes.
    >
    > I need some kind of caching system so I'm not hitting the disk on every
    > access. Ideally, I'd be able to use a kind of buffered RandomAccessFile
    > object which would magically cache hunks of the file appropriately.
    >
    > I know this problem is similar to the problem databases have: how to
    > cache pages optimally. There's a difference, though, because my pages
    > are of variable length and pages in a database are usually of fixed
    > length. So a fixed pool of uniformly-sized pages organized in an LRU
    > cache won't work.
    >
    > Anyone have any insights on what I should do?


    Structure it like:

    business logic
    buffer layer
    IO layer
    Java RT (RandomAccessFile)

    Pick one of the standard cache packages - they are capable
    of caching objects of different sizes and can use different
    eviction algorithms.

    Arne
     
    Arne Vajhøj, Mar 25, 2010
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mike
    Replies:
    0
    Views:
    1,208
  2. Harald Kirsch

    Reader on top of RandomAccessFile

    Harald Kirsch, Aug 21, 2003, in forum: Java
    Replies:
    0
    Views:
    577
    Harald Kirsch
    Aug 21, 2003
  3. Replies:
    9
    Views:
    706
    Michael Wojcik
    Aug 23, 2005
  4. Knute Johnson

    Re: Buffered RandomAccessFile

    Knute Johnson, Mar 25, 2010, in forum: Java
    Replies:
    6
    Views:
    846
    Kevin McMurtrie
    Mar 26, 2010
  5. Roedy Green

    Re: Buffered RandomAccessFile

    Roedy Green, Mar 26, 2010, in forum: Java
    Replies:
    1
    Views:
    592
    Arne Vajhøj
    Mar 26, 2010
Loading...

Share This Page