Use disks as large read-write buffer?

[johnfofawn]

Hi,

I have an application where digital data is streamed to me at a rate of
20 MB/sec. I have 6 250GB disks totallying 1.5 TB of storage.

I want to store as much data as possible and age off old data as new
data arrives. I need to allow "readers" to pull from the 20 MB/sec data
stream at any point in the stream. I need to allow for "real-time"
readers that can essentially pull data as it is written into the "disk
buffer".

Are there any software development tools or packages that do this?

My code is in C, compiled using Intel compiler, developed using Visual
Studio on Windows 2003 Server Standard.

My code is portable (it also runs on Linux). If there is a way that I
can do this a preserve portability I would like to do that.

Ideas? Thoughts? Comments?

THANKS!
John

 

[Malcolm]

I have an application where digital data is streamed to me at a rate of
20 MB/sec. I have 6 250GB disks totallying 1.5 TB of storage.

I want to store as much data as possible and age off old data as new
data arrives. I need to allow "readers" to pull from the 20 MB/sec data
stream at any point in the stream. I

My code is portable (it also runs on Linux). If there is a way that I
can do this a preserve portability I would like to do that.

Ideas? Thoughts? Comments?

Take this to comp.programming.

In C terms, the only portable way of doing what you want is to build a file
structure on top of calls to fopen(). Whilst this may just be workable,
almost certainly you want to mange the disks at a lower level.
ANSI C also has no support for signals, ports, or multithreading, though all
can be done with extensions.
It shouldn't be too difficult to build a stream server that will do what you
want.