Methods for fast binary file output?

Discussion in 'C Programming' started by email.ttindustries@gmail.com, Jul 19, 2007.

  1. Guest

    Hello,

    I am a C/C++ newbie and I have a simple question concerning fast data
    writing to binary files. Are there any other faster methods than the
    standard write() method to write to binary data files? I ask this
    question because I have to store a big amount of data coming from an
    PCI A/D card with a sampling frequency of 20 MB/s. Now I have to
    implement some additional computations and would need to speed up the
    data transfer. I am using a Windows XP computer and GCC 3.4.3 and
    Borland 5.5 respectively. Are there any special free libraries for
    this purpose?

    Thank you very much

    Daniel
    , Jul 19, 2007
    #1
    1. Advertising

  2. santosh Guest

    On Thursday 19 Jul 2007 8:17 pm, <
    > wrote in message
    <>:

    > Hello,
    >
    > I am a C/C++ newbie and I have a simple question concerning fast
    > data writing to binary files. Are there any other faster methods
    > than the standard write() method to write to binary data files? I
    > ask this question because I have to store a big amount of data
    > coming from an PCI A/D card with a sampling frequency of 20 MB/s.
    > Now I have to implement some additional computations and would need
    > to speed up the data transfer. I am using a Windows XP computer and
    > GCC 3.4.3 and Borland 5.5 respectively. Are there any special free
    > libraries for this purpose?
    >
    > Thank you very much


    Standard C only has fwrite for this purpose. If this is not
    satisfactory, (have you actually measured and found it to be so?),
    you'll have to see what your system provides. There may be special
    API by which you can specify increased buffering, asynchronous I/O
    and the like. But ultimately, the limiting factor is likely to be
    the speed of your disks. If you really need the writes to as fast as
    possible, invest in a RAID array of fast SCSI disks and more memory.

    At 20 MB/s, I don't think you need to worry about the disk writes
    stalling your program.
    santosh, Jul 19, 2007
    #2
    1. Advertising

  3. <> wrote in message
    news:...
    > Hello,
    >
    > I am a C/C++ newbie and I have a simple question concerning fast data
    > writing to binary files. Are there any other faster methods than the
    > standard write() method to write to binary data files? I ask this
    > question because I have to store a big amount of data coming from an
    > PCI A/D card with a sampling frequency of 20 MB/s. Now I have to
    > implement some additional computations and would need to speed up
    > the data transfer. I am using a Windows XP computer and GCC 3.4.3
    > and Borland 5.5 respectively. Are there any special free libraries for
    > this purpose?
    >

    Not nowadays. If fwrite() won't write the data fast enough then you need a
    physically superior backing store. That didn't use to be the case and you
    could play around with buffers to make them match physical disk segments,
    but with a modern processor it won't make any difference.

    --
    Free games and programming goodies.
    http://www.personal.leeds.ac.uk/~bgy1mm
    Malcolm McLean, Jul 19, 2007
    #3
  4. SM Ryan Guest

    wrote:
    # Hello,
    #
    # I am a C/C++ newbie and I have a simple question concerning fast data
    # writing to binary files. Are there any other faster methods than the
    # standard write() method to write to binary data files? I ask this

    Maybe memory mapped files if you have them available. Memory map
    uses the CPU to write bytes and only goes to kernel on page faults.
    The kernel might also guess to preallocate pages before you fault.

    --
    SM Ryan http://www.rawbw.com/~wyrmwif/
    Why are we here?
    whrp
    SM Ryan, Jul 21, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    643
  2. Michele Simionato

    Python is darn fast (was: How fast is Python)

    Michele Simionato, Aug 23, 2003, in forum: Python
    Replies:
    13
    Views:
    546
  3. Replies:
    3
    Views:
    423
    Just me
    Jul 23, 2007
  4. Juha Nieminen
    Replies:
    22
    Views:
    987
    Kai-Uwe Bux
    Oct 12, 2007
  5. Kenneth McDonald
    Replies:
    5
    Views:
    296
    Kenneth McDonald
    Sep 26, 2008
Loading...

Share This Page