Connecting File Streams

Discussion in 'C++' started by woessner@gmail.com, Jan 17, 2007.

  1. Guest

    This is not a really well-formed question so please bear with me.

    I have a very large binary file (tens of GB). I would like to write a
    quick program to take a contiguous piece of the file and save it to
    another file. Conceptually, this is very simple:

    Open ifstream
    Open ofstream
    Seek ifstream to beginning of piece
    Read ifstream in to buffer
    Write buffer to ofstream

    I'm not very familiar with streams, but I think even I could handle
    that. My concern is the with the buffer. I'm going to want to copy
    far more data than will fit in memory. That means I will have to use a
    loop to read and write the data in chunks. OK, that's still not a
    problem. But it does raise the question of how big to make the buffer.
    My gut instinct tells me to make the buffer as big as possible, but
    I'm not really sure.

    I'm hoping there's a way to avoid this problem, altogether. Is it
    possible to "connect" an ifstream and an ofstream? Conceptually, this
    is what I'm after:

    Open ifstream
    Open ofstream
    Connect ifstream to ofstream
    Seek ifstream to beginning of piece
    Read ifstream (automatically writes to ofstream)

    My hope is that the stream classes will allocate whatever buffers they
    need and that those buffers will be appropriately sized.

    I've looked through the Josuttis book a little bit. I read the section
    on connecting streams, but that seems to be for synchronization. Does
    anybody know if this is possible (and a good idea)? If not, I'm open
    to any other suggestions.

    Thanks in advance,
    Bill
    , Jan 17, 2007
    #1
    1. Advertising

  2. Yahooooooooo Guest

    streambuf may help to connect these streams

    std::streambuf
    std::cout.rdbuf



    wrote:
    > This is not a really well-formed question so please bear with me.
    >
    > I have a very large binary file (tens of GB). I would like to write a
    > quick program to take a contiguous piece of the file and save it to
    > another file. Conceptually, this is very simple:
    >
    > Open ifstream
    > Open ofstream
    > Seek ifstream to beginning of piece
    > Read ifstream in to buffer
    > Write buffer to ofstream
    >
    > I'm not very familiar with streams, but I think even I could handle
    > that. My concern is the with the buffer. I'm going to want to copy
    > far more data than will fit in memory. That means I will have to use a
    > loop to read and write the data in chunks. OK, that's still not a
    > problem. But it does raise the question of how big to make the buffer.
    > My gut instinct tells me to make the buffer as big as possible, but
    > I'm not really sure.
    >
    > I'm hoping there's a way to avoid this problem, altogether. Is it
    > possible to "connect" an ifstream and an ofstream? Conceptually, this
    > is what I'm after:
    >
    > Open ifstream
    > Open ofstream
    > Connect ifstream to ofstream
    > Seek ifstream to beginning of piece
    > Read ifstream (automatically writes to ofstream)
    >
    > My hope is that the stream classes will allocate whatever buffers they
    > need and that those buffers will be appropriately sized.
    >
    > I've looked through the Josuttis book a little bit. I read the section
    > on connecting streams, but that seems to be for synchronization. Does
    > anybody know if this is possible (and a good idea)? If not, I'm open
    > to any other suggestions.
    >
    > Thanks in advance,
    > Bill
    Yahooooooooo, Jan 18, 2007
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Paul L
    Replies:
    2
    Views:
    330
    Paul L
    Apr 11, 2005
  2. Shea Martin

    Multiple streams per file

    Shea Martin, Aug 16, 2005, in forum: Java
    Replies:
    4
    Views:
    1,179
    Mike Schilling
    Aug 17, 2005
  3. Saulius

    Using file streams in DLL

    Saulius, Aug 16, 2003, in forum: C++
    Replies:
    3
    Views:
    467
    Gianni Mariani
    Aug 17, 2003
  4. Marc Cromme
    Replies:
    1
    Views:
    2,554
    Jorge L Rivera
    Dec 2, 2003
  5. Robert Schweikert
    Replies:
    1
    Views:
    348
    Jack Klein
    Jan 6, 2004
Loading...

Share This Page