is std::ifstream buffered or not?

[Karl]

Hey everyone!

I've got a quick question on whether std::ifstream is buffered or not.
The reason is that I have a homework assignment that requires me to
benchmark copying files using different buffer sizes. I ended up doing
this using std::istream::readsome() and std:stream::write():

// now try writing to the destination file
std::ifstream sourceStream(sourceFilename.c_str(),
std::ios::binary);
std:fstream destinationStream(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);

// determine the size of the file
sourceStream.seekg(0, std::ios::end);
const std::streamsize totalLength = sourceStream.tellg();
sourceStream.seekg(0, std::ios::beg);

// now writing the actual data
char buffer[numberOfBytes];
std::streamsize length = 0;
while(length != totalLength)
{
int l = sourceStream.readsome(buffer, numberOfBytes);
destinationStream.write(buffer, l);
length += l;
}
// now close the stream
sourceStream.close();
destinationStream.close();

Now, the validity of benchmarking this could possibly be in question
because std::istream::readsome() could be doing some buffering
(readahead?) in the back, so using different buffer sizes would
possibly be meaningless.

I understand std::cout and std::cin is buffered, and it makes sense
that they are. However, I do not see why std::ifstream and
std:fstream would be buffered because the filesystem and even the
harddrive does some buffering. Wouldn't that be meaningless?

Then again, this could depend on the implementation. I'm not familiar
enough with the C++ STL specification.

Can anyone help me out? Any help would be greatly appreciated.

 

[Tom Widmer]

Hey everyone!

I've got a quick question on whether std::ifstream is buffered or not.

It is, typically.

The reason is that I have a homework assignment that requires me to
benchmark copying files using different buffer sizes. I ended up doing
this using std::istream::readsome() and std:stream::write():

// now try writing to the destination file
std::ifstream sourceStream(sourceFilename.c_str(),
std::ios::binary);
std:fstream destinationStream(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);

// determine the size of the file
sourceStream.seekg(0, std::ios::end);
const std::streamsize totalLength = sourceStream.tellg();
sourceStream.seekg(0, std::ios::beg);

// now writing the actual data
char buffer[numberOfBytes];
std::streamsize length = 0;
while(length != totalLength)
{
int l = sourceStream.readsome(buffer, numberOfBytes);
destinationStream.write(buffer, l);
length += l;
}
// now close the stream
sourceStream.close();
destinationStream.close();

Now, the validity of benchmarking this could possibly be in question
because std::istream::readsome() could be doing some buffering
(readahead?) in the back, so using different buffer sizes would
possibly be meaningless.

I understand std::cout and std::cin is buffered, and it makes sense
that they are. However, I do not see why std::ifstream and
std:fstream would be buffered because the filesystem and even the
harddrive does some buffering. Wouldn't that be meaningless?

Not at all. Buffering is useful at many levels. For iostreams, buffering
is done to reduce the number of virtual function calls made, and to
reduce the number of OS read calls made, both of which improve
performance regardless of any buffering happening at lower levels.

Then again, this could depend on the implementation. I'm not familiar
enough with the C++ STL specification.

Can anyone help me out? Any help would be greatly appreciated.

You can disable iostream level buffering like this:
std::ifstream sourceStream;
sourceStream.rdbuf()->pubsetbuf(0, 0);
sourceStream.open(sourceFilename.c_str(), std::ios::binary);

std:fstream destinationStream;
destinationStream.rdbuf()->pubsetbuf(0, 0);
destinationStream.open(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);

Note for it to be sure to work, you need to set up the buffer before
opening the file.

Tom

 

[Karl]

You can disable iostream level buffering like this:
std::ifstream sourceStream;
sourceStream.rdbuf()->pubsetbuf(0, 0);
sourceStream.open(sourceFilename.c_str(), std::ios::binary);

std:fstream destinationStream;
destinationStream.rdbuf()->pubsetbuf(0, 0);
destinationStream.open(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);

Note for it to be sure to work, you need to set up the buffer before
opening the file.

Thank you Tom, that's perfect! I looked at the pubsetbuf()
documentation and see that that does exactly what I need. Thanks!