std::vector::reserve and std::ifstream::read

[mathieu]

Dear all,

I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:


#include <fstream>
#include <vector>

int main(int argc, char *argv[])
{
const char *filename = argv[1];
std::ifstream is( filename );
const size_t l = 512;
std::vector<char> v;
v.reserve( l ); // need push_back or insert
is.read( &v[0], l ); // ?

return 0;
}


thanks for your help !

 

[gwowen]

Dear all,

  I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:

#include <fstream>
#include <vector>

int main(int argc, char *argv[])
{
  const char *filename = argv[1];
  std::ifstream is( filename );
  const size_t l = 512;
  std::vector<char> v;
  v.reserve( l ); // need push_back or insert

If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()). The
reserve() may well speed your code up.

char tmp;
is.read(&tmp,1);
v.push_back(tmp);

// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));

 

[Michael Doubez]

Dear all,

  I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:

#include <fstream>
#include <vector>

int main(int argc, char *argv[])
{
  const char *filename = argv[1];
  std::ifstream is( filename );
  const size_t l = 512;
  std::vector<char> v;
  v.reserve( l ); // need push_back or insert

If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()).  The
reserve() may well speed your code up.

char tmp;
is.read(&tmp,1);
v.push_back(tmp);

// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));

Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

 

[mathieu]

Dear all,
  I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:
#include <fstream>
#include <vector>
int main(int argc, char *argv[])
{
  const char *filename = argv[1];
  std::ifstream is( filename );
  const size_t l = 512;
  std::vector<char> v;
  v.reserve( l ); // need push_back or insert

If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()).  The
reserve() may well speed your code up.

char tmp;
is.read(&tmp,1);
v.push_back(tmp);

// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));

Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

It does not work for me. istream_iterator does not implement +()
operator:

v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

Thanks

 

[Thomas J. Gritzan]

Am 24.05.2011 13:51, schrieb mathieu:

On May 24, 9:58 am, mathieu<[email protected]> wrote:

Dear all,

I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:

#include<fstream>
#include<vector>

int main(int argc, char *argv[])
{
const char *filename = argv[1];
std::ifstream is( filename );
const size_t l = 512;
std::vector<char> v;
v.reserve( l ); // need push_back or insert

If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()). The
reserve() may well speed your code up.

char tmp;
is.read(&tmp,1);
v.push_back(tmp);

// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));

Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

It does not work for me. istream_iterator does not implement +()
operator:

v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

You are not supposed to do +x on this.

istream_iterator<char>(is) is an iterator refering to the begin of the
stream while istream_iterator<char>() refers to the end of the stream,
just like c.begin() and c.end() for any container.
That pair of iterators reads from the stream until it hits the end of
the stream (like end of file).

If you want to read into a fixed buffer of std::vector:

std::ifstream is( filename );
const size_t size = 512;
std::vector<char> buffer(size);
is.read(&buffer[0], size);

Check for errors after this and use is.gcount() to get the number of
bytes read.

 

[Michael Doubez]

Dear all,
  I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:
#include <fstream>
#include <vector>
int main(int argc, char *argv[])
{
  const char *filename = argv[1];
  std::ifstream is( filename );
  const size_t l = 512;
  std::vector<char> v;
  v.reserve( l ); // need push_back or insert
If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()).  The
reserve() may well speed your code up.
char tmp;
is.read(&tmp,1);
v.push_back(tmp);
// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));

Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

It does not work for me. istream_iterator does not implement +()
operator:

Yes, they are not random access input iterator.

  v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

You want to read char by char ?

char c;

If you want formatted input then:
is>>c;
If you want unformatted input then:
is.get(c);

And you loop:
while( is.get(c) ) {
v.push_back(c);
}

 

[mathieu]

Am 24.05.2011 13:51, schrieb mathieu:

Dear all,
   I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:
#include<fstream>
#include<vector>
int main(int argc, char *argv[])
{
   const char *filename = argv[1];
   std::ifstream is( filename );
   const size_t l = 512;
   std::vector<char>  v;
   v.reserve( l ); // need push_back or insert
If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()).  The
reserve() may well speed your code up.
char tmp;
is.read(&tmp,1);
v.push_back(tmp);
// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char>  V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));
Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

It does not work for me. istream_iterator does not implement +()
operator:

   v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

You are not supposed to do +x on this.

istream_iterator<char>(is) is an iterator refering to the begin of the
stream while istream_iterator<char>() refers to the end of the stream,
just like c.begin() and c.end() for any container.
That pair of iterators reads from the stream until it hits the end of
the stream (like end of file).

If you want to read into a fixed buffer of std::vector:

std::ifstream is( filename );
const size_t size = 512;
std::vector<char> buffer(size);

As said in my original post, I do not want to initialize the vector,
simply because this should not be required.

Thanks

 

[mathieu]

Dear all,
  I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:
#include <fstream>
#include <vector>
int main(int argc, char *argv[])
{
  const char *filename = argv[1];
  std::ifstream is( filename );
  const size_t l = 512;
  std::vector<char> v;
  v.reserve( l ); // need push_back or insert
If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()).  The
reserve() may well speed your code up.
char tmp;
is.read(&tmp,1);
v.push_back(tmp);
// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));
Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

It does not work for me. istream_iterator does not implement +()
operator:

Yes, they are not random access input iterator.

  v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

[...]
is.get(c);

And you loop:
while( is.get(c) ) {
  v.push_back(c);

}

Ok. Let's hope multiple function calls are not expensive.

thanks

 

[gwowen]

As said in my original post, I do not want to initialize the vector,
simply because this should not be required.

In which case, reserve() the size, then push_back() the elements.

 

[Juha Nieminen]

const char *filename = argv[1];
std::ifstream is( filename );
const size_t l = 512;
std::vector<char> v;
v.reserve( l ); // need push_back or insert
is.read( &v[0], l ); // ?

If the size of that buffer is indeed a compile-time constant, then you
could simply use a static array instead of std::vector. It will be much
more efficient that way too. (In fact, if you want it to be as efficient
as possible, use std::fread() instead of std::ifstream::read(), but that's
a different issue.)

 

[Kai-Uwe Bux]

Am 24.05.2011 13:51, schrieb mathieu:

On 24 mai, 11:16, gwowen<[email protected]> wrote:

On May 24, 9:58 am, mathieu<[email protected]> wrote:

Dear all,

I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:

#include<fstream>
#include<vector>

int main(int argc, char *argv[])
{
const char *filename = argv[1];
std::ifstream is( filename );
const size_t l = 512;
std::vector<char> v;
v.reserve( l ); // need push_back or insert

If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()). The
reserve() may well speed your code up.

char tmp;
is.read(&tmp,1);
v.push_back(tmp);

// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char> V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));

Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());

It does not work for me. istream_iterator does not implement +()
operator:

v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

You are not supposed to do +x on this.

istream_iterator<char>(is) is an iterator refering to the begin of the
stream while istream_iterator<char>() refers to the end of the stream,
just like c.begin() and c.end() for any container.
That pair of iterators reads from the stream until it hits the end of
the stream (like end of file).

If you want to read into a fixed buffer of std::vector:

std::ifstream is( filename );
const size_t size = 512;
std::vector<char> buffer(size);

As said in my original post, I do not want to initialize the vector,
simply because this should not be required.

Would it be fine, to construct the vector from a pair of istream_iterators?

std::vector< char > v
( std::istream_iterator<char>( is ), std::istream_iterator<char>() );

Now, this will put the _whole_ stream into the vector. If you only want part
of it and know how many characters, you could use

template < typename FromIter, typename ToIter, typename Int >
FromIter copy_n ( FromIter from, ToIter to, Int n ) {
for ( Int i = 0; i < n; ++i ) {
*from = *to;
++ from;
++ to;
}
return ( from );
}

and then

std::vector< char > v;
copy_n( std::istream_iterator< char >( is ),
std::back_inserter( v ),
the_length );


Best,

Kai-Uwe Bux

 

[Joshua Maurice]

Am 24.05.2011 13:51, schrieb mathieu:

Dear all,
   I do not understand how to use the STL vector class with the
ifstream class. I would like to reserve a chunk of memory (no
initialization is required) and fill it with values from a file. As
far as I understand vector::reserve requires a subsequent call to
push_back or insert. However I do not see how I can do this in the
following example:
#include<fstream>
#include<vector>
int main(int argc, char *argv[])
{
   const char *filename = argv[1];
   std::ifstream is( filename );
   const size_t l = 512;
   std::vector<char>  v;
   v.reserve( l ); // need push_back or insert
If you reserve() space, you can't read or write to that space until
you also adjust the size (as push_back() will, or resize()).  The
reserve() may well speed your code up.
char tmp;
is.read(&tmp,1);
v.push_back(tmp);
// the C++/STL like solution will use an istream_iterator and a
back_inserter
// I've got to say, I don't care for it...
vector<char>  V;
copy(istream_iterator<char>(is), istream_iterator<char>(),
back_inserter(V));
Or simply:
v.assign(stream_iterator<char>(is), istream_iterator<char>());
It does not work for me. istream_iterator does not implement +()
operator:
   v.assign( std::istream_iterator<char>(is),
std::istream_iterator<char>(is)+l);

You are not supposed to do +x on this.

istream_iterator<char>(is) is an iterator refering to the begin of the
stream while istream_iterator<char>() refers to the end of the stream,
just like c.begin() and c.end() for any container.
That pair of iterators reads from the stream until it hits the end of
the stream (like end of file).

If you want to read into a fixed buffer of std::vector:

std::ifstream is( filename );
const size_t size = 512;
std::vector<char> buffer(size);

As said in my original post, I do not want to initialize the vector,
simply because this should not be required.

There's no way to do what you want with std::vector. Perhaps the
compiler may optimize whatever you do to be as good as if you used a
raw array. I don't know.

This is a slight annoying with std::vector. It requires all of its
contained elements to be initialized to make the guarantee that the
vector object is always in a copyable state, as that guarantee was
desirable. This means that sometimes std::vector is indeed measurably
slower than raw arrays. Are you prematuring optimizing though?

 

[Jeff Flinn]

const char *filename = argv[1];
std::ifstream is( filename );
const size_t l = 512;
std::vector<char> v;
v.reserve( l ); // need push_back or insert
is.read( &v[0], l ); // ?

If the size of that buffer is indeed a compile-time constant, then you
could simply use a static array instead of std::vector. It will be much
more efficient that way too. (In fact, if you want it to be as efficient
as possible, use std::fread() instead of std::ifstream::read(), but that's
a different issue.)

Even more efficient would be to avoid the copy altogether accessing the
file via memory mapping. boost iostreams library provides a portable
implementation. See
http://www.boost.org/doc/libs/1_46_1/libs/iostreams/doc/index.html

Jeff