stream to file a matrix

[bob]

Hi,

given a vector of vectors of strings thus;

typedef std::vector<std::string> product;
typedef std::vector<product> product_matrix;


whats the fastest, most efficient means of streaming the
"product_matrix" to a file? I.E. without using two "for" or "while"
loops. Is there a means of using copy ? I'd like to use stl's copy
along with my 2 dimensional vector of strings. Is that possible? Or
should I just iterate over the contents myself ?

thanks much

G

 

[Ben Pope]

Hi,

given a vector of vectors of strings thus;

typedef std::vector<std::string> product;
typedef std::vector<product> product_matrix;


whats the fastest, most efficient means of streaming the
"product_matrix" to a file? I.E. without using two "for" or "while"
loops.

How do you intend to do that?

Is there a means of using copy ?

Yes, you would essentially need to call copy for each element of
product_matrix, and then each element of the products.

I'd like to use stl's copy
along with my 2 dimensional vector of strings. Is that possible? Or
should I just iterate over the contents myself ?

How do you think copy is implemented?

I would create a stream operator for product and product_matrix:


#include <vector>
#include <string>
#include <iostream>

typedef std::vector<std::string> product;
typedef std::vector<product> product_matrix;

std:stream& operator<<(std:stream& s, const product& p) {
std::copy(p.begin(), p.end(),
std:stream_iterator<std::string>(s, ", "));
return s;
}

std:stream& operator<<(std:stream& s, const product_matrix& p) {
std::copy(p.begin(), p.end(),
std:stream_iterator<product>(s, "\n"));
return s;
}

int main() {
product p(10, "test");
product_matrix m(10, p);

std::cout << m;
}


The only problem is that doesn't actually work. What did I do wrong?

Yes, there are nested loops, but there really is no way around that.

Ben Pope

 

[Dietmar Kuehl]

given a vector of vectors of strings thus;

typedef std::vector<std::string> product;
typedef std::vector<product> product_matrix;


whats the fastest, most efficient means of streaming the
"product_matrix" to a file? I.E. without using two "for" or "while"
loops. Is there a means of using copy ? I'd like to use stl's copy
along with my 2 dimensional vector of strings. Is that possible? Or
should I just iterate over the contents myself ?

'std::copy()' only works on sequences. There are several option,
however, to process a matrix as a sequence:

- You can have 'std::copy()' process each row of the matrix and
arrange for the rows to use 'std::copy()' internally - at least
this would be the case if you would use a user defined type: the
issue with this approach is that there is no output operator
defined for 'std::vector<T>' and you are only allowed to define
one if the type involves a user defined type somehow. On the other
hand, something like the following works in practice but it is not
guaranteed to work:

namespace std {
std:stream& operator(std:stream& out,
std::vector<std::string> const& v)
std::copy(v.begin(), v.end(),
std:stream_iterator<std::string>(out, ","));
return out;
}

Now you can use 'std::copy()' with an appropriate output iterator
over 'product's.

- Instead of using 'std::copy()' you could use 'std::transform()'
using the above operator with an appropriate name and put it into
an appropriate namespace. This is, in some sense, the portable
alternative to the non-portable use of 'std::copy()'.

- You could create a special iterator which actually consists of
two iterators internally, one for the current row and one for the
current column within the row. This would give a kind of a "flat"
view of the matrix which an be used directly with 'std::copy()'.

 

[Rolf Magnus]

Hi,

given a vector of vectors of strings thus;

typedef std::vector<std::string> product;
typedef std::vector<product> product_matrix;


whats the fastest, most efficient means of streaming the
"product_matrix" to a file? I.E. without using two "for" or "while"
loops.

What makes you believe a loop is slow or inefficient?

Is there a means of using copy ? I'd like to use stl's copy
along with my 2 dimensional vector of strings. Is that possible? Or
should I just iterate over the contents myself ?

I don't think there is a way to use copy that way.

 

[Victor Bazarov]

[..] On the other
hand, something like the following works in practice but it is not
guaranteed to work:

namespace std {
std:stream& operator(std:stream& out,

Was it supposed to be

std:stream& operator << (std:stream& out,

? Or did my newsreader eat the "less-than" signs?

std::vector<std::string> const& v)
std::copy(v.begin(), v.end(),
std:stream_iterator<std::string>(out, ","));
return out;
}
[...]

V

 

[Jacek Dziedzic]

Hi,

given a vector of vectors of strings thus;

typedef std::vector<std::string> product;
typedef std::vector<product> product_matrix;


whats the fastest, most efficient means of streaming the
"product_matrix" to a file? I.E. without using two "for" or "while"
loops. Is there a means of using copy ? I'd like to use stl's copy
along with my 2 dimensional vector of strings. Is that possible? Or
should I just iterate over the contents myself ?

What makes you think you need it in the first place?
I'd bet the I/O would be the bottleneck here in nine out
of ten cases. If you optimize the streaming, it will only
_wait more quickly_ for the I/O.

You didn't mention if you write the matrix in binary or
text format. The latter is usually slow when the streams
are used because of the extra overhead of conversion.

HTH,
- J.

 

[Dietmar Kuehl]

[..] On the other
hand, something like the following works in practice but it is not
guaranteed to work:

namespace std {
std:stream& operator(std:stream& out,

Was it supposed to be

std:stream& operator << (std:stream& out,

?

Yes. I should probably try to compile the code instead of just
typing it in the newsreader... Thank you for catching this error.

 

[Dietmar Kuehl]

I would create a stream operator for product and product_matrix:

Note, that this code is not supposed to compile! The problem is
that output operator used by 'std:stream_iterator<int>' is sought
only in namespace 'std'. However, since the involved types are all
built-in types, you are - strictly speaking - not allowed to define
the output operator there! Technically, it is likely to work if you
just plug the output operators into namespace 'std' but the
behavior of the program is not defined.

 

[Ben Pope]

Note, that this code is not supposed to compile! The problem is
that output operator used by 'std:stream_iterator<int>' is sought
only in namespace 'std'. However, since the involved types are all
built-in types, you are - strictly speaking - not allowed to define
the output operator there! Technically, it is likely to work if you
just plug the output operators into namespace 'std' but the
behavior of the program is not defined.

Thanks for that Dietmar.

I guess if you want to do this, it's best to wrap the types rather than
use a typedef.

Ben Pope

 

[Fei Liu]

Note, that this code is not supposed to compile! The problem is
that output operator used by 'std:stream_iterator<int>' is sought
only in namespace 'std'. However, since the involved types are all

what do you mean 'sought only in name space std'?

built-in types, you are - strictly speaking - not allowed to define
the output operator there! Technically, it is likely to work if you

I don't see the connection between built-in type and reason why output
operator is not allowed to be fined in 'std', can you elaborate on this
please?

just plug the output operators into namespace 'std' but the
behavior of the program is not defined.

Not sure about the meaning of 'plug the output operators into name
std'...And why is the program behavior not defined? The code seem to
compile/run perfectly.

 

[Dietmar Kuehl]

what do you mean 'sought only in name space std'?

Which part you didn't understand? I'm not qualified to give you a
lecture in English and thus I assume you understood the statement
per se (otherwise use a dictionary).

On a technical level, 'std:stream_iterator<T>' is a template and
look-up of names in separated into two phases which is thus called
two-phase-look-up. For the dependent names, i.e. those depending on
the template arguments, only the second phase, i.e. argument dependent
look-up, applies. The rules for this are that only namespace are
considered in which one of the arguments is defined (the specification
of this are tricky but just collecting all namespace qualifiers of the
fully qualified names of the arguments will to the trick). The only
namespace involved in you class is 'std', i.e. this is the only
namespace in which names are sought.

I don't see the connection between built-in type and reason why output
operator is not allowed to be fined in 'std', can you elaborate on this
please?

It is a restriction imposed to protect the library implementation
from being hosed up by people defining more or less arbitrary
functions in namespace 'std': as a user of the standard C++ library
you are only allowed to define certain functions in namespace 'std'
and when you are allowed to define a function the signature has to at
least contain one user defined type.

Not sure about the meaning of 'plug the output operators into name
std'...And why is the program behavior not defined? The code seem to
compile/run perfectly.

I suggested to disobey the rules and define a function in namespace
'std' which are, strictly speaking, not allowed to define in
namespace 'std'. This is likely to work but the standard does not
give you any guarantee that it indeed will work because this violates
a specific restriction. These restrictions are there to protect the
library implementations: it is pretty easy to put functions into
namespace 'std' which will almost certainly cause the library to fail
in funny way. Also, compilation might easily fail if the library
implementor choose to put similar or even the same function into his
implementation.

 

[Fei Liu]

Which part you didn't understand? I'm not qualified to give you a
lecture in English and thus I assume you understood the statement
per se (otherwise use a dictionary).

On a technical level, 'std:stream_iterator<T>' is a template and
look-up of names in separated into two phases which is thus called
two-phase-look-up. For the dependent names, i.e. those depending on
the template arguments, only the second phase, i.e. argument dependent
look-up, applies. The rules for this are that only namespace are
considered in which one of the arguments is defined (the specification
of this are tricky but just collecting all namespace qualifiers of the
fully qualified names of the arguments will to the trick). The only
namespace involved in you class is 'std', i.e. this is the only
namespace in which names are sought.


It is a restriction imposed to protect the library implementation
from being hosed up by people defining more or less arbitrary
functions in namespace 'std': as a user of the standard C++ library
you are only allowed to define certain functions in namespace 'std'
and when you are allowed to define a function the signature has to at
least contain one user defined type.


I suggested to disobey the rules and define a function in namespace
'std' which are, strictly speaking, not allowed to define in
namespace 'std'. This is likely to work but the standard does not
give you any guarantee that it indeed will work because this violates
a specific restriction. These restrictions are there to protect the
library implementations: it is pretty easy to put functions into
namespace 'std' which will almost certainly cause the library to fail
in funny way. Also, compilation might easily fail if the library
implementor choose to put similar or even the same function into his
implementation.

Thank you, I didn't know there exists such kind of limitation on
implementing function in namespace 'std'. Too many rules...