How to pass a huge matrix to a function

[Bernhard Reinhardt]

Hi,


I read data from a file into a 4 dimensional array. The data has a size
of 5MB (could later be up to 500MB). I want to pass the matrix to
subroutines.

What is the cleverst (and for a beginner easies) way to do so?

I read a bit about pointers on the web and in books but the examples do
only cover one-dimensional arrays.

Regards

Bernhard

 

[g.vukoman]

Hi,

I read data from a file into a 4 dimensional array. The data has a size
of 5MB (could later be up to 500MB). I want to pass the matrix to
subroutines.

What is the cleverst (and for a beginner easies) way to do so?

I read a bit about pointers on the web and in books but the examples do
only cover one-dimensional arrays.

Regards

Bernhard

In your case I would use a stl container. By example a pointer points
to a vector container...

 

[Victor Bazarov]

I read data from a file into a 4 dimensional array. The data has a
size of 5MB (could later be up to 500MB). I want to pass the matrix to
subroutines.

What is the cleverst (and for a beginner easies) way to do so?

Wrap it into a class/struct and pass it by reference.

I read a bit about pointers on the web and in books but the examples
do only cover one-dimensional arrays.

One-dimensional arrays decay to pointers so easily that it's not worth
talking about one-dimensional array in such case. Two-dimensional ones
and up are better as far as examples are concerned. But then again,
wrapping them in a struct is probably still the simplest. And if your
array is dynamic, you're much better off with a class that handles
allocation/deallocation for you, and has element access functionality
implemented. Search the web for "Array template" or something.

V

 

[Bernhard Reinhardt]

In your case I would use a stl container. By example a pointer points
to a vector container...

Ok it seems I´m on the right way. I already convert my array[][][][] to
a vector < vector < vector < vector <double> > > >

I´m partly working with the vector an partly with the normal array
because I don´t know how to address a unique element of the matrix in
vector form.

I can do
vector <double> a;
cout<<a[0];

but

vector <vector <double>> b;
cout<<b[0][0];

wont compile.


Next question:
Is it possible to pass a vector to a function in such manner that the
function will work on the original data and will not make a local copy?

In a later stage my matrix may have 10 dimensions. So it will become a
vector < vector < vector < vector < vector < vector < vector < vector <
vector < vector < vector < vector < vector <double> > > > > > > > > >

that seems not quite handy to me.

Regards

Bernhard

 

[Victor Bazarov]

[..]
I can do
vector <double> a;
cout<<a[0];

but

vector <vector <double>> b;
cout<<b[0][0];

wont compile.

Well, if you put the space between the closing angle brackets, it should.

vector<vector<double> > b;
cout << b[0][0];

If you still have a problem, read the FAQ 5.8.

Next question:
Is it possible to pass a vector to a function in such manner that the
function will work on the original data and will not make a local
copy?

Passing a vector by a reference or a pointer should accomplish that.

In a later stage my matrix may have 10 dimensions. So it will become a
vector < vector < vector < vector < vector < vector < vector < vector


that seems not quite handy to me.

Use typedef to shortent this.

typedef vector<vector<vector<double> > > vd3D;
typedef vector<vector<vector<vd3D> > > vd6D;
typedef vector<vector<vector<vd6D> > > vd9D;

and so on.

V

 

[Guest]

In your case I would use a stl container. By example a pointer points
to a vector container...

Ok it seems I´m on the right way. I already convert my array[][][][] to
a vector < vector < vector < vector <double> > > >

I´m partly working with the vector an partly with the normal array
because I don´t know how to address a unique element of the matrix in
vector form.

I can do
vector <double> a;
cout<<a[0];

but

vector <vector <double>> b;
cout<<b[0][0];

wont compile.

It is a defect in the language, you must put a space between the two >
or the compiler will interpret it as the >> operator, i.e.

vector<vector<double> > b;

Notice that your code above will have undefined behaviour since the
vectors are empty, so trying to access an non-existent element is a bad
idea.

Next question:
Is it possible to pass a vector to a function in such manner that the
function will work on the original data and will not make a local copy?

Pass the parameters as references:

void foo(vector<double>& vec);

Notice the '&' after the type, this means that it is a reference to a
vector<double>. Be aware that when working with references all changes
made to vec in foo() will affect the original vector. To prevent
modifications in the function you can pass the vector as a const reference:

void foo(const vector<double>& vec);

 

[Vladislav.Lazarenko]

It is a defect in the language, you must put a space between the two >
or the compiler will interpret it as the >> operator, i.e.

It is not a defect! C++ is really powerful and complex language. In
template argument list you should always use "> >" to tell compiler
that this is not right shift operator. See
http://www.comeaucomputing.com/techtalk/templates/#shiftshift

 

[Victor Bazarov]

It is not a defect! C++ is really powerful and complex language. In
template argument list you should always use "> >" to tell compiler
that this is not right shift operator. See
http://www.comeaucomputing.com/techtalk/templates/#shiftshift

There was a discussion about possibly adjusting the grammar so that
it accommodates the angle brackets not separated by whitespace, and
Andrew Koenig (IIRC) said it was possible. I don't have a link to
the discussion, but it can probably be located on Google Groups
relatively quickly. In the code

vector<vector<double>> v;

there can be no right shift, since a type (double) cannot be right-
shifted. But the ambiguity becomes apparent in this situation:

template<int a> class foo {};

vector<foo<12>> b;

Are we shifting 12 to the right 'b' times, or are we defining 'b'
as a 'vector' of 'foo<12>'? If the former, there are syntax errors
since the template argument list is not closed, or maybe 'b' is not
defined. If the latter, the code is OK (unless 'b' has been already
declared or some such). The compiler cannot decide to interpret the
code in such a way that it's well-formed.

To solve this, the same requirement as with a single angle bracket
(that can be interpreted as the greater-than operator) would be used:
if it can close the declaration of a template, it shall. Ambiguity
is resolved with parentheses.

Here is another (contrived) example of what would be ambiguous code
had there been no rule "disambiguation is in favor of closing angle
bracket":

template<int a> class foo { foo(int); };

template<int a>
bool operator > (foo<a> const&, int);

const int y = 2;
const int z = 3;

int main() {
foo< 12 > (z) > y; // ambiguious?
}

The ambiguity is resolved by the "any angle bracket closes the
previous angle bracket", and the interpretation is:

( foo<12>(z) ) > y; // temporary of type foo<12> constructed
// with 3 as argument is compared to 2

And if you want it to be interpreted as the 'greater-than'
operator, use parentheses:

foo< (12 > (z)) > y; // definition of 'y' of type foo<1>

V

 

[BobR]

Erik Wikström wrote in message...

(e-mail address removed) wrote:

Ok it seems I´m on the right way. I already convert my array[][][][] to
a vector < vector < vector < vector <double> > > >

I´m partly working with the vector an partly with the normal array
because I don´t know how to address a unique element of the matrix in
vector form.

I can do
vector <double> a;
cout<<a[0];

but

vector <vector <double>> b;
cout<<b[0][0];

wont compile.

It is a defect in the language, you must put a space between the two >
or the compiler will interpret it as the >> operator, i.e.

vector<vector<double> > b;

Notice that your code above will have undefined behaviour since the
vectors are empty, so trying to access an non-existent element is a bad
idea.

OP: To watch that happen, try this (cut&paste):

#include <iostream>
#include <vector>
#include <stdexcept>
int main(){
std::vector<std::vector<double> > vvd;
try{
std::cout<< vvd.at(0).at(0); // cout<<vvd[0][0];
} // try
catch( std:ut_of_range const &Oor ){
std::cout<<"caught "<<Oor.what()<<std::endl;
}
return 0;
} // main()

You can init them:

std::vector<std::vector<double> > vvd( 2, 2 ); // a 2x2

std::vector<std::vector<double> > vvd24( 2,
std::vector<double>(4, 3.14 ) ); // a 2x4 all inited to 3.14

Quad init gets ugly, unless you use typedef(s):

std::vector<std::vector<std::vector<
std::vector<double> > > > QuadArray( 2,
std::vector<std::vector<std::vector<double> > >( 3,
std::vector<std::vector<double> >( 4, 5 ) ) ); //
2x3x4x5
QuadArray.at( 1 ).at( 2 ).at( 3 ).at( 4 ) = 42.31;
// same: QuadArray[1][2][3][4] = 42.31;

 

[James Kanze]

It is not a defect!

The standards committee disagrees with you. In the next version
of the standard, the space will not be necessary; if a >> token
occurs in a context where a > which closes a template argument
list (but not a > which means greater than) is legal, it is
broken down into two '>' tokens. Note that this change will
break code like:

template< int I > class T ;

T< x >> 3 > t ;

But such code is probably not very frequent, and there is a
simple work-around:

T< (x >> 3) > t ;

Now if there were only such a simple fix for the most
embarassing parse problem (where what you thought defined a
variable in fact declares a function).

 

[xtrigger303]

The standards committee disagrees with you. In the next version
of the standard, the space will not be necessary; if a >> token
occurs in a context where a > which closes a template argument
list (but not a > which means greater than) is legal, it is
broken down into two '>' tokens. Note that this change will
break code like:

template< int I > class T ;

T< x >> 3 > t ;

But such code is probably not very frequent, and there is a
simple work-around:

T< (x >> 3) > t ;

Now if there were only such a simple fix for the most
embarassing parse problem (where what you thought defined a
variable in fact declares a function).

--
James Kanze (GABI Software) email:[email protected]
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

Hi to all,
you can try a little metaprogramming to avoid nested typedefs. Try if
something like this works.
Bye,
Francesco

#include <iostream>
#include <vector>
#include <string>

struct CMultiDimContainer
{
typedef TContainer< typename CMultiDimContainer< TContainer, T,
KSize - 1 >::CType > CType;
};

template< template< typename > class TContainer, typename T >
struct CMultiDimContainer< TContainer, T, 1 >
{
typedef TContainer< T > CType;
};

//

template< int KInt >
struct CTypeFromInt
{
enum { kResult = KInt };
};

//

template< typename T, template< typename > class TTemplClass >
struct CIsInstanceOf
{
enum { kResult = false };
};

template< typename T, template< typename > class TTemplClass >
struct CIsInstanceOf< TTemplClass< T >, TTemplClass >
{
enum { kResult = true };
};

//

template< template< typename T > class TContainer, typename T >
void MultiDimResize( TContainer< T > & inContainer, int inSize );

template< typename TContainer >
void MultiDimResize ( TContainer & inContainer, int inSize,
CTypeFromInt< true > )
{
for( typename TContainer::iterator theIter = inContainer.begin();
theIter != inContainer.end(); ++theIter )
MultiDimResize( *theIter, inSize );
}

template< typename TContainer >
void MultiDimResize( TContainer & inContainer, int inSize,
CTypeFromInt< false > )
{}

template< template< typename T > class TContainer, typename T >
void MultiDimResize( TContainer< T > & inContainer, int inSize )
{
inContainer.resize( inSize );
MultiDimResize( inContainer, inSize, CTypeFromInt< CIsInstanceOf<
T, TContainer >::kResult >() );
}

//

int main( void )
{
CMultiDimContainer< std::vector, int, 3 >::CType theMultiVec;

MultiDimResize( theMultiVec, 10 );

std::cout << theMultiVec.size() << std::endl;
std::cout << theMultiVec[ 4 ].size() << std::endl;
std::cout << theMultiVec[ 5 ][ 3 ].size() << std::endl;
std::cout << theMultiVec[ 2 ][ 7 ][ 4 ] << std::endl;

std::string str; // just to block this *#$^&$
std::cin >> str;
}