Memory-hogging variables - swap to disk?

[mrstephengross]

Hi there... I've got a problem with memory usage. Specifically, I've
got some data structures that grow to be really gigantic over the
course of the program's execution. I'm wondering if there might be a
way to ensure that the data structure is stored on disk rather than in
memory. I realize that the OS is supposed to do this anyway for large
programs, but in this case I'm wondering if it's possible to implement
for a specific data structure. Consider the following class:

// A class that stores results as they're generated:
struct ResultsMgr
{
void addResults(const Results & results) { /* ... */ }
void exportResults() { /* ... */ }
};

And let's say it's used like so:

ResultsMgr mgr;
for(AnalysisItr i = analyses.begin(); i != analyses.end(); ++i)
{
mgr.addResults(i->execute());
}
mgr.exportResults();

In the above example, the ResultsMgr is only used (1) when results are
ready to be added to it, and (2) at the end, to export the results
somehow.

Since the ResultsMgr is only used in a few places, it might make sense
to let it be store on in a random-access file. Is there any way to do
this in c++? Has anyone written such a utility?

Thanks in advance,
--Steve ([email protected])

 

[mlimber]

Hi there... I've got a problem with memory usage. Specifically, I've
got some data structures that grow to be really gigantic over the
course of the program's execution. I'm wondering if there might be a
way to ensure that the data structure is stored on disk rather than in
memory. I realize that the OS is supposed to do this anyway for large
programs, but in this case I'm wondering if it's possible to implement
for a specific data structure. Consider the following class:

// A class that stores results as they're generated:
struct ResultsMgr
{
void addResults(const Results & results) { /* ... */ }
void exportResults() { /* ... */ }
};

And let's say it's used like so:

ResultsMgr mgr;
for(AnalysisItr i = analyses.begin(); i != analyses.end(); ++i)
{
mgr.addResults(i->execute());
}
mgr.exportResults();

In the above example, the ResultsMgr is only used (1) when results are
ready to be added to it, and (2) at the end, to export the results
somehow.

Since the ResultsMgr is only used in a few places, it might make sense
to let it be store on in a random-access file. Is there any way to do
this in c++? Has anyone written such a utility?

You might be looking for some form of serialization, which allows you
to store or transmit objects for later reconstruction. See these FAQs:

http://www.parashift.com/c++-faq-lite/serialization.html

and check out Boost.Serialization:

http://boost.org/libs/serialization/doc/index.html

Cheers! --M