Need help extending stl list.

J

JustSomeGuy

I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....

TIA.
B.
 
J

Jeff Flinn

JustSomeGuy said:
I have an stl list that grows to be too huge to maintain effectivly in
memory.
There are elements within the list that could be stored on disk until
accessed.

However I don't want to expose this to the application class.

How can I extent the stl list to write some elements to disk when they
are put in the list
and read them from disk when they are read from the list.

All the list methods need to still work like iterators and sort
methods....

See GoF Proxy Pattern.

Jeff F
 
J

JustSomeGuy

Jeff said:
See GoF Proxy Pattern.

Jeff F

Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...
 
J

Jeff Flinn

Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...

"Design Patterns" by Gamma, Helm, Johnson, Vlissides ( the Gang of Four )

I assume you are specifying a list because you need to do some constant time
insertions at arbitrary locations within the list. If not provide some more
detail, and there may be better approaches. You wouldn't extend std::list,
in
that you wouldn't derive from it. You don't provide much detail. But if it
is
costly to construct each item, then you would create a lightweight proxy
class
that represents each item.

class proxy
{
some_handle mHdl;
bool mLoaded;

void LoadIt(){ if( !mLoaded ){ /* load from disk using the handle
data*/ } }

public:

proxy( some_hdl aHdl ):mHdl(aHdl),mLoaded(false){}

void some_method(){ LoadIt(); /* some other actions */ }
}

typedef std::list<proxy> tProxyList;

some_func()
{
tProxyList lPs;

lPs.push_back( 1 );
lPs.push_back( 2 );

...

lPs.back().some_method(); // only this one is actually loaded

}

Jeff F
 
J

John Harrison

JustSomeGuy said:
Sorry I don't have this book... Nor do I know what you mean... But thanks
I'll try to figure out what you are trying to tell me...

I don't have the book to hand either but I think what Jeff is saying is that
you shouldn't be trying to extend std::list. Instead you should use a
regular list but in that list you should store objects which are proxies for
your real objects.

The real objects maybe on disk, they may be in memory, the proxies will
handle that. But that is where you should be adding the smart behaviour to
your code, in the proxy objects not in the list.

For instance the proxy class could maintain a pool of frequently accessed
objects, those would be stored in memory and the remainder on disk. Some
algorithm could determine when an object counts as frequently accessed and
is added to the pool (and when a less frequently accessed object is removed
from the pool).

john


john
 
J

JustSomeGuy

Jeff Flinn said:
"Design Patterns" by Gamma, Helm, Johnson, Vlissides ( the Gang of Four )

I assume you are specifying a list because you need to do some constant time
insertions at arbitrary locations within the list. If not provide some more
detail, and there may be better approaches. You wouldn't extend std::list,
in
that you wouldn't derive from it. You don't provide much detail. But if it
is
costly to construct each item, then you would create a lightweight proxy
class
that represents each item.

class proxy
{
some_handle mHdl;
bool mLoaded;

void LoadIt(){ if( !mLoaded ){ /* load from disk using the handle
data*/ } }

public:

proxy( some_hdl aHdl ):mHdl(aHdl),mLoaded(false){}

void some_method(){ LoadIt(); /* some other actions */ }
}

typedef std::list<proxy> tProxyList;

some_func()
{
tProxyList lPs;

lPs.push_back( 1 );
lPs.push_back( 2 );

...

lPs.back().some_method(); // only this one is actually loaded

}

Jeff F

I think you are on the right track and understand my problem.
My problem is that some objects are simply too large to be maintained in
memory
effectivly.. yet the requirements that they be orginized as a list is sound.
I was thinking that over-riding operator= for the class of objects in the
list might
be an easy way to impliment this.. However I guess there is no way to tell
if I actually
need the data or the handle to it. Adding a 'get' method is what I believe
you are suggesting,
but that changes the class interface doesn't it?
 
J

JustSomeGuy

John said:
I don't have the book to hand either but I think what Jeff is saying is that
you shouldn't be trying to extend std::list. Instead you should use a
regular list but in that list you should store objects which are proxies for
your real objects.

The real objects maybe on disk, they may be in memory, the proxies will
handle that. But that is where you should be adding the smart behaviour to
your code, in the proxy objects not in the list.

For instance the proxy class could maintain a pool of frequently accessed
objects, those would be stored in memory and the remainder on disk. Some
algorithm could determine when an object counts as frequently accessed and
is added to the pool (and when a less frequently accessed object is removed
from the pool).

john

john

Sorry my design pattern knowlege is slim, I found the uml representation but
I'm not sure
how this helps me.
However if I understand you, then the list contains instances of the Proxy
class not
instances of the actual class. Is that correct?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top