Need help extending stl list.

Discussion in 'C++' started by JustSomeGuy, May 28, 2004.

  1. JustSomeGuy

    JustSomeGuy Guest

    I have an stl list that grows to be too huge to maintain effectivly in
    memory.
    There are elements within the list that could be stored on disk until
    accessed.

    However I don't want to expose this to the application class.

    How can I extent the stl list to write some elements to disk when they
    are put in the list
    and read them from disk when they are read from the list.

    All the list methods need to still work like iterators and sort
    methods....

    TIA.
    B.
     
    JustSomeGuy, May 28, 2004
    #1
    1. Advertising

  2. JustSomeGuy

    Jeff Flinn Guest

    "JustSomeGuy" <> wrote in message
    news:...
    > I have an stl list that grows to be too huge to maintain effectivly in
    > memory.
    > There are elements within the list that could be stored on disk until
    > accessed.
    >
    > However I don't want to expose this to the application class.
    >
    > How can I extent the stl list to write some elements to disk when they
    > are put in the list
    > and read them from disk when they are read from the list.
    >
    > All the list methods need to still work like iterators and sort
    > methods....


    See GoF Proxy Pattern.

    Jeff F
     
    Jeff Flinn, May 28, 2004
    #2
    1. Advertising

  3. JustSomeGuy

    JustSomeGuy Guest

    Jeff Flinn wrote:

    > "JustSomeGuy" <> wrote in message
    > news:...
    > > I have an stl list that grows to be too huge to maintain effectivly in
    > > memory.
    > > There are elements within the list that could be stored on disk until
    > > accessed.
    > >
    > > However I don't want to expose this to the application class.
    > >
    > > How can I extent the stl list to write some elements to disk when they
    > > are put in the list
    > > and read them from disk when they are read from the list.
    > >
    > > All the list methods need to still work like iterators and sort
    > > methods....

    >
    > See GoF Proxy Pattern.
    >
    > Jeff F


    Sorry I don't have this book... Nor do I know what you mean... But thanks
    I'll try to figure out what you are trying to tell me...
     
    JustSomeGuy, May 28, 2004
    #3
  4. JustSomeGuy

    Jeff Flinn Guest


    > Jeff Flinn wrote:
    >
    > > "JustSomeGuy" <> wrote in message
    > > news:...
    > > > I have an stl list that grows to be too huge to maintain effectivly in
    > > > memory.
    > > > There are elements within the list that could be stored on disk until
    > > > accessed.
    > > >
    > > > However I don't want to expose this to the application class.
    > > >
    > > > How can I extent the stl list to write some elements to disk when they
    > > > are put in the list
    > > > and read them from disk when they are read from the list.
    > > >
    > > > All the list methods need to still work like iterators and sort
    > > > methods....

    > >
    > > See GoF Proxy Pattern.
    > >
    > > Jeff F

    >
    > Sorry I don't have this book... Nor do I know what you mean... But thanks
    > I'll try to figure out what you are trying to tell me...


    "Design Patterns" by Gamma, Helm, Johnson, Vlissides ( the Gang of Four )

    I assume you are specifying a list because you need to do some constant time
    insertions at arbitrary locations within the list. If not provide some more
    detail, and there may be better approaches. You wouldn't extend std::list,
    in
    that you wouldn't derive from it. You don't provide much detail. But if it
    is
    costly to construct each item, then you would create a lightweight proxy
    class
    that represents each item.

    class proxy
    {
    some_handle mHdl;
    bool mLoaded;

    void LoadIt(){ if( !mLoaded ){ /* load from disk using the handle
    data*/ } }

    public:

    proxy( some_hdl aHdl ):mHdl(aHdl),mLoaded(false){}

    void some_method(){ LoadIt(); /* some other actions */ }
    }

    typedef std::list<proxy> tProxyList;

    some_func()
    {
    tProxyList lPs;

    lPs.push_back( 1 );
    lPs.push_back( 2 );

    ...

    lPs.back().some_method(); // only this one is actually loaded

    }

    Jeff F
     
    Jeff Flinn, May 28, 2004
    #4
  5. "JustSomeGuy" <> wrote in message
    news:...
    > Jeff Flinn wrote:
    >
    > > "JustSomeGuy" <> wrote in message
    > > news:...
    > > > I have an stl list that grows to be too huge to maintain effectivly in
    > > > memory.
    > > > There are elements within the list that could be stored on disk until
    > > > accessed.
    > > >
    > > > However I don't want to expose this to the application class.
    > > >
    > > > How can I extent the stl list to write some elements to disk when they
    > > > are put in the list
    > > > and read them from disk when they are read from the list.
    > > >
    > > > All the list methods need to still work like iterators and sort
    > > > methods....

    > >
    > > See GoF Proxy Pattern.
    > >
    > > Jeff F

    >
    > Sorry I don't have this book... Nor do I know what you mean... But thanks
    > I'll try to figure out what you are trying to tell me...
    >


    I don't have the book to hand either but I think what Jeff is saying is that
    you shouldn't be trying to extend std::list. Instead you should use a
    regular list but in that list you should store objects which are proxies for
    your real objects.

    The real objects maybe on disk, they may be in memory, the proxies will
    handle that. But that is where you should be adding the smart behaviour to
    your code, in the proxy objects not in the list.

    For instance the proxy class could maintain a pool of frequently accessed
    objects, those would be stored in memory and the remainder on disk. Some
    algorithm could determine when an object counts as frequently accessed and
    is added to the pool (and when a less frequently accessed object is removed
    from the pool).

    john


    john
     
    John Harrison, May 28, 2004
    #5
  6. JustSomeGuy

    JustSomeGuy Guest

    "Jeff Flinn" <> wrote in message
    news:c9877b$r7s$...
    >
    > > Jeff Flinn wrote:
    > >
    > > > "JustSomeGuy" <> wrote in message
    > > > news:...
    > > > > I have an stl list that grows to be too huge to maintain effectivly

    in
    > > > > memory.
    > > > > There are elements within the list that could be stored on disk

    until
    > > > > accessed.
    > > > >
    > > > > However I don't want to expose this to the application class.
    > > > >
    > > > > How can I extent the stl list to write some elements to disk when

    they
    > > > > are put in the list
    > > > > and read them from disk when they are read from the list.
    > > > >
    > > > > All the list methods need to still work like iterators and sort
    > > > > methods....
    > > >
    > > > See GoF Proxy Pattern.
    > > >
    > > > Jeff F

    > >
    > > Sorry I don't have this book... Nor do I know what you mean... But

    thanks
    > > I'll try to figure out what you are trying to tell me...

    >
    > "Design Patterns" by Gamma, Helm, Johnson, Vlissides ( the Gang of Four )
    >
    > I assume you are specifying a list because you need to do some constant

    time
    > insertions at arbitrary locations within the list. If not provide some

    more
    > detail, and there may be better approaches. You wouldn't extend std::list,
    > in
    > that you wouldn't derive from it. You don't provide much detail. But if it
    > is
    > costly to construct each item, then you would create a lightweight proxy
    > class
    > that represents each item.
    >
    > class proxy
    > {
    > some_handle mHdl;
    > bool mLoaded;
    >
    > void LoadIt(){ if( !mLoaded ){ /* load from disk using the handle
    > data*/ } }
    >
    > public:
    >
    > proxy( some_hdl aHdl ):mHdl(aHdl),mLoaded(false){}
    >
    > void some_method(){ LoadIt(); /* some other actions */ }
    > }
    >
    > typedef std::list<proxy> tProxyList;
    >
    > some_func()
    > {
    > tProxyList lPs;
    >
    > lPs.push_back( 1 );
    > lPs.push_back( 2 );
    >
    > ...
    >
    > lPs.back().some_method(); // only this one is actually loaded
    >
    > }
    >
    > Jeff F
    >
    >


    I think you are on the right track and understand my problem.
    My problem is that some objects are simply too large to be maintained in
    memory
    effectivly.. yet the requirements that they be orginized as a list is sound.
    I was thinking that over-riding operator= for the class of objects in the
    list might
    be an easy way to impliment this.. However I guess there is no way to tell
    if I actually
    need the data or the handle to it. Adding a 'get' method is what I believe
    you are suggesting,
    but that changes the class interface doesn't it?
     
    JustSomeGuy, May 30, 2004
    #6
  7. JustSomeGuy

    JustSomeGuy Guest

    John Harrison wrote:

    > "JustSomeGuy" <> wrote in message
    > news:...
    > > Jeff Flinn wrote:
    > >
    > > > "JustSomeGuy" <> wrote in message
    > > > news:...
    > > > > I have an stl list that grows to be too huge to maintain effectivly in
    > > > > memory.
    > > > > There are elements within the list that could be stored on disk until
    > > > > accessed.
    > > > >
    > > > > However I don't want to expose this to the application class.
    > > > >
    > > > > How can I extent the stl list to write some elements to disk when they
    > > > > are put in the list
    > > > > and read them from disk when they are read from the list.
    > > > >
    > > > > All the list methods need to still work like iterators and sort
    > > > > methods....
    > > >
    > > > See GoF Proxy Pattern.
    > > >
    > > > Jeff F

    > >
    > > Sorry I don't have this book... Nor do I know what you mean... But thanks
    > > I'll try to figure out what you are trying to tell me...
    > >

    >
    > I don't have the book to hand either but I think what Jeff is saying is that
    > you shouldn't be trying to extend std::list. Instead you should use a
    > regular list but in that list you should store objects which are proxies for
    > your real objects.
    >
    > The real objects maybe on disk, they may be in memory, the proxies will
    > handle that. But that is where you should be adding the smart behaviour to
    > your code, in the proxy objects not in the list.
    >
    > For instance the proxy class could maintain a pool of frequently accessed
    > objects, those would be stored in memory and the remainder on disk. Some
    > algorithm could determine when an object counts as frequently accessed and
    > is added to the pool (and when a less frequently accessed object is removed
    > from the pool).
    >
    > john
    >
    > john


    Sorry my design pattern knowlege is slim, I found the uml representation but
    I'm not sure
    how this helps me.
    However if I understand you, then the list contains instances of the Proxy
    class not
    instances of the actual class. Is that correct?
     
    JustSomeGuy, Jun 1, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Brian O'Brien

    Need help extending stl list.

    Brian O'Brien, May 28, 2004, in forum: C++
    Replies:
    0
    Views:
    321
    Brian O'Brien
    May 28, 2004
  2. Barry Hynes

    Extending stl::list???

    Barry Hynes, Feb 11, 2005, in forum: C++
    Replies:
    2
    Views:
    694
    barman
    Feb 13, 2005
  3. Alan
    Replies:
    4
    Views:
    352
    Victor Bazarov
    Jun 25, 2005
  4. Replies:
    5
    Views:
    507
    Markus Schoder
    Apr 16, 2006
  5. Replies:
    2
    Views:
    397
Loading...

Share This Page