STL and shared memory

Discussion in 'C++' started by Michael Schuler, Jan 12, 2004.

  1. The use of STL in shared memory poses a real problem since
    (non-smart) pointers are not allowed there.

    Is there any solution for containers in shared memory using
    smart pointers? Where can I find templates?
    Michael Schuler, Jan 12, 2004
    #1
    1. Advertising

  2. Michael Schuler

    void Guest

    U¿ytkownik Michael Schuler napisa³, On 2004-01-12 15:01:

    > The use of STL in shared memory poses a real problem since
    > (non-smart) pointers are not allowed there.
    >
    > Is there any solution for containers in shared memory using
    > smart pointers? Where can I find templates?
    >

    AFAIR to use shared memory you should use boost library you can find it at:
    http://www.boost.org/.
    Documentation:
    http://www.boost.org/libs/smart_ptr/smart_ptr.htm

    Best Regards
    Darek Ostolski
    --
    ERROR 1164 HOW IN THE HELL DID YOU GET HERE
    void, Jan 12, 2004
    #2
    1. Advertising

  3. Hello,

    take a look to <http://tplusplus.sourceforge.net/> resp. at its source code. It uses special
    allocators to place the contents of STL containers into shared memory. It's been presented
    at OOPSLA 2003.

    Cheers,
    Philipp.


    "Michael Schuler" <> wrote in message news:btu95r$25f$-siemens.com...
    > The use of STL in shared memory poses a real problem since
    > (non-smart) pointers are not allowed there.
    >
    > Is there any solution for containers in shared memory using
    > smart pointers? Where can I find templates?
    >
    Philipp Bachmann, Jan 12, 2004
    #3
  4. Michael Schuler

    tom_usenet Guest

    On Mon, 12 Jan 2004 15:01:07 +0100, Michael Schuler
    <> wrote:

    >The use of STL in shared memory poses a real problem since
    >(non-smart) pointers are not allowed there.
    >
    >Is there any solution for containers in shared memory using
    >smart pointers? Where can I find templates?


    If you can allocate your shared memory at a fixed address in each
    process' address space, then you just need to write a custom allocator
    to allocate from that memory. But if you can't allocate at a fixed
    address, then you have to write a complex allocator that uses "offset"
    smart pointers (and references!) that store an offset into the shared
    memory rather than a pure pointer, and possibly a "shared memory ID"
    or similar if you want to have more than one shared memory area in any
    process. This is hard to do in a reliable way since C++ doesn't really
    support smart references.

    Tom

    C++ FAQ: http://www.parashift.com/c -faq-lite/
    C FAQ: http://www.eskimo.com/~scs/C-faq/top.html
    tom_usenet, Jan 12, 2004
    #4
  5. Thanks Tom,

    but to my knowledge if I use an allocator, only the items are in shared
    memory not the container itself. In my case teh container also must lie
    in shared memory to survive a process failure :-(

    tom_usenet wrote:
    > On Mon, 12 Jan 2004 15:01:07 +0100, Michael Schuler
    > <> wrote:
    >
    >
    >>The use of STL in shared memory poses a real problem since
    >>(non-smart) pointers are not allowed there.
    >>
    >>Is there any solution for containers in shared memory using
    >>smart pointers? Where can I find templates?

    >
    >
    > If you can allocate your shared memory at a fixed address in each
    > process' address space, then you just need to write a custom allocator
    > to allocate from that memory. But if you can't allocate at a fixed
    > address, then you have to write a complex allocator that uses "offset"
    > smart pointers (and references!) that store an offset into the shared
    > memory rather than a pure pointer, and possibly a "shared memory ID"
    > or similar if you want to have more than one shared memory area in any
    > process. This is hard to do in a reliable way since C++ doesn't really
    > support smart references.
    >
    > Tom
    >
    > C++ FAQ: http://www.parashift.com/c -faq-lite/
    > C FAQ: http://www.eskimo.com/~scs/C-faq/top.html
    Michael Schuler, Jan 12, 2004
    #5
  6. Hello,

    that's true, but imposes no problem at all, because there's "placement new",
    i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
    you can write sth like

    ...
    typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
    shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|0700);
    void *const p=(shmIntMap *)shmat(shm,NULL,0);
    shmIntMap *m=new(p) shmIntMap;
    ...

    Then, don't forget to manually call the destructor afterwards.

    Now, you might ask, o.k., now both the map and the ints contained within the map
    go into the shared memory - but what's about the linked structure that establishes
    the tree the map internally consists of, i.e. that glues together the container and
    its elements? The answer is the "rebind<>" member template class a conforming
    allocator has to have.

    Cheers,
    Philipp.

    > but to my knowledge if I use an allocator, only the items are in shared
    > memory not the container itself. In my case teh container also must lie
    > in shared memory to survive a process failure :-(
    >
    > >>Is there any solution for containers in shared memory using
    > >>smart pointers? Where can I find templates?

    > >
    > > If you can allocate your shared memory at a fixed address in each
    > > process' address space, then you just need to write a custom allocator
    > > to allocate from that memory. But if you can't allocate at a fixed
    > > address, then you have to write a complex allocator that uses "offset"
    > > smart pointers (and references!) that store an offset into the shared
    > > memory rather than a pure pointer, and possibly a "shared memory ID"
    > > or similar if you want to have more than one shared memory area in any
    > > process. This is hard to do in a reliable way since C++ doesn't really
    > > support smart references.
    Philipp Bachmann, Jan 12, 2004
    #6
  7. "void" <chq@nie_spamuj.wp.pl> wrote in message
    news:29333-1073917569@213.17.192.132...
    > U¿ytkownik Michael Schuler napisa³, On 2004-01-12 15:01:
    >
    > > The use of STL in shared memory poses a real problem since
    > > (non-smart) pointers are not allowed there.
    > >
    > > Is there any solution for containers in shared memory using
    > > smart pointers? Where can I find templates?
    > >

    > AFAIR to use shared memory you should use boost library you can find

    it at:
    > http://www.boost.org/.
    > Documentation:
    > http://www.boost.org/libs/smart_ptr/smart_ptr.htm


    Boost has excellent smart pointers, but does not provide any help with
    accessing shared memory.

    Jonathan
    Jonathan Turkanis, Jan 12, 2004
    #7
  8. Thanks Philipp, good hint,

    but I still have the problem with pointers inside the container,
    whereas pointers pose problems since the shared memory may have
    different addresses in different processes :-(

    Philipp Bachmann
    > Hello,
    >
    > that's true, but imposes no problem at all, because there's "placement new",
    > i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
    > you can write sth like
    >
    > ...
    > typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
    > shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|0700);
    > void *const p=(shmIntMap *)shmat(shm,NULL,0);
    > shmIntMap *m=new(p) shmIntMap;
    > ...
    >
    > Then, don't forget to manually call the destructor afterwards.
    >
    > Now, you might ask, o.k., now both the map and the ints contained within the map
    > go into the shared memory - but what's about the linked structure that establishes
    > the tree the map internally consists of, i.e. that glues together the container and
    > its elements? The answer is the "rebind<>" member template class a conforming
    > allocator has to have.
    >
    > Cheers,
    > Philipp.
    >
    >
    >>but to my knowledge if I use an allocator, only the items are in shared
    >>memory not the container itself. In my case teh container also must lie
    >>in shared memory to survive a process failure :-(
    >>
    >>
    >>>>Is there any solution for containers in shared memory using
    >>>>smart pointers? Where can I find templates?
    >>>
    >>>If you can allocate your shared memory at a fixed address in each
    >>>process' address space, then you just need to write a custom allocator
    >>>to allocate from that memory. But if you can't allocate at a fixed
    >>>address, then you have to write a complex allocator that uses "offset"
    >>>smart pointers (and references!) that store an offset into the shared
    >>>memory rather than a pure pointer, and possibly a "shared memory ID"
    >>>or similar if you want to have more than one shared memory area in any
    >>>process. This is hard to do in a reliable way since C++ doesn't really
    >>>support smart references.

    >
    >
    >
    Michael Schuler, Jan 13, 2004
    #8
  9. The T++ I've referenced seems to use a shared memory pool of a fixed size,
    if I correctly understood its source code. But check this out, please:
    <http://allocator.sourceforge.net/>. This looks quite promising, doesn't
    it?

    Cheers,
    Philipp.

    > but I still have the problem with pointers inside the container,
    > whereas pointers pose problems since the shared memory may have
    > different addresses in different processes :-(
    > > that's true, but imposes no problem at all, because there's "placement new",
    > > i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
    > > you can write sth like
    > >
    > > ...
    > > typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
    > > shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|0700);
    > > void *const p=(shmIntMap *)shmat(shm,NULL,0);
    > > shmIntMap *m=new(p) shmIntMap;
    > > ...
    > >
    > > Then, don't forget to manually call the destructor afterwards.
    > >
    > > Now, you might ask, o.k., now both the map and the ints contained within the map
    > > go into the shared memory - but what's about the linked structure that establishes
    > > the tree the map internally consists of, i.e. that glues together the container and
    > > its elements? The answer is the "rebind<>" member template class a conforming
    > > allocator has to have.
    > >>but to my knowledge if I use an allocator, only the items are in shared
    > >>memory not the container itself. In my case teh container also must lie
    > >>in shared memory to survive a process failure :-(
    > >>>>Is there any solution for containers in shared memory using
    > >>>>smart pointers? Where can I find templates?
    > >>>
    > >>>If you can allocate your shared memory at a fixed address in each
    > >>>process' address space, then you just need to write a custom allocator
    > >>>to allocate from that memory. But if you can't allocate at a fixed
    > >>>address, then you have to write a complex allocator that uses "offset"
    > >>>smart pointers (and references!) that store an offset into the shared
    > >>>memory rather than a pure pointer, and possibly a "shared memory ID"
    > >>>or similar if you want to have more than one shared memory area in any
    > >>>process. This is hard to do in a reliable way since C++ doesn't really
    > >>>support smart references.
    Philipp Bachmann, Jan 13, 2004
    #9
  10. Michael Schuler

    tom_usenet Guest

    On Tue, 13 Jan 2004 15:05:50 +0100, Michael Schuler
    <> wrote:

    >Thanks Philipp, good hint,
    >
    >but I still have the problem with pointers inside the container,
    >whereas pointers pose problems since the shared memory may have
    >different addresses in different processes :-(


    Solution 1 (easy if your OS supports it):

    Map the shared memory to the same address in every process that
    accesses it. Some OSes allow you to at least attempt this, and if it
    fails you can bail out.


    Solution 2 (a nightmare, not really worth it?):

    Assuming your standard library is up to it (the only one I'm aware of
    is Dinkumware, Metrowerks might be, libstdc++ and STLport aren't), it
    might be possible to write a smart pointer and smart reference that
    work using an offset into the shared memory (which will be the same
    for every process). You'll obviously have to use those classes for any
    other pointers/references you want to put in shared memory. I wrote
    some such class templates a while back, with mixed success.


    Solution 2 (probably best if you can't map to a fixed address):

    Don't use the STL, but your own simpler code. Use explicit offsets,
    not pointers.

    Tom

    C++ FAQ: http://www.parashift.com/c -faq-lite/
    C FAQ: http://www.eskimo.com/~scs/C-faq/top.html
    tom_usenet, Jan 13, 2004
    #10
  11. Thanks. Great answer. I will probably map the shared memory at some
    fixed address.

    tom_usenet wrote:
    > On Tue, 13 Jan 2004 15:05:50 +0100, Michael Schuler
    > <> wrote:
    >
    >
    >>Thanks Philipp, good hint,
    >>
    >>but I still have the problem with pointers inside the container,
    >>whereas pointers pose problems since the shared memory may have
    >>different addresses in different processes :-(

    >
    >
    > Solution 1 (easy if your OS supports it):
    >
    > Map the shared memory to the same address in every process that
    > accesses it. Some OSes allow you to at least attempt this, and if it
    > fails you can bail out.
    >
    >
    > Solution 2 (a nightmare, not really worth it?):
    >
    > Assuming your standard library is up to it (the only one I'm aware of
    > is Dinkumware, Metrowerks might be, libstdc++ and STLport aren't), it
    > might be possible to write a smart pointer and smart reference that
    > work using an offset into the shared memory (which will be the same
    > for every process). You'll obviously have to use those classes for any
    > other pointers/references you want to put in shared memory. I wrote
    > some such class templates a while back, with mixed success.
    >
    >
    > Solution 2 (probably best if you can't map to a fixed address):
    >
    > Don't use the STL, but your own simpler code. Use explicit offsets,
    > not pointers.
    >
    > Tom
    >
    > C++ FAQ: http://www.parashift.com/c -faq-lite/
    > C FAQ: http://www.eskimo.com/~scs/C-faq/top.html
    Michael Schuler, Jan 14, 2004
    #11
  12. Philipp Bachmann
    > The T++ I've referenced seems to use a shared memory pool of a fixed size,
    > if I correctly understood its source code. But check this out, please:
    > <http://allocator.sourceforge.net/>. This looks quite promising, doesn't
    > it?
    >


    Yes, looks great. Thanks for the hint.
    > Cheers,
    > Philipp.
    >
    >
    >>but I still have the problem with pointers inside the container,
    >>whereas pointers pose problems since the shared memory may have
    >>different addresses in different processes :-(
    >>
    >>>that's true, but imposes no problem at all, because there's "placement new",
    >>>i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
    >>>you can write sth like
    >>>
    >>> ...
    >>> typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
    >>> shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|0700);
    >>> void *const p=(shmIntMap *)shmat(shm,NULL,0);
    >>> shmIntMap *m=new(p) shmIntMap;
    >>> ...
    >>>
    >>>Then, don't forget to manually call the destructor afterwards.
    >>>
    >>>Now, you might ask, o.k., now both the map and the ints contained within the map
    >>>go into the shared memory - but what's about the linked structure that establishes
    >>>the tree the map internally consists of, i.e. that glues together the container and
    >>>its elements? The answer is the "rebind<>" member template class a conforming
    >>>allocator has to have.
    >>>
    >>>>but to my knowledge if I use an allocator, only the items are in shared
    >>>>memory not the container itself. In my case teh container also must lie
    >>>>in shared memory to survive a process failure :-(
    >>>>
    >>>>>>Is there any solution for containers in shared memory using
    >>>>>>smart pointers? Where can I find templates?
    >>>>>
    >>>>>If you can allocate your shared memory at a fixed address in each
    >>>>>process' address space, then you just need to write a custom allocator
    >>>>>to allocate from that memory. But if you can't allocate at a fixed
    >>>>>address, then you have to write a complex allocator that uses "offset"
    >>>>>smart pointers (and references!) that store an offset into the shared
    >>>>>memory rather than a pure pointer, and possibly a "shared memory ID"
    >>>>>or similar if you want to have more than one shared memory area in any
    >>>>>process. This is hard to do in a reliable way since C++ doesn't really
    >>>>>support smart references.

    >
    >
    >
    Michael Schuler, Jan 14, 2004
    #12
  13. Michael Schuler

    grbVRCPP

    Joined:
    Mar 13, 2007
    Messages:
    3
    Hi, I'm trying to use STL containers in shared memory across two arbitary processes. Can you please suggest an implementation of C++ STL shared memory allocator which can be used for this? This is in context of this same message thread.
    grbVRCPP, Mar 13, 2007
    #13
  14. Michael Schuler

    grbVRCPP

    Joined:
    Mar 13, 2007
    Messages:
    3
    Dear Sir, May I know which solution did you use? Do you have an implementation of this? Can you please guide me in having C++ STL shared memory allocation for STL containers for two different processes?
    grbVRCPP, Mar 13, 2007
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eric Sasser
    Replies:
    2
    Views:
    1,087
    jlchereau
    Nov 16, 2008
  2. Replies:
    14
    Views:
    3,122
    Gianni Mariani
    May 23, 2005
  3. grbVRCPP
    Replies:
    0
    Views:
    821
    grbVRCPP
    Mar 13, 2007
  4. Me
    Replies:
    17
    Views:
    652
    JohnQ
    Jul 27, 2007
  5. dbtouch
    Replies:
    15
    Views:
    2,084
    Chris M. Thomasson
    Mar 1, 2009
Loading...

Share This Page