Question on use of "placement" new

Discussion in 'C++' started by letz@grame.fr, May 17, 2008.

  1. Guest

    Hi,

    We have a class whose objects are to be allocated in shared memory. To
    do that a ShmManager base class is defined so that operator new and
    delete are redefined to allocate segments in shared memory. A typical
    class "Foo" then inherit from ShmManager to have get this behaviour.


    class ShmManager {

    void* operator new(size_t size);
    void operator delete(void* p, size_t size);

    };

    class Foo : public ShmManager
    {

    int fData1;
    Barr fData2[16];

    };

    In the previous Foo example, the size of fData2 array is known at
    compilation time, but we need to make this size "dynamic", but keeping
    the object memory layout "flat".

    We are using the "placement" new syntax doing:

    class Foo1 : public ShmManager
    {

    int fData1;
    Barr fData2[]; // will be extented using "placement" new

    };

    ShmManager* ptr = ShmManager::eek:perator new(sizeof(Foo1) + num *
    sizeof(Barr));
    Foo1* = new(ptr) Foo1();

    So that Foo1 object nows gets a dynamic "num" number of elements. This
    seems to work, but is this safe to do that? Are we obliged to put the
    fData2 fied as the *last* element in the Foo1? Is there any better/
    safer manner to implement the same pattern?

    Thanks

    Stéphane Letz
     
    , May 17, 2008
    #1
    1. Advertising

  2. peter koch Guest

    On 17 Maj, 18:15, wrote:
    > Hi,
    >
    > We have a class whose objects are to be allocated in shared memory. To
    > do that a ShmManager base class is defined so that operator new and
    > delete are redefined to allocate segments in shared memory. A typical
    > class "Foo" then inherit from ShmManager to have get this behaviour.
    >
    > class ShmManager {
    >
    >         void* operator new(size_t size);
    >         void operator delete(void* p, size_t size);
    >
    > };
    >
    > class Foo : public ShmManager
    > {
    >
    >           int fData1;
    >           Barr fData2[16];
    >
    > };
    >
    > In the previous Foo example, the size of fData2 array is known at
    > compilation time, but we need to make this size "dynamic", but keeping
    > the object memory layout "flat".


    Why must it be flat? You can't do that portably in C++ and there are
    potential problems here - e.g. if Barrs alignment is not compatible
    with that of int.
    I will assume that a solution where fData2 is a pointer to memory not
    adjacent to fData1.
    >
    > We are using the "placement" new syntax doing:
    >
    > class Foo1 : public ShmManager
    > {
    >
    >           int fData1;
    >           Barr fData2[];  // will be extented using "placement" new
    >
    > };
    >
    > ShmManager* ptr = ShmManager::eek:perator new(sizeof(Foo1) + num *
    > sizeof(Barr));
    > Foo1* = new(ptr) Foo1();


    There is no reason to inherit from ShmManager here. This would have
    been necessary only in the case where you would be able to new Foo1
    directly.

    >
    > So that Foo1 object nows gets a dynamic "num" number of elements. This
    > seems to work, but is this safe to do that? Are we obliged to put the
    > fData2 fied as the *last* element in the Foo1? Is there any better/
    > safer manner to implement the same pattern?


    There are problems:
    * your code will have 16 calls to the Barr constructor and this is
    only correct when num is 16. To solve this problem, you need to extend
    your code.
    * The alignment problems mentioned before.
    * The nonportability. The code above is not valid C++, and you might
    get into troubles for that.

    If the requirement was that only the fData2 elements would have to be
    contigious, much better solutions exist. Your solution is nonportable
    and fragile, but sometimes you just might have to live with that.

    /Peter
     
    peter koch, May 17, 2008
    #2
    1. Advertising

  3. Guest

    On 17 mai, 19:49, peter koch <> wrote:
    > On 17 Maj, 18:15, wrote:
    >
    >
    >
    > > Hi,

    >
    > > We have a class whose objects are to be allocated in shared memory. To
    > > do that a ShmManager base class is defined so that operator new and
    > > delete are redefined to allocate segments in shared memory. A typical
    > > class "Foo" then inherit from ShmManager to have get this behaviour.

    >
    > > class ShmManager {

    >
    > >         void* operator new(size_t size);
    > >         void operator delete(void* p, size_t size);

    >
    > > };

    >
    > > class Foo : public ShmManager
    > > {

    >
    > >           int fData1;
    > >           Barr fData2[16];

    >
    > > };

    >
    > > In the previous Foo example, the size of fData2 array is known at
    > > compilation time, but we need to make this size "dynamic", but keeping
    > > the object memory layout "flat".

    >
    > Why must it be flat?


    Remerber the object lives in shared mem, then having it "flat" greatly
    simplify it's use: any process that needs it gets the base object
    pointer and can access any field "directly".


    You can't do that portably in C++ and there are
    > potential problems here - e.g. if Barrs alignment is not compatible
    > with that of int.
    > I will assume that a solution where fData2 is a pointer to memory not
    > adjacent to fData1.


    If we switch to a solution where fData2 a allocated elsewhere, (that
    is in another shared memory segment in our case) then we have to deal
    with more complex memory access scheme, again because of the shared
    memory issue.
    >
    >
    >
    >
    > If the requirement was that only the fData2 elements would have to be
    > contigious, much better solutions exist. Your solution is nonportable
    > and fragile, but sometimes you just might have to live with that.
    >


    The whole point of the initial question was: how to allow to have a
    dynamic sized array as a field *and* keep the "flat" acess property.
    What kind of "much better solutions exist" are you thinking of?

    Thanks

    Stephane Letz
     
    , May 17, 2008
    #3
  4. peter koch Guest

    On 17 Maj, 20:17, wrote:
    > On 17 mai, 19:49, peter koch <> wrote:
    >
    >
    >
    >
    >
    > > On 17 Maj, 18:15, wrote:

    >
    > > > Hi,

    >
    > > > We have a class whose objects are to be allocated in shared memory. To
    > > > do that a ShmManager base class is defined so that operator new and
    > > > delete are redefined to allocate segments in shared memory. A typical
    > > > class "Foo" then inherit from ShmManager to have get this behaviour.

    >
    > > > class ShmManager {

    >
    > > >         void* operator new(size_t size);
    > > >         void operator delete(void* p, size_t size);

    >
    > > > };

    >
    > > > class Foo : public ShmManager
    > > > {

    >
    > > >           int fData1;
    > > >           Barr fData2[16];

    >
    > > > };

    >
    > > > In the previous Foo example, the size of fData2 array is known at
    > > > compilation time, but we need to make this size "dynamic", but keeping
    > > > the object memory layout "flat".

    >
    > > Why must it be flat?

    >
    > Remerber the object lives in shared mem, then having it "flat" greatly
    > simplify it's use: any process that needs it gets the base object
    > pointer and can access any field "directly".
    >
    >  You can't do that portably in C++ and there are
    >
    > > potential problems here - e.g. if Barrs alignment is not compatible
    > > with that of int.
    > > I will assume that a solution where fData2 is a pointer to memory not
    > > adjacent to fData1.

    >
    > If we switch to a solution where fData2 a allocated elsewhere, (that
    > is in another shared memory segment in our case) then we have to deal
    > with more complex memory access scheme, again because of the shared
    > memory issue.
    >
    >
    >
    > > If the requirement was that only the fData2 elements would have to be
    > > contigious, much better solutions exist. Your solution is nonportable
    > > and fragile, but sometimes you just might have to live with that.

    >
    > The whole point of the initial question was: how to allow to have a
    > dynamic sized array as a field *and* keep the "flat" acess property.
    > What kind of "much better solutions exist" are you thinking of?
    >

    I believe now that you are in a situation where you have a piece
    memory that is shared between different processes but where the adress
    of each segment is different from process to process. In that case, I
    believe there is not a better solution than what you proposed.
    If the adress is the same in each process, I would use a std::vector
    with an allocator allocating from ShmMem.

    /Peter
     
    peter koch, May 17, 2008
    #4
  5. Guest

    On 17 mai, 20:31, peter koch <> wrote:
    > On 17 Maj, 20:17, wrote:
    >
    > > On 17 mai, 19:49, peter koch <> wrote:

    >
    > > > On 17 Maj, 18:15, wrote:

    >
    > > > > Hi,

    >
    > > > > We have a class whose objects are to be allocated in shared memory. To
    > > > > do that a ShmManager base class is defined so that operator new and
    > > > > delete are redefined to allocate segments in shared memory. A typical
    > > > > class "Foo" then inherit from ShmManager to have get this behaviour.

    >
    > > > > class ShmManager {

    >
    > > > >         void* operator new(size_t size);
    > > > >         void operator delete(void* p, size_t size);

    >
    > > > > };

    >
    > > > > class Foo : public ShmManager
    > > > > {

    >
    > > > >           int fData1;
    > > > >           Barr fData2[16];

    >
    > > > > };

    >
    > > > > In the previous Foo example, the size of fData2 array is known at
    > > > > compilation time, but we need to make this size "dynamic", but keeping
    > > > > the object memory layout "flat".

    >
    > > > Why must it be flat?

    >
    > > Remerber the object lives in shared mem, then having it "flat" greatly
    > > simplify it's use: any process that needs it gets the base object
    > > pointer and can access any field "directly".

    >
    > >  You can't do that portably in C++ and there are

    >
    > > > potential problems here - e.g. if Barrs alignment is not compatible
    > > > with that of int.
    > > > I will assume that a solution where fData2 is a pointer to memory not
    > > > adjacent to fData1.

    >
    > > If we switch to a solution where fData2 a allocated elsewhere, (that
    > > is in another shared memory segment in our case) then we have to deal
    > > with more complex memory access scheme, again because of the shared
    > > memory issue.

    >
    > > > If the requirement was that only the fData2 elements would have to be
    > > > contigious, much better solutions exist. Your solution is nonportable
    > > > and fragile, but sometimes you just might have to live with that.

    >
    > > The whole point of the initial question was: how to allow to have a
    > > dynamic sized array as a field *and* keep the "flat" acess property.
    > > What kind of "much better solutions exist" are you thinking of?

    >
    > I believe now that you are in a situation where you have a piece
    > memory that is shared between different processes but where the adress
    > of each segment is different from process to process. In that case, I
    > believe there is not a better solution than what you proposed.
    > If the adress is the same in each process, I would use a std::vector
    > with an allocator allocating from ShmMem.
    >
    > /Peter


    I finally found this : http://www.devmaster.net/forums/showthread.php?t=11310
    that basically does what I what..
    .
    Any comments?

    Thanks

    Stephane
     
    , May 17, 2008
    #5
  6. Greg Herlihy Guest

    On May 17, 12:28 pm, wrote:
    > On 17 mai, 20:31, peter koch <> wrote:
    >
    >
    >
    > > On 17 Maj, 20:17, wrote:

    >
    > > > On 17 mai, 19:49, peter koch <> wrote:

    >
    > > > > On 17 Maj, 18:15, wrote:

    >
    > > > > > Hi,

    >
    > > > > > We have a class whose objects are to be allocated in shared memory.. To
    > > > > > do that a ShmManager base class is defined so that operator new and
    > > > > > delete are redefined to allocate segments in shared memory. A typical
    > > > > > class "Foo" then inherit from ShmManager to have get this behaviour.

    >
    > > > > > class ShmManager {

    >
    > > > > >         void* operator new(size_t size);
    > > > > >         void operator delete(void* p, size_t size);

    >
    > > > > > };

    >
    > > > > > class Foo : public ShmManager
    > > > > > {

    >
    > > > > >           int fData1;
    > > > > >           Barr fData2[16];

    >
    > > > > > };

    >
    > > > > > In the previous Foo example, the size of fData2 array is known at
    > > > > > compilation time, but we need to make this size "dynamic", but keeping
    > > > > > the object memory layout "flat".

    >
    > > > > Why must it be flat?

    >
    > > > Remerber the object lives in shared mem, then having it "flat" greatly
    > > > simplify it's use: any process that needs it gets the base object
    > > > pointer and can access any field "directly".

    >
    > > >  You can't do that portably in C++ and there are

    >
    > > > > potential problems here - e.g. if Barrs alignment is not compatible
    > > > > with that of int.
    > > > > I will assume that a solution where fData2 is a pointer to memory not
    > > > > adjacent to fData1.

    >
    > > > If we switch to a solution where fData2 a allocated elsewhere, (that
    > > > is in another shared memory segment in our case) then we have to deal
    > > > with more complex memory access scheme, again because of the shared
    > > > memory issue.

    >
    > > > > If the requirement was that only the fData2 elements would have to be
    > > > > contigious, much better solutions exist. Your solution is nonportable
    > > > > and fragile, but sometimes you just might have to live with that.

    >
    > > > The whole point of the initial question was: how to allow to have a
    > > > dynamic sized array as a field *and* keep the "flat" acess property.
    > > > What kind of "much better solutions exist" are you thinking of?

    >
    > > I believe now that you are in a situation where you have a piece
    > > memory that is shared between different processes but where the adress
    > > of each segment is different from process to process. In that case, I
    > > believe there is not a better solution than what you proposed.
    > > If the adress is the same in each process, I would use a std::vector
    > > with an allocator allocating from ShmMem.

    >
    > > /Peter

    >
    > I finally found this :http://www.devmaster.net/forums/showthread.php?t=11310
    > that basically does what I what..
    > .
    > Any comments?


    There would be no problem as far as C++ is concerned, if the program
    used placement new() to allocate a char array in shared memory - and
    then dedicated the first two (or four) chars in the array to hold
    (either in big or little-endian order) the size of the allocation.

    Greg
     
    Greg Herlihy, May 18, 2008
    #6
  7. James Kanze Guest

    On 17 mai, 18:15, wrote:
    > We have a class whose objects are to be allocated in shared memory. To
    > do that a ShmManager base class is defined so that operator new and
    > delete are redefined to allocate segments in shared memory. A typical
    > class "Foo" then inherit from ShmManager to have get this behaviour.


    > class ShmManager {
    > void* operator new(size_t size);
    > void operator delete(void* p, size_t size);
    > };


    > class Foo : public ShmManager
    > {
    > int fData1;
    > Barr fData2[16];
    > };


    > In the previous Foo example, the size of fData2 array is known
    > at compilation time, but we need to make this size "dynamic",
    > but keeping the object memory layout "flat".


    Formally, it can't be done. Practically, see below. (I'm
    assuming the Barr is a POD type. Otherwise, you'll run into any
    number of problems.)

    > We are using the "placement" new syntax doing:


    > class Foo1 : public ShmManager
    > {
    > int fData1;
    > Barr fData2[]; // will be extented using "placement" new
    > };


    > ShmManager* ptr = ShmManager::eek:perator new(sizeof(Foo1) + num *
    > sizeof(Barr));
    > Foo1* = new(ptr) Foo1();


    That's legal C99, but not legal C++. In C++, it would cause
    some additional problems (e.g. if you inherit from ShmManager).
    To date, no one has done the work necessary to solve them, so it
    probably won't be adopted into C.

    The way I've worked around this in the (distant) past is to
    define something like:

    class Foo : public ShmManager
    {
    int fData1 ;
    Barr* fData2() { return this + 1 ; }

    public:
    void* operator new( size_t n, size_t elementCount )
    {
    return ShmManager::eek:perator new(
    n + elementCount * sizeof( Barr ) ) ;
    }
    } ;

    Note, however, that this may create problems with alignment.
    In my case, Barr was in fact char, so the problem didn't
    occur. If Barr is something more complicated, you'll have to
    take additional steps to ensure that sizeof( Foo ) is a multiple
    of the alignment needed for Barr.

    Note that this requires the client code to use a somewhat
    special syntax:

    new ( n ) Foo ;

    and that it only really works if Barr is a POD (but my
    experience is that it's best to stick with POD's in shared
    memory anyway).

    > So that Foo1 object nows gets a dynamic "num" number of
    > elements. This seems to work, but is this safe to do that?


    It's not legal to specify an empty array specifier here, and if
    you specify [1], and extend it, you have undefined behavior when
    you attempt to access anything but the first element.

    > Are we obliged to put the fData2 fied as the *last* element in
    > the Foo1?


    Yes. Otherwise, how would the compiler (not knowing n) know how
    to find the other elements.

    > Is there any better/ safer manner to implement the same
    > pattern?


    Even in shared memory, I'd keep variable length arrays separate.
    Perhaps using some sort of smart pointer which only stores the
    offset from the beginning of shared memory, and uses a global
    variable (in the non-shared memory of each process) to calculate
    the real address.

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
     
    James Kanze, May 18, 2008
    #7
  8. Frank Birbacher, May 18, 2008
    #8
  9. Ian Collins Guest

    Frank Birbacher wrote:
    > Hi!
    >
    > Paavo Helde schrieb:
    >> I guess with this you meant to say that e.g. std::vector is outlawed.

    >
    > Well, boost allows vectors, maps, and other stuff in shared memory:
    > http://www.boost.org/doc/libs/1_35_...process.quick_guide.qg_interprocess_container
    >

    You don't need boost for that, all you have to do is provide an
    appropriate allocator to manage a memory pool in shared memory.

    --
    Ian Collins.
     
    Ian Collins, May 19, 2008
    #9
  10. Hi!

    Ian Collins schrieb:
    > You don't need boost for that, all you have to do is provide an
    > appropriate allocator to manage a memory pool in shared memory.


    Right. My point was: it works, you can use vectors in shared memory. The
    OP seems to oppose to this solution. I cannot agree on some fragile
    solution using the variable-sized-object-and-int[]-hack (which is
    illegal in C++) when there is a cleaner solution (using a vector with
    shmem alloc).

    Regards, Frank
     
    Frank Birbacher, May 19, 2008
    #10
  11. Ian Collins Guest

    Frank Birbacher wrote:
    > Hi!
    >
    > Ian Collins schrieb:
    >> You don't need boost for that, all you have to do is provide an
    >> appropriate allocator to manage a memory pool in shared memory.

    >
    > Right. My point was: it works, you can use vectors in shared memory. The
    > OP seems to oppose to this solution. I cannot agree on some fragile
    > solution using the variable-sized-object-and-int[]-hack (which is
    > illegal in C++) when there is a cleaner solution (using a vector with
    > shmem alloc).
    >

    I agree. The only real taboos with hared memory objects are static data
    members and virtual functions.

    --
    Ian Collins.
     
    Ian Collins, May 19, 2008
    #11
  12. peter koch Guest

    On 19 Maj, 01:01, Ian Collins <> wrote:
    > Frank Birbacher wrote:
    > > Hi!

    >
    > > Paavo Helde schrieb:
    > >> I guess with this you meant to say that e.g. std::vector is outlawed.

    >
    > > Well, boost allows vectors, maps, and other stuff in shared memory:
    > >http://www.boost.org/doc/libs/1_35_0/doc/html/interprocess/quick_guid...

    >
    > You don't need boost for that, all you have to do is provide an
    > appropriate allocator to manage a memory pool in shared memory.


    This requires the memory pool to have the same adress in every
    process. Apparantly this is not the case for the OP.

    /Peter
     
    peter koch, May 19, 2008
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. gcc
    Replies:
    11
    Views:
    15,394
    David Dorward
    Jun 10, 2004
  2. Ioannis Vranos

    Placement operator new/delete question

    Ioannis Vranos, Oct 16, 2004, in forum: C++
    Replies:
    11
    Views:
    5,704
    Ioannis Vranos
    Oct 16, 2004
  3. Mark P
    Replies:
    6
    Views:
    824
    James Dennett
    Apr 27, 2005
  4. Replies:
    13
    Views:
    913
    Alf P. Steinbach
    Jul 7, 2006
  5. Michael Tsang
    Replies:
    8
    Views:
    682
    Alf P. Steinbach
    Dec 14, 2009
Loading...

Share This Page