Question on use of "placement" new

[letz]

Hi,

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.


class ShmManager {

void* operator new(size_t size);
void operator delete(void* p, size_t size);

};

class Foo : public ShmManager
{

int fData1;
Barr fData2[16];

};

In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".

We are using the "placement" new syntax doing:

class Foo1 : public ShmManager
{

int fData1;
Barr fData2[]; // will be extented using "placement" new

};

ShmManager* ptr = ShmManager:perator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();

So that Foo1 object nows gets a dynamic "num" number of elements. This
seems to work, but is this safe to do that? Are we obliged to put the
fData2 fied as the *last* element in the Foo1? Is there any better/
safer manner to implement the same pattern?

Thanks

Stéphane Letz

 

[peter koch]

Hi,

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.

class ShmManager {

        void* operator new(size_t size);
        void operator delete(void* p, size_t size);

};

class Foo : public ShmManager
{

          int fData1;
          Barr fData2[16];

};

In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".

Why must it be flat? You can't do that portably in C++ and there are
potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.

We are using the "placement" new syntax doing:

class Foo1 : public ShmManager
{

          int fData1;
          Barr fData2[];  // will be extented using "placement" new

};

ShmManager* ptr = ShmManager:perator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();

There is no reason to inherit from ShmManager here. This would have
been necessary only in the case where you would be able to new Foo1
directly.

So that Foo1 object nows gets a dynamic "num" number of elements. This
seems to work, but is this safe to do that? Are we obliged to put the
fData2 fied as the *last* element in the Foo1? Is there any better/
safer manner to implement the same pattern?

There are problems:
* your code will have 16 calls to the Barr constructor and this is
only correct when num is 16. To solve this problem, you need to extend
your code.
* The alignment problems mentioned before.
* The nonportability. The code above is not valid C++, and you might
get into troubles for that.

If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.

/Peter

 

[letz]

Hi,

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.

class ShmManager {

        void* operator new(size_t size);
        void operator delete(void* p, size_t size);

class Foo : public ShmManager
{

          int fData1;
          Barr fData2[16];

};

In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".

Why must it be flat?

Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".


You can't do that portably in C++ and there are

potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.

If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.

If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.

The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?

Thanks

Stephane Letz

 

[peter koch]

On 17 Maj, 18:15, (e-mail address removed) wrote:

Hi,
We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
        void* operator new(size_t size);
        void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
          int fData1;
          Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".

Why must it be flat?

Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".

 You can't do that portably in C++ and there are

potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.

If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.

If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.

The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?

I believe now that you are in a situation where you have a piece
memory that is shared between different processes but where the adress
of each segment is different from process to process. In that case, I
believe there is not a better solution than what you proposed.
If the adress is the same in each process, I would use a std::vector
with an allocator allocating from ShmMem.

/Peter

 

[letz]

On 17 Maj, 18:15, (e-mail address removed) wrote:
Hi,
We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
        void* operator new(size_t size);
        void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
          int fData1;
          Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".
Why must it be flat?

Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".

 You can't do that portably in C++ and there are

If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.

The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?

I believe now that you are in a situation where you have a piece
memory that is shared between different processes but where the adress
of each segment is different from process to process. In that case, I
believe there is not a better solution than what you proposed.
If the adress is the same in each process, I would use a std::vector
with an allocator allocating from ShmMem.

/Peter

I finally found this : http://www.devmaster.net/forums/showthread.php?t=11310
that basically does what I what..
.
Any comments?

Thanks

Stephane

 

[Greg Herlihy]

On 17 Maj, 20:17, (e-mail address removed) wrote:

On 17 Maj, 18:15, (e-mail address removed) wrote:
Hi,
We have a class whose objects are to be allocated in shared memory.. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
        void* operator new(size_t size);
        void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
          int fData1;
          Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".
Why must it be flat?
Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".
 You can't do that portably in C++ and there are
potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.
If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.
If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.
The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?

I believe now that you are in a situation where you have a piece
memory that is shared between different processes but where the adress
of each segment is different from process to process. In that case, I
believe there is not a better solution than what you proposed.
If the adress is the same in each process, I would use a std::vector
with an allocator allocating from ShmMem.

/Peter

I finally found this :http://www.devmaster.net/forums/showthread.php?t=11310
that basically does what I what..
.
Any comments?

There would be no problem as far as C++ is concerned, if the program
used placement new() to allocate a char array in shared memory - and
then dedicated the first two (or four) chars in the array to hold
(either in big or little-endian order) the size of the allocation.

Greg

 

[James Kanze]

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.

class ShmManager {
void* operator new(size_t size);
void operator delete(void* p, size_t size);
};

class Foo : public ShmManager
{
int fData1;
Barr fData2[16];
};

In the previous Foo example, the size of fData2 array is known
at compilation time, but we need to make this size "dynamic",
but keeping the object memory layout "flat".

Formally, it can't be done. Practically, see below. (I'm
assuming the Barr is a POD type. Otherwise, you'll run into any
number of problems.)

We are using the "placement" new syntax doing:

class Foo1 : public ShmManager
{
int fData1;
Barr fData2[]; // will be extented using "placement" new
};

ShmManager* ptr = ShmManager:perator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();

That's legal C99, but not legal C++. In C++, it would cause
some additional problems (e.g. if you inherit from ShmManager).
To date, no one has done the work necessary to solve them, so it
probably won't be adopted into C.

The way I've worked around this in the (distant) past is to
define something like:

class Foo : public ShmManager
{
int fData1 ;
Barr* fData2() { return this + 1 ; }

public:
void* operator new( size_t n, size_t elementCount )
{
return ShmManager:perator new(
n + elementCount * sizeof( Barr ) ) ;
}
} ;

Note, however, that this may create problems with alignment.
In my case, Barr was in fact char, so the problem didn't
occur. If Barr is something more complicated, you'll have to
take additional steps to ensure that sizeof( Foo ) is a multiple
of the alignment needed for Barr.

Note that this requires the client code to use a somewhat
special syntax:

new ( n ) Foo ;

and that it only really works if Barr is a POD (but my
experience is that it's best to stick with POD's in shared
memory anyway).

So that Foo1 object nows gets a dynamic "num" number of
elements. This seems to work, but is this safe to do that?

It's not legal to specify an empty array specifier here, and if
you specify [1], and extend it, you have undefined behavior when
you attempt to access anything but the first element.

Are we obliged to put the fData2 fied as the *last* element in
the Foo1?

Yes. Otherwise, how would the compiler (not knowing n) know how
to find the other elements.

Is there any better/ safer manner to implement the same
pattern?

Even in shared memory, I'd keep variable length arrays separate.
Perhaps using some sort of smart pointer which only stores the
offset from the beginning of shared memory, and uses a global
variable (in the non-shared memory of each process) to calculate
the real address.

 

[Frank Birbacher]

Hi!

I guess with this you meant to say that e.g. std::vector is outlawed.

Well, boost allows vectors, maps, and other stuff in shared memory:
http://www.boost.org/doc/libs/1_35_...process.quick_guide.qg_interprocess_container

Regards, Frank

 

[Ian Collins]

Hi!



Well, boost allows vectors, maps, and other stuff in shared memory:
http://www.boost.org/doc/libs/1_35_...process.quick_guide.qg_interprocess_container

You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.

 

[Frank Birbacher]

Hi!

You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.

Right. My point was: it works, you can use vectors in shared memory. The
OP seems to oppose to this solution. I cannot agree on some fragile
solution using the variable-sized-object-and-int[]-hack (which is
illegal in C++) when there is a cleaner solution (using a vector with
shmem alloc).

Regards, Frank

 

[Ian Collins]

Hi!

You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.

Right. My point was: it works, you can use vectors in shared memory. The
OP seems to oppose to this solution. I cannot agree on some fragile
solution using the variable-sized-object-and-int[]-hack (which is
illegal in C++) when there is a cleaner solution (using a vector with
shmem alloc).

I agree. The only real taboos with hared memory objects are static data
members and virtual functions.

 

[peter koch]

You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.

This requires the memory pool to have the same adress in every
process. Apparantly this is not the case for the OP.

/Peter