Confining std::map to a region in memory

[Michael Mol]

What would be involved if one wanted to confine all of the data
associated with std::map to a specific region in memory? The specific
example I had in mind when I pondered this earlier today was as a way
of organizing shared-memory IPC. For the production project, I'm
going with a simpler approach, but it still seemed like an interesting
problem.

I have a vague understanding that this would require a custom
allocator, that the allocator's bookkeeping information would need to
reside within the shared memory region as well, that the types serving
as keys and data could not allocate their own memory (or would need to
use the same custom allocator) and that those types could not have
virtual functions (A function pointer becomes meaningless once you
find yourself in a different address space.).

There's also the obvious requirement of adequate synchronization and
class member padding.

Am I missing anything?

 

[Maxim Yegorushkin]

What would be involved if one wanted to confine all of the data
associated with std::map to a specific region in memory?  The specific
example I had in mind when I pondered this earlier today was as a way
of organizing shared-memory IPC.  For the production project, I'm
going with a simpler approach, but it still seemed like an interesting
problem.

You may like to take a look at boost interprocess library for details:
http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess/allocators_containers.html

I have a vague understanding that this would require a custom
allocator, that the allocator's bookkeeping information would need to
reside within the shared memory region as well, that the types serving
as keys and data could not allocate their own memory (or would need to
use the same custom allocator) and that those types could not have
virtual functions (A function pointer becomes meaningless once you
find yourself in a different address space.).

If you are using std::map, than its binary tree nodes contain pointers
to parent, left and right. For these pointers to make sense in another
process, the shared memory must be mapped at the same virtual
addresses in the communicating processes.

There's also the obvious requirement of adequate synchronization

Use process shared mutexes.

and class member padding.

Use placement new to place your objects into the shared memory.

 

[Michael Mol]

You may like to take a look at boost interprocess library for details:http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess/allocators...

Very interesting. I'll be poking through that page over the course of
the day.

If you are using std::map, than its binary tree nodes contain pointers
to parent, left and right. For these pointers to make sense in another
process, the shared memory must be mapped at the same virtual
addresses in the communicating processes.

Point. Is it possible to use instruct std::map to use relative
pointers for parent, left and right?

Use process shared mutexes.


Use placement new to place your objects into the shared memory.

That lets you control where the class is constructed, but doesn't
guarantee that both sides of the IPC channel are going to assume the
same distance between a char member and a subsequent double.

 

[Maxim Yegorushkin]

Very interesting.  I'll be poking through that page over the course of
the day.



Point.  Is it possible to use instruct std::map to use relative
pointers for parent, left and right?

You'll need to copy std::map code from your standard library and
modify it to use offsets instead of pointers.

It looks that they did something like that in boost: #include <boost/
interprocess/containers/map.hpp>

That lets you control where the class is constructed, but doesn't
guarantee that both sides of the IPC channel are going to assume the
same distance between a char member and a subsequent double.

Your C++ compilers must use the same C++ ABI (recent gcc and intel for
linux do).

Otherwise you'll need to define your process shared data structures
using PODs (C-compatible structures). C ABI is the same for all
compilers for the same platform.

 

[DerTopper]

What would be involved if one wanted to confine all of the data
associated with std::map to a specific region in memory?  The specific
example I had in mind when I pondered this earlier today was as a way
of organizing shared-memory IPC.  For the production project, I'm
going with a simpler approach, but it still seemed like an interesting
problem.

I have a vague understanding that this would require a custom
allocator, that the allocator's bookkeeping information would need to
reside within the shared memory region as well, that the types serving
as keys and data could not allocate their own memory (or would need to
use the same custom allocator) and that those types could not have
virtual functions (A function pointer becomes meaningless once you
find yourself in a different address space.).

There's also the obvious requirement of adequate synchronization and
class member padding.

Am I missing anything?

Just in case you were developing for Win32: Have you considered using
the facilities of COM? Of course, this won't be as efficient as your
original approach because you wouldn't be working on shared memory,
but since you need to use some inter-process locking-mechanism anyway,
the additional performance penalty of COM may not be too bad.

Regards,
Stuart