In-place memory manager, mmap (was: Fastest way to store ints andfloats on disk)

[castironpi]

Hi,

I've got an "in-place" memory manager that uses a disk-backed memory-
mapped buffer. Among its possibilities are: storing variable-length
strings and structures for persistence and interprocess communication
with mmap.

It allocates segments of a generic buffer by length and returns an
offset to the reserved block, which can then be used with struct to
pack values to store. The data structure is adapted from the GNU PAVL
binary tree.

Allocated blocks can be cast to ctypes.Structure instances using some
monkey patching, which is optional.

Want to open-source it. Any interest?

 

[Kris Kennaway]

Hi,

I've got an "in-place" memory manager that uses a disk-backed memory-
mapped buffer. Among its possibilities are: storing variable-length
strings and structures for persistence and interprocess communication
with mmap.

It allocates segments of a generic buffer by length and returns an
offset to the reserved block, which can then be used with struct to
pack values to store. The data structure is adapted from the GNU PAVL
binary tree.

Allocated blocks can be cast to ctypes.Structure instances using some
monkey patching, which is optional.

Want to open-source it. Any interest?

Just do it. That way users can come along later.

Kris

 

[castironpi]

Just do it.  That way users can come along later.

Kris

How? My website? Google Code? Too small for source forge, I think.

 

[Mohamed Yousef]

source-forge places no limits and is a very good place (specific
project site / releases / usage counters)

 

[Kris Kennaway]

How? My website? Google Code? Too small for source forge, I think.

Any of those 3 would work fine, but the last two are probably better
(sourceforge hosts plenty of tiny projects) if you don't want to have to
manage your server and related infrastructure yourself.

Kris

 

[castironpi]

Any of those 3 would work fine, but the last two are probably better
(sourceforge hosts plenty of tiny projects) if you don't want to have to
manage your server and related infrastructure yourself.

Kris

I decided on Google Code. The demo creates 'mappedtree.dat' at 3000
bytes, and allocates or frees memory blocks in it at random. There is
a insert-stress test, and a concurrent read-write test too. Tested on
WinXP with Python 2.5. Have a look.

http://code.google.com/p/pymmapstruct/source/browse/trunk/allocbuf.py