mmap and shared memory

M

Matias Surdi

Suppose I've a process P1, which generates itself a lot of data , for
example 2Mb.
Then, I've a process P2 which must access P1 shared memory and,
probably, modify this data.
To accomplish this, I've been digging around python's mmap module, but I
can't figure how to use it without files.

Could anybody explain me how could this be accomplished? An example will
be very appreciated. Thanks a lot for your help.
 
C

Carl Banks

Suppose I've a process P1, which generates itself a lot of data , for
example 2Mb.
Then, I've a process P2 which must access P1 shared memory and,
probably, modify this data.
To accomplish this, I've been digging around python's mmap module, but I
can't figure how to use it without files.

Could anybody explain me how could this be accomplished? An example will
be very appreciated. Thanks a lot for your help.

In C you can use the mmap call to request a specific physical location
in memory (whence I presume two different processes can mmap anonymous
memory block in the same location), but Python doesn't expose this.

There isn't really as much drawback to using a file as you might
expect, and there are benefits. If you use an anonymous map the OS
could still write that memory to swap space, which, if your swap space
is fixed and limited, might negatively affect performance for the rest
of the system.


Carl Banks
 
T

Tim Roberts

Matias Surdi said:
Suppose I've a process P1, which generates itself a lot of data , for
example 2Mb.
Then, I've a process P2 which must access P1 shared memory and,
probably, modify this data.
To accomplish this, I've been digging around python's mmap module, but I
can't figure how to use it without files.

So, let it use a temporary file. What's the harm? An anonymous mmap
region is still mapped to the swap file. Might as well give it a name.
 
G

greg

Carl said:
In C you can use the mmap call to request a specific physical location
in memory (whence I presume two different processes can mmap anonymous
memory block in the same location)

Um, no, it lets you specify the *virtual* address in the process's
address space at which the object you specify is to be mapped.

As far as I know, the only way two unrelated processes can share
memory via mmap is by mapping a file. An anonymous block is known
only to the process that creates it -- being anonymous, there's
no way for another process to refer to it.

However, if one process is forked from the other, the parent
can mmap an anonymous block and the child will inherit that
mapping.

(I suppose if both processes had sufficient privileges they
could map physical memory out of /dev/mem, but that would be
*really* dangerous!)
 
J

Jeff Schwab

greg said:
Um, no, it lets you specify the *virtual* address in the process's
address space at which the object you specify is to be mapped.

As far as I know, the only way two unrelated processes can share
memory via mmap is by mapping a file. An anonymous block is known
only to the process that creates it -- being anonymous, there's
no way for another process to refer to it.

On POSIX systems, you can create a shared memory object without a file
using shm_open. The function returns a file descriptor.
 
N

Nikita the Spider

Jeff Schwab said:
Thanks; I just downloaded it. It seems to be missing the INSTALL file;
any idea where I could find that, or should I write to the author?

The main author has been AWOL for some years now. I'm the current
maintainer. Sorry about the missing INSTALL file. Looks like I made
reference to it in the README but never created it. It installs with the
normal setup.py:

sudo python setup.py

I'll put out an updated package with an INSTALL file one of these days.
Thanks for pointing that out.

Cheers
 
R

Ryan Smith-Roberts

Suppose I've a process P1, which generates itself a lot of data , for
example 2Mb.
Then, I've a process P2 which must access P1 shared memory and,
probably, modify this data.
To accomplish this, I've been digging around python's mmap module, but I
can't figure how to use it without files.

Could anybody explain me how could this be accomplished? An example will
be very appreciated. Thanks a lot for your help.

A non-portable solution, for modern Linux systems, is to create and
mmap a file in the /dev/shm directory. This is a more "unix-y"
solution than the SysV SHM interface mentioned elsewhere in the
thread, since it gives you files you can cat, ls, chown, etc. Files
created in this directory may hit swap, but don't consume normal disk
space.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top