mmap -- memory mapped file

Discussion in 'Python' started by Carl Mackey, Jun 29, 2006.

  1. Carl Mackey

    Carl Mackey Guest

    hi, i'm new to this list and new to python as well.

    i have a question on the memory mapped file ability python has. when i
    use a mmap on a file, will it copy the whole thing to ram or just
    whatever part of it i'm working on? basically, i'm wondering if it would
    be ok for me to have multiple mmap's open on very large files as i read
    or write from them.
    Carl Mackey, Jun 29, 2006
    #1
    1. Advertising

  2. Carl Mackey <> wrote:

    > hi, i'm new to this list and new to python as well.
    >
    > i have a question on the memory mapped file ability python has. when i
    > use a mmap on a file, will it copy the whole thing to ram or just
    > whatever part of it i'm working on? basically, i'm wondering if it would
    > be ok for me to have multiple mmap's open on very large files as i read
    > or write from them.


    Python relies on the operating system to do the "memory mapping"
    sensibly; generally, that means the whole file is mapped in the
    *virtual* memory of the process -- the process's address space -- but
    only the necessary pages actually get into RAM. This should work fine
    with any modern Linux, *BSD, MacOSX, or just about any other Unix that
    is still sold, or, also, on Windows (at least in the NT/2000/XP line, I
    wouldn't be so sure on some ancient Windows/98 box:).

    Still, assuming that my "very large files" you mean several gigabytes,
    mmap will not work well on them with a 32-bit machine, or a 64-bit
    machine hobbled by a 32-bit operating system -- the process's address
    space being limited to no more 4GB, you may be unable to mmap even just
    one "very large file" (this is totally unrelated to how much RAM you may
    have: the limitation is with the *address space* of each process).


    Alex
    Alex Martelli, Jun 29, 2006
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    2
    Views:
    412
  2. Srijit Kumar Bhadra

    Memory mapped File (Python win32 extensions)

    Srijit Kumar Bhadra, Mar 17, 2005, in forum: Python
    Replies:
    1
    Views:
    408
    Thomas Heller
    Mar 17, 2005
  3. Replies:
    3
    Views:
    285
    Walter Roberson
    Jan 10, 2006
  4. Replies:
    0
    Views:
    256
  5. Replies:
    8
    Views:
    390
    Keith Thompson
    Jan 10, 2006
Loading...

Share This Page