memory mapped tar file contents

Discussion in 'Python' started by Chris Brooks, Nov 13, 2008.

  1. Chris Brooks

    Chris Brooks Guest

    Hi,

    I would like to read directly from a tar file into memory so I can
    manipulate a file (quickly) and write its changes out to another file. I
    thought I could do something like:

    #!/usr/bin/env python

    import tarfile
    import mmap

    fil = tarfile.open( "out.tar.gz" , "r:gz" )
    tarinf = fil.next()
    myfils = {}
    while tarinf != None:
    tarinf = fil.next()
    ref = fil.extractfile( tarinf )
    myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )

    But the extractfile() function of TarInfo doesn't seem to give me a fileno,
    so I can't pass this to mmap.

    Thoughts on a way to accomplish this?

    Chris
    --
    View this message in context: http://www.nabble.com/memory-mapped-tar-file-contents-tp20473925p20473925.html
    Sent from the Python - python-list mailing list archive at Nabble.com.
     
    Chris Brooks, Nov 13, 2008
    #1
    1. Advertising

  2. Chris Brooks

    Aaron Brady Guest

    On Nov 12, 8:51 pm, Chris Brooks <> wrote:
    > Hi,
    >
    > I would like to read directly from a tar file into memory so I can
    > manipulate a file (quickly) and write its changes out to another file.  I
    > thought I could do something like:
    >
    > #!/usr/bin/env python
    >
    > import tarfile
    > import mmap
    >
    > fil = tarfile.open( "out.tar.gz" , "r:gz" )
    > tarinf = fil.next()
    > myfils = {}
    > while tarinf != None:
    >     tarinf = fil.next()
    >     ref = fil.extractfile( tarinf )
    >     myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )
    >
    > But the extractfile() function of TarInfo doesn't seem to give me a fileno,
    > so I can't pass this to mmap.
    >
    > Thoughts on a way to accomplish this?


    It appears you have to read the contents of the file into the mmap.
    You can create an anonymous map of size tarinfoobj.size, then set
    mapobj[:]= fil.extractfile( tarinf ).read( ) . Untested.
     
    Aaron Brady, Nov 13, 2008
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Claudio Grondi
    Replies:
    4
    Views:
    589
    Claudio Grondi
    Aug 20, 2005
  2. Replies:
    3
    Views:
    316
    Walter Roberson
    Jan 10, 2006
  3. Replies:
    0
    Views:
    285
  4. Replies:
    8
    Views:
    418
    Keith Thompson
    Jan 10, 2006
  5. Replies:
    2
    Views:
    454
    Michael Hoffman
    Apr 24, 2007
Loading...

Share This Page