Create TarFile using string buffers

[aurora00]

I have a program that generates a number of files that will be
packaged into a tarball. Can I stream the content into TarFile without
first writing them out to the file system? All add(), addfile() and
gettarinfo() seems to assume there is a file in the disk. But for me I
seems inefficient to write all the content to the disk and then have
it read back by the TarFile module.

Thank you for your help

wy

 

[Lars =?iso-8859-15?Q?Gust=E4bel?=]

I have a program that generates a number of files that will be
packaged into a tarball. Can I stream the content into TarFile without
first writing them out to the file system? All add(), addfile() and
gettarinfo() seems to assume there is a file in the disk. But for me I
seems inefficient to write all the content to the disk and then have
it read back by the TarFile module.

addfile()'s fileobj argument can be anything that has a read()
method. The amount of bytes to read is taken from the
tarinfo.size attribute. You could let your program write its
data to a StringIO object and pass that object to addfile().

http://docs.python.org/lib/tarfile-objects.html

 

[Gabriel Genellina]

En Mon, 19 Mar 2007 16:06:39 -0300, (e-mail address removed)

I have a program that generates a number of files that will be
packaged into a tarball. Can I stream the content into TarFile without
first writing them out to the file system? All add(), addfile() and
gettarinfo() seems to assume there is a file in the disk. But for me I
seems inefficient to write all the content to the disk and then have
it read back by the TarFile module.

You can create a TarInfo object directly, and use addfile with a StringIO
object:

import tarfile
from cStringIO import StringIO

data = "Some text, maybe containing\ntwo or more lines\n" + "The quick
brown fox jumps over the lazy dog\n" * 20
fobj = StringIO(data)

tar = tarfile.open("sample.tar", "w")
tarinfo = tarfile.TarInfo("foo.txt")
tarinfo.size = len(data)
tar.addfile(tarinfo, fobj)
tar.close()

tar = tarfile.open("sample.tar", "r")
tar.list()
foo = tar.extractfile("foo.txt")
data_read = foo.read()
print "foo.txt:\n%s" % data_read
tar.close()
assert data == data_read

 

[=?iso-8859-1?Q?Andr=E9s?= Martinelli]

I'm using the example of the site
http://docs.python.org/lib/module-struct.html :

import struct
pack('hhl', 1, 2, 3)

I should get:
I know it should work. The code is OK.
What could be wrong? Doesnt find the library module?
I repeat that the unpack works ok.
I use debian etch and sarge. Same situation in both of them.

Thanks.
Andrés M.

 

[skip]

Andrés> I'm using the example of the site
Andrés> http://docs.python.org/lib/module-struct.html :

Andrés> import struct
Andrés> pack('hhl', 1, 2, 3)

Andrés> I should get:
Try

import struct
struct.pack('hhl', 1, 2, 3)

or

from struct import pack
pack('hhl', 1, 2, 3)

instead.

Skip

 

[Gabriel Genellina]

En Mon, 19 Mar 2007 19:32:41 -0300, Andrés Martinelli

I'm using the example of the site
http://docs.python.org/lib/module-struct.html :

import struct
pack('hhl', 1, 2, 3)

That code should raise a NameError. Either you are using *another* pack
function, or that is not what you actually typed.
This is what I get:

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
py> import struct
py> struct.pack('hhl', 1, 2, 3)
'\x01\x00\x02\x00\x03\x00\x00\x00'

I should get:

Maybe on another platform - on i386 I get the byte ordering as above.

 

[aurora00]

Thanks. It almost works. The problem is I don't know the size of the
file until it has finished streaming. It looks like the tar file
format need the file size written at the beginning

 

[Gabriel Genellina]

En Mon, 19 Mar 2007 21:55:30 -0300, (e-mail address removed)

Thanks. It almost works. The problem is I don't know the size of the
file until it has finished streaming. It looks like the tar file
format need the file size written at the beginning

Yes, maybe because it's originally a tape format. Anyway it could be done;
addfile() could seek back to the header and patch it after the file size
is known...