Re: Python-list Digest, Vol 75, Issue 226

Discussion in 'Python' started by Gabriel Genellina, Dec 22, 2009.

  1. En Mon, 21 Dec 2009 16:30:13 -0300, Pulkit Agrawal
    <> escribió:

    > I am writing a script wherein I need to merge files into existing tar.gz
    > files. Currently, I am using tarfile module. I extract the tar.gz to a
    > tempdir and copy the new file there and re-compress all the files back
    > into
    > a tar.gz. Is there a better way to do it?


    Since noone answered yet: no, I don't think you can avoid to decompress
    and recompress those files.

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 22, 2009
    #1
    1. Advertising

  2. Gabriel Genellina

    r0g Guest

    Gabriel Genellina wrote:
    > En Mon, 21 Dec 2009 16:30:13 -0300, Pulkit Agrawal
    > <> escribió:
    >
    >> I am writing a script wherein I need to merge files into existing tar.gz
    >> files. Currently, I am using tarfile module. I extract the tar.gz to a
    >> tempdir and copy the new file there and re-compress all the files back
    >> into
    >> a tar.gz. Is there a better way to do it?

    >
    > Since noone answered yet: no, I don't think you can avoid to decompress
    > and recompress those files.
    >




    Erm, I always thought it was OK to simply cat gzipped files together...


    From man gzip...


    Multiple compressed files can be concatenated. In this case,
    gunzip will extract all members at once. For example:

    gzip -c file1 > foo.gz
    gzip -c file2 >> foo.gz

    Then

    gunzip -c foo

    is equivalent to

    cat file1 file2



    Roger.
    r0g, Dec 22, 2009
    #2
    1. Advertising

  3. En Tue, 22 Dec 2009 16:30:58 -0300, r0g <>
    escribió:
    > Gabriel Genellina wrote:
    >> En Mon, 21 Dec 2009 16:30:13 -0300, Pulkit Agrawal
    >> <> escribió:
    >>
    >>> I am writing a script wherein I need to merge files into existing
    >>> tar.gz
    >>> files. Currently, I am using tarfile module. I extract the tar.gz to a
    >>> tempdir and copy the new file there and re-compress all the files back
    >>> into
    >>> a tar.gz. Is there a better way to do it?

    >>
    >> Since noone answered yet: no, I don't think you can avoid to decompress
    >> and recompress those files.

    >
    > Erm, I always thought it was OK to simply cat gzipped files together...


    Maybe, but still I don't think this could help the OP. As I understand the
    problem, originally there were e.g.: file1, file2, file3; they were tarred
    into file123.tar and gzipped into file123.tar.gz. And now file2 must be
    replaced by a newer version. It should go into the internal .tar file,
    replacing the old one; I don't see how to do that without decompressing
    it. (Ok, once the tar is decompressed one might replace the old file with
    the newer one in-place using the tar command, but this cannot be done with
    the tarfile Python module)

    --
    Gabriel Genellina
    Gabriel Genellina, Dec 22, 2009
    #3
  4. Gabriel Genellina

    r0g Guest

    Gabriel Genellina wrote:
    > En Tue, 22 Dec 2009 16:30:58 -0300, r0g <>
    > escribió:
    >> Gabriel Genellina wrote:
    >>> En Mon, 21 Dec 2009 16:30:13 -0300, Pulkit Agrawal
    >>> <> escribió:
    >>>
    >>>> I am writing a script wherein I need to merge files into existing
    >>>> tar.gz
    >>>> files. Currently, I am using tarfile module. I extract the tar.gz to a
    >>>> tempdir and copy the new file there and re-compress all the files back
    >>>> into
    >>>> a tar.gz. Is there a better way to do it?
    >>>
    >>> Since noone answered yet: no, I don't think you can avoid to decompress
    >>> and recompress those files.

    >>
    >> Erm, I always thought it was OK to simply cat gzipped files together...

    >
    > Maybe, but still I don't think this could help the OP. As I understand
    > the problem, originally there were e.g.: file1, file2, file3; they were
    > tarred into file123.tar and gzipped into file123.tar.gz. And now file2
    > must be replaced by a newer version. It should go into the internal .tar
    > file, replacing the old one; I don't see how to do that without
    > decompressing it. (Ok, once the tar is decompressed one might replace
    > the old file with the newer one in-place using the tar command, but this
    > cannot be done with the tarfile Python module)
    >



    Oh I didn't see the original posting! I agree, if files within the
    tarball need to be replaced (as opposed to more new files added) that
    can only be done on uncompressed tar archives.

    If performance is a issue and the files aren't too gargantuan it may be
    best to de(and re)compress the gzips to memory/ramdisk rather than
    writing it all out to disk. If the files are too large for memory but
    the OP can afford a bit more storage then storing plain uncompressed
    tarballs would allow use of tar's delete, replace and append
    functionality. If that would take up too much space maybe they could use
    plain tar on a block compressed filesystem instead.

    Cheers,

    Roger.
    r0g, Dec 23, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Andrew James

    Re: Python-list Digest, Vol 17, Issue 54

    Andrew James, Feb 3, 2005, in forum: Python
    Replies:
    2
    Views:
    306
    Steve Holden
    Feb 3, 2005
  2. Charles Hartman

    Re: Python-list Digest, Vol 18, Issue 208

    Charles Hartman, Mar 14, 2005, in forum: Python
    Replies:
    10
    Views:
    506
    Roy Smith
    Mar 16, 2005
  3. Charles Hartman

    Re: Python-list Digest, Vol 18, Issue 391

    Charles Hartman, Mar 24, 2005, in forum: Python
    Replies:
    1
    Views:
    315
    Peter Hansen
    Mar 25, 2005
  4. Replies:
    2
    Views:
    280
  5. Adrian Wood
    Replies:
    0
    Views:
    256
    Adrian Wood
    Jan 10, 2008
Loading...

Share This Page