CRC-checksum failed in gzip

Discussion in 'Python' started by andrea crotti, Aug 1, 2012.

  1. We're having some really obscure problems with gzip.
    There is a program running with python2.7 on a 2.6.18-128.el5xen (red
    hat I think) kernel.

    Now this program does the following:
    if filename == 'out2.txt':
    out2 = open('out2.txt')
    elif filename == 'out2.txt.gz'
    out2 = open('out2.txt.gz')

    text = out2.read()

    out2.close()

    very simple right? But sometimes we get a checksum error.
    Reading the code I got the following:

    - CRC is at the end of the file and is computed against the whole
    file (last 8 bytes)
    - after the CRC there is the \0000 marker for the EOF
    - readline() doesn't trigger the checksum generation in the
    beginning, but only when the EOF is reached
    - until a file is flushed or closed you can't read the new content in it

    but the problem is that we can't reproduce it, because doing it
    manually on the same files it works perfectly,
    and the same files some time work some time don't work.

    The files are on a shared NFS drive, I'm starting to think that it's a
    network/fs problem, which might truncate the file
    adding an EOF before the end and thus making the checksum fail..
    But is it possible?
    Or what else could it be?
     
    andrea crotti, Aug 1, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy

    Re: CRC-checksum failed in gzip

    Laszlo Nagy, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    151
    Laszlo Nagy
    Aug 1, 2012
  2. andrea crotti

    Re: CRC-checksum failed in gzip

    andrea crotti, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    155
    andrea crotti
    Aug 1, 2012
  3. Laszlo Nagy

    Re: CRC-checksum failed in gzip

    Laszlo Nagy, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    187
    Laszlo Nagy
    Aug 1, 2012
  4. andrea crotti

    Re: CRC-checksum failed in gzip

    andrea crotti, Aug 1, 2012, in forum: Python
    Replies:
    11
    Views:
    438
    andrea crotti
    Aug 2, 2012
  5. Laszlo Nagy

    Re: CRC-checksum failed in gzip

    Laszlo Nagy, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    212
    Laszlo Nagy
    Aug 1, 2012
Loading...

Share This Page