Re: CRC-checksum failed in gzip

Discussion in 'Python' started by andrea crotti, Aug 1, 2012.

  1. 2012/8/1 Laszlo Nagy <>:
    >> there seems to be no clear pattern and just randmoly fails. The file
    >> is also just open for read from this program,
    >> so in theory no way that it can be corrupted.

    >
    > Yes, there is. Gzip stores CRC for compressed *blocks*. So if the file is
    > not flushed to the disk, then you can only read a fragment of the block, and
    > that changes the CRC.
    >
    >>
    >> I also checked with lsof if there are processes that opened it but
    >> nothing appears..

    >
    > lsof doesn't work very well over nfs. You can have other processes on
    > different computers (!) writting the file. lsof only lists the processes on
    > the system it is executed on.
    >
    >>
    >> - can't really try on the local disk, might take ages unfortunately
    >> (we are rewriting this system from scratch anyway)
    >>

    >



    Thanks a lotl, someone that writes on the file while reading might be
    an explanation, the problem is that everyone claims that they are only
    reading the file.

    Apparently this file is generated once and a long time after only read
    by two different tools (in sequence), so this could not be possible
    either in theory.. I'll try to investigate more in this sense since
    it's the only reasonable explation I've found so far.
    andrea crotti, Aug 1, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. andrea crotti

    CRC-checksum failed in gzip

    andrea crotti, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    134
    andrea crotti
    Aug 1, 2012
  2. Laszlo Nagy

    Re: CRC-checksum failed in gzip

    Laszlo Nagy, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    142
    Laszlo Nagy
    Aug 1, 2012
  3. andrea crotti

    Re: CRC-checksum failed in gzip

    andrea crotti, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    148
    andrea crotti
    Aug 1, 2012
  4. Laszlo Nagy

    Re: CRC-checksum failed in gzip

    Laszlo Nagy, Aug 1, 2012, in forum: Python
    Replies:
    0
    Views:
    174
    Laszlo Nagy
    Aug 1, 2012
  5. andrea crotti

    Re: CRC-checksum failed in gzip

    andrea crotti, Aug 1, 2012, in forum: Python
    Replies:
    11
    Views:
    421
    andrea crotti
    Aug 2, 2012
Loading...

Share This Page