Comparing files in a zip to files on drive

[kj7ny]

I am attempting to incrementally back up files using python. How can I
compare a file on a hard drive to a file in a python created zip file
to tell if the file has changed? Or should I be using some other
method to determine if a file has changed?

 

[Justin Ezequiel]

compare a file on a hard drive to a file in a python created zip file
to tell if the file has changed?

fileondisk = r'C:\Documents and Settings\power\Desktop\ES3dc+Template.3f'
zipondisk = r'C:\Documents and Settings\power\Desktop\temps.zip'
import zipfile
z = zipfile.ZipFile(zipondisk)
z.namelist() ['ES1+Template.3f', 'ES3sc+Template.3f', 'ES3dc+Template.3f']
info = z.getinfo('ES3dc+Template.3f')
info.CRC 1620938006
import binascii
checksum = 0
fp = open(fileondisk, 'rb')
while 1:

.... data = fp.read(32*1024)
.... if not data: break
.... checksum = binascii.crc32(data, checksum)
....

 

[Yu-Xi Lim]

I am attempting to incrementally back up files using python. How can I
compare a file on a hard drive to a file in a python created zip file
to tell if the file has changed? Or should I be using some other
method to determine if a file has changed?

You can easily determine size and date from the hard drive/filesystem
and from zip files. Look at the Python docs for the os and zipfile
modules. That would be the fastest way to detect changes for most files
you will encounter.

Zip files have the CRC of each file (generated from the original
uncompressed file) for checksumming purposes. You can calculate the CRC
for the file on the hard drive and then compare the CRC values.
Calculating the CRC should be faster than decompressing the ZIPped file
or compressing the file on the hard drive. While not perfect it should
definitely give you a very good indication of changed content. Python
doesn't any built-in functions for CRC calculations, AFAIK. There are
third party modules around.

A better plan would be to store a list of MD5 or SHA1 hashes for each
file in your archive. These functions are built in to Python and are
more suitable for detection large differences in files (these are hash
functions while CRC is a checksum function and thus are less likely to
have collisions). You can store the list of hashes in a separate file,
or as a file in the ZIP file, or even in the ZIP file header.

Your last resort would be to decompress each file and compare it with
the one on disk. This will take a lot of time and memory. Not at all
advisable but definitely foolproof.

There are other issues you will have to contend with, such as renaming
of files and directories, and deletions. If I were you, I'd take a look
at the work done on programs like rsync, unison (both file
synchronization programs), or CVS and Subversion (both version or source
control programs) to see how they deal with such issues.