zipfile module

[LC]

Hi,

I'm having a problem using the zipfile module in Windows 2000 sp4. When I
use it to zip a small file it works fine, but large file doesnt. Here's the
error msg i get...
---------------------------------------------------------------------
dynamite4_db_archive.py", line 45, in ?
file.write(name, os.path.basename(name), zipfile.ZIP_DEFLATED)
File "C:\Python22\lib\zipfile.py", line 426, in write
zinfo.file_size))
OverflowError: long int too large to convert to int
---------------------------------------------------------------------
I've read a couple of problems with this module with long int, and it says
it's a know problem in Python earlier than 2.3
So I upgraded to 2.3 and i'm still getting the error msg. I'm not proficient
at python so I'm not sure what to do at this point. The file I'm currently
trying to zip is about 2.5GB.

Here's my code that's causing the error...
----------------------------------------------------------------------
currentDirFiles = os.listdir(mydir) #list log file in dir
os.mkdir(todays_dir)

for item in currentDirFiles: #find the file with the date from 7 days ago

file_to_zip = mydir + item
destination = todays_dir + str('/') + item + str('_') + todays_date +
str('.zip')

print "\n" + file_to_zip + "\n" + destination

file = zipfile.ZipFile(destination, "w")

for name in glob.glob(file_to_zip):
file.write(name, os.path.basename(name), zipfile.ZIP_DEFLATED)

file.close()

 

[Skip Montanaro]

lco> dynamite4_db_archive.py", line 45, in ?
lco> file.write(name, os.path.basename(name), zipfile.ZIP_DEFLATED)
lco> File "C:\Python22\lib\zipfile.py", line 426, in write
lco> zinfo.file_size))
lco> OverflowError: long int too large to convert to int

...

lco> for name in glob.glob(file_to_zip):
lco> file.write(name, os.path.basename(name), zipfile.ZIP_DEFLATED)

How about you write the file in chunks? I'm not familiar with the zipfile
module, but it looks like writestr will do the job for you. My guess is
something like

...
file = zipfile.ZipFile(destination, "w", zipfile.ZIP_DEFLATE)
for name in glob.glob(file_to_zip):
...
f = open(name)
data = f.read(8192)
while data:
file.writestr(os.path.basename(name), data)
data = f.read(8192)
f.close()
file.close()

might do the trick. (The zipfile module's interface to writing data looks a
bit odd to me.)

Skip