Problem with FTPLib and incomplete files on some downloads

P

Peter A. Schott

I download a lot of 4-6 KB files and regularly run into issues with files that
don't get downloaded all the way or otherwise get corrupt.

I do something like:

RemoteList = nlstdir()
for filename in RemoteList:
LocalFile = open(filename, "wb")
LocalFile.write( "get file code here" )
LocalFile.close()
#ftplib call to delete the remote file

I've tried to insert a pause into the code between the close and the remote
delete, but that doesn't seem to help. For some reason it doesn't seem like the
buffer is completely written or read before the remote delete is called. That
leads to a file that's not 100% complete and thus can't be decrypted.

Any ideas on what to look for? I don't have my exact code handy at the moment
or I'd post some actual snippets. It's also not consistent or it would be a
whole lot easier to troubleshoot. :)

Running Python 2.4.2 on Win32.

Thanks.

-Pete Schott
 
M

Martin Franklin

Peter said:
I download a lot of 4-6 KB files and regularly run into issues with files that
don't get downloaded all the way or otherwise get corrupt.

I do something like:

RemoteList = nlstdir()
for filename in RemoteList:
LocalFile = open(filename, "wb")
LocalFile.write( "get file code here" )
LocalFile.close()
#ftplib call to delete the remote file

I've tried to insert a pause into the code between the close and the remote
delete, but that doesn't seem to help. For some reason it doesn't seem like the
buffer is completely written or read before the remote delete is called. That
leads to a file that's not 100% complete and thus can't be decrypted.

Any ideas on what to look for? I don't have my exact code handy at the moment
or I'd post some actual snippets. It's also not consistent or it would be a
whole lot easier to troubleshoot. :)

Running Python 2.4.2 on Win32.

Thanks.

-Pete Schott


Pete,


Not sure but here is an example that seems to always work for me:
(tested this morning before pasting here on ~20 files around 7 - 300 k)

import ftplib
import os

os.chdir("D:/TEMP/dump")

ftp = ftplib.FTP("server")
print ftp.login("anonymous", "(e-mail address removed)")

ftp.cwd("pub/Python_Applications")




for rFile in ftp.nlst():
print "Getting : ", rFile
rSize = ftp.size(rFile)
lFile = open(rFile, "wb")
ftp.retrbinary("RETR %s" %rFile, lFile.write)
lSize = lFile.tell()
lFile.close()
if rSize==lSize:
print "Transfer complete"
else:
print "BAD Transfer", rSize, lSize


ftp.close()

HTH
Martin
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,898
Latest member
BlairH7607

Latest Threads

Top