Problem with FTPLib and incomplete files on some downloads

Discussion in 'Python' started by Peter A. Schott, Jan 17, 2006.

  1. I download a lot of 4-6 KB files and regularly run into issues with files that
    don't get downloaded all the way or otherwise get corrupt.

    I do something like:

    RemoteList = nlstdir()
    for filename in RemoteList:
    LocalFile = open(filename, "wb")
    LocalFile.write( "get file code here" )
    LocalFile.close()
    #ftplib call to delete the remote file

    I've tried to insert a pause into the code between the close and the remote
    delete, but that doesn't seem to help. For some reason it doesn't seem like the
    buffer is completely written or read before the remote delete is called. That
    leads to a file that's not 100% complete and thus can't be decrypted.

    Any ideas on what to look for? I don't have my exact code handy at the moment
    or I'd post some actual snippets. It's also not consistent or it would be a
    whole lot easier to troubleshoot. :)

    Running Python 2.4.2 on Win32.

    Thanks.

    -Pete Schott
    Peter A. Schott, Jan 17, 2006
    #1
    1. Advertising

  2. Peter A.Schott wrote:
    > I download a lot of 4-6 KB files and regularly run into issues with files that
    > don't get downloaded all the way or otherwise get corrupt.
    >
    > I do something like:
    >
    > RemoteList = nlstdir()
    > for filename in RemoteList:
    > LocalFile = open(filename, "wb")
    > LocalFile.write( "get file code here" )
    > LocalFile.close()
    > #ftplib call to delete the remote file
    >
    > I've tried to insert a pause into the code between the close and the remote
    > delete, but that doesn't seem to help. For some reason it doesn't seem like the
    > buffer is completely written or read before the remote delete is called. That
    > leads to a file that's not 100% complete and thus can't be decrypted.
    >
    > Any ideas on what to look for? I don't have my exact code handy at the moment
    > or I'd post some actual snippets. It's also not consistent or it would be a
    > whole lot easier to troubleshoot. :)
    >
    > Running Python 2.4.2 on Win32.
    >
    > Thanks.
    >
    > -Pete Schott



    Pete,


    Not sure but here is an example that seems to always work for me:
    (tested this morning before pasting here on ~20 files around 7 - 300 k)

    import ftplib
    import os

    os.chdir("D:/TEMP/dump")

    ftp = ftplib.FTP("server")
    print ftp.login("anonymous", "")

    ftp.cwd("pub/Python_Applications")




    for rFile in ftp.nlst():
    print "Getting : ", rFile
    rSize = ftp.size(rFile)
    lFile = open(rFile, "wb")
    ftp.retrbinary("RETR %s" %rFile, lFile.write)
    lSize = lFile.tell()
    lFile.close()
    if rSize==lSize:
    print "Transfer complete"
    else:
    print "BAD Transfer", rSize, lSize


    ftp.close()

    HTH
    Martin
    Martin Franklin, Jan 18, 2006
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    725
    Steve Holden
    Sep 12, 2003
  2. Peter A. Schott
    Replies:
    0
    Views:
    250
    Peter A. Schott
    Aug 23, 2005
  3. Gabriel Genellina
    Replies:
    0
    Views:
    415
    Gabriel Genellina
    Jan 25, 2007
  4. Gabriel Genellina
    Replies:
    0
    Views:
    365
    Gabriel Genellina
    Jan 28, 2007
  5. Replies:
    3
    Views:
    345
    Rick Johnson
    Feb 28, 2013
Loading...

Share This Page