how to know if completely donwloaded

[Dy Dymy]

i have tried combining all what i research...

#code starts here:

listofFile = [ ... ] #<-- internet files like PDF, etc...

listofFile.each do |each_file|
fileBaseName = File.basename(each_file)
filePath = Dir.pwd + '/' + fileBaseName
begin
if File.exist?(fileBaseName) != true
openFileRead = open(each_file)
else
puts "message: File already exist = #{fileBaseName}"
end
rescue
retry
else
begin
writeOut = open(filePath, 'wb+')
#<-- if internet cuts here, how do i know if its completely downloaded
writeOut.write(openFileRead.read)
rescue => err
puts err.message
puts "error: #{each_file} -- download error."
else
writeOut.close
end
end
end

# code ends here

actually i have internet connection problem here... if the cuts the file
will be corrupted...

please advice me.. or do i miss something...

thanks

 

[Dy Dymy]

guy please help me... i cant detect if internet connection cuts the file
will be corrupted...

 

[Luke Cowell]

guy please help me... i cant detect if internet connection cuts the file
will be corrupted...

It appears that you have already downloaded the files and are attempting
to validate them after the download process is complete.

I'd suggest a couple of different strategies: one would be to have the
download process raise an error if the download does not complete and
the other would be to check the files against the server you are
downloading from. If possible use an md5 checksum and otherwise you
could check the size.

If you were downloading zip, gzip files or some other compressed format
they will usually have built in checksumming, so you could verify the
file that way. Other file formats might have ways of being validated
too.

Luke

 

[Dy Dymy]

It appears that you have already downloaded the files and are attempting
to validate them after the download process is complete.

I'd suggest a couple of different strategies: one would be to have the
download process raise an error if the download does not complete and
the other would be to check the files against the server you are
downloading from. If possible use an md5 checksum and otherwise you
could check the size.

If you were downloading zip, gzip files or some other compressed format
they will usually have built in checksumming, so you could verify the
file that way. Other file formats might have ways of being validated
too.

Luke

hi Luke,

thanks for the reply...

firstly its correct that i am validating files after i download the
file.

"raise an error if the download does not complete"
question: Where i will put my raise error in my code?... i put a "begin
- rescue - else - end" in my code... hardest part is ".read" has no
error exemption to check if it reads all throughout the file.

"check the files against the server you are downloading from."
question: how do i check the files against the server?

"md5 checksum"
question: what is md5 checksum?

"check the size"
comment: after i made this thread, i added my code to check the size.
Before i download the file i check the size using "uri meta .length" and
compare it after i download it using "File.size"... but still no use
because if it break at ".read" the size that i will get is still same as
what i downloaded... meaning uri.length = File.size

thank ahead...

 

[Luke Cowell]

"raise an error if the download does not complete"
question: Where i will put my raise error in my code?... i put a "begin
- rescue - else - end" in my code... hardest part is ".read" has no
error exemption to check if it reads all throughout the file.

I'm talking about adding an exception into the code that does the
downloading, not the code that reads off the disk. If you can detect
that a download has ended unexpectedly, you can queue the download to
start again.

"check the files against the server you are downloading from."
question: how do i check the files against the server?

By md5 checksum or the file size.

"md5 checksum"
question: what is md5 checksum?

Do some googling my friend.

"check the size"
comment: after i made this thread, i added my code to check the size.
Before i download the file i check the size using "uri meta .length" and
compare it after i download it using "File.size"... but still no use
because if it break at ".read" the size that i will get is still same as
what i downloaded... meaning uri.length = File.size

If you know the length the content is supposed to be doing something
like this:

Net::HTTP.start("www.example.com", 80) {|res| h = res.head("/"); puts
h.to_yaml }

You can check that against the size of the downloaded file.

I think I might be able to help you more if you posted your download
code.

thank ahead...

No problem.

Luke