Rails threaded file upload via cURL.

[gberz3]

Hi All,

I am currently working on a project that needs to upload multiple
files to a server. It needs to do so in quick fashion. Originally I
had a script that simply uploaded files serially and it worked like a
charm. My client, however, requires it to work "somewhat faster, and
upload more at a time". So, I added threading. The problem is, that
the application is supposed to repeat and always upload whenever files
are present. As such I have it inside a "while" loop.

The problem I'm having is that with Threading, the final cURL
transaction seems to linger eternally and the process will never
repeat. However, if I go back to serial processing, everything works
fine. What aspect of threading am I not taking into account?

The semi-demi-gist of the code is as follows:


while 1
my_threads = []

list_of_files = function_fill_file_list()

list_of_files.each {|cur_file|
my_threads << Thread.new {

command = "curl -F filedata=#{file} http://10.0.0.1/awesome.php"
system(command)
deletefile(cur_file)

}
}


my_threads.each { |cur_thread| cur_thread.join}


sleep(10)

end



Best Regards.