T
The Weeg
I wrote a pretty simple web scraping tool, and i'm having it organize
the results into folders based upon search queries. After six hundred
or so of these folders are created, i get a bizarre, "too many files
open" error(Errno::EMFILE).
This is the line of code it crashes on:
`mkdir #{foldername}`
And the only time I ever open and close files in my program is in
these few lines, but the program isn't crashing on these lines:
fileTempFile = File.new(filePath + "full_" + fileName +
".html","w")
fileTempFile.puts(response)
fileLinkText = File.new(filePath + "relevant_" + fileName +
".txt","w")
fileLinkText.puts(link + "\n" +
match.to_s.gsub(/<.*?>/,"").gsub(/&[a-zA-Z0-9#]*?;/,''))
fileLinkText.close
fileTempFile.close
As you can see, the files are being opened and closed almost
immediately.
Is there a different way I should be creating directories rather than
through a shell command?
the results into folders based upon search queries. After six hundred
or so of these folders are created, i get a bizarre, "too many files
open" error(Errno::EMFILE).
This is the line of code it crashes on:
`mkdir #{foldername}`
And the only time I ever open and close files in my program is in
these few lines, but the program isn't crashing on these lines:
fileTempFile = File.new(filePath + "full_" + fileName +
".html","w")
fileTempFile.puts(response)
fileLinkText = File.new(filePath + "relevant_" + fileName +
".txt","w")
fileLinkText.puts(link + "\n" +
match.to_s.gsub(/<.*?>/,"").gsub(/&[a-zA-Z0-9#]*?;/,''))
fileLinkText.close
fileTempFile.close
As you can see, the files are being opened and closed almost
immediately.
Is there a different way I should be creating directories rather than
through a shell command?