Too many files open?

Discussion in 'Ruby' started by The Weeg, Dec 16, 2004.

  1. The Weeg

    The Weeg Guest

    I wrote a pretty simple web scraping tool, and i'm having it organize
    the results into folders based upon search queries. After six hundred
    or so of these folders are created, i get a bizarre, "too many files
    open" error(Errno::EMFILE).

    This is the line of code it crashes on:
    `mkdir #{foldername}`

    And the only time I ever open and close files in my program is in
    these few lines, but the program isn't crashing on these lines:

    fileTempFile = File.new(filePath + "full_" + fileName +
    ".html","w")
    fileTempFile.puts(response)
    fileLinkText = File.new(filePath + "relevant_" + fileName +
    ".txt","w")
    fileLinkText.puts(link + "\n" +
    match.to_s.gsub(/<.*?>/,"").gsub(/&[a-zA-Z0-9#]*?;/,''))
    fileLinkText.close
    fileTempFile.close


    As you can see, the files are being opened and closed almost
    immediately.

    Is there a different way I should be creating directories rather than
    through a shell command?
    The Weeg, Dec 16, 2004
    #1
    1. Advertising

  2. The Weeg

    Eric Hodel Guest

    On 16 Dec 2004, at 10:37, The Weeg wrote:

    > I wrote a pretty simple web scraping tool, and i'm having it organize
    > the results into folders based upon search queries. After six hundred
    > or so of these folders are created, i get a bizarre, "too many files
    > open" error(Errno::EMFILE).
    >
    > This is the line of code it crashes on:
    > `mkdir #{foldername}`


    This opens a file to recieve the shell's response, instead use
    Dir.mkdir or FileUtils.

    > And the only time I ever open and close files in my program is in
    > these few lines, but the program isn't crashing on these lines:
    >
    > fileTempFile = File.new(filePath + "full_" + fileName +
    > ".html","w")
    > fileTempFile.puts(response)
    > fileLinkText = File.new(filePath + "relevant_" + fileName +
    > ".txt","w")
    > fileLinkText.puts(link + "\n" +
    > match.to_s.gsub(/<.*?>/,"").gsub(/&[a-zA-Z0-9#]*?;/,''))
    > fileLinkText.close
    > fileTempFile.close


    You should be using blocks here, to ensure the file is really closed.

    match = match.to_s
    match = match.gsub(/<.*?>/, "")
    match = match.gsub(/&[a-zA-Z0-9#]*?;/, "")

    link_text = "#{link}\n#{match}"

    File.open "#{file_path}full_#{file_name}.html", "w" do |temp_file|
    temp_file.puts response
    end

    File.open "#{file_path}relevant_#{file_name}.txt", "w" do |link_file|
    link_file.puts link_text
    end

    > As you can see, the files are being opened and closed almost
    > immediately.


    If an exception is raised, your File may not be closed if you don't use
    the block form.

    > Is there a different way I should be creating directories rather than
    > through a shell command?


    Dir.mkdir, and FileUtils has (I believe) mkpath as well.
    Eric Hodel, Dec 16, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Collin VanDyck

    (Too many open files)

    Collin VanDyck, Oct 13, 2003, in forum: Java
    Replies:
    5
    Views:
    2,854
    Collin VanDyck
    Oct 14, 2003
  2. Cathy  Hui
    Replies:
    6
    Views:
    9,036
    Wiseguy
    Mar 4, 2005
  3. bond
    Replies:
    1
    Views:
    3,175
    Mark Jeffcoat
    Jun 28, 2007
  4. bond
    Replies:
    3
    Views:
    22,387
  5. Dag Sunde
    Replies:
    4
    Views:
    344
Loading...

Share This Page