Timeout error using Find module

[Mike --]

Hey everyone,

I'm relatively new to Ruby, and am loving it! However, I'm getting some
trouble trying to implement a simple search application. The idea is to
search a directory (specified by the user) for a given reg. expression
in the filename (again specified by the user). Easy, really.

This works fine for the most part, but if I choose to search from the
root directory, I get a operation timeout error like so:

/usr/local/lib/ruby/1.8/find.rb:43:in `open': Operation timed out -
/net/broadcasthost (Errno::ETIMEDOUT)
from /usr/local/lib/ruby/1.8/find.rb:43:in `find'
from /usr/local/lib/ruby/1.8/find.rb:38:in `catch'
from /usr/local/lib/ruby/1.8/find.rb:38:in `find'
from ruby_find3.rb:10

I'm a little confused as to why it would timeout if it's simply
traversing the local directory tree? I would be grateful if anyone would
be able to give any assistance - I'm sure it's probably a simple fix!

Apologies if something like this has been discussed already...I have had
a look at some forum posts and have only been able to find
server-related timeout threads so far. I did learn about rescuing these
timeout errors from some threads, which I had a stab at implementing
briefly at the end.

Here's my code:

==========================
require 'find'
path = gets.chomp
search1 = gets.chomp
search_exp = Regexp.new(search1, Regexp::IGNORECASE)

puts "<<-- Beginning Search -->>"

begin
Find.find(path) do |p|
if FileTest.directory?(p)
if File.basename(p) =~ search_exp
puts "Dir: " + File.basename(p)

# Ignore directories beginning with "."
elsif File.basename(p)[0] == ?.
Find.prune
end
end

if FileTest.file?(p)
if (File.basename(p) =~ search_exp)
puts File.basename(p) + " : " + File.expand_path(p)
else
Find.prune
end
end
end
rescue SystemCallError
puts "Timeout error :S"
end

puts "<<-- End of Search -->"

 

[Robert Klemme]

I'm relatively new to Ruby, and am loving it! However, I'm getting some
trouble trying to implement a simple search application. The idea is to
search a directory (specified by the user) for a given reg. expression
in the filename (again specified by the user). Easy, really.

This works fine for the most part, but if I choose to search from the
root directory, I get a operation timeout error like so:

/usr/local/lib/ruby/1.8/find.rb:43:in `open': Operation timed out -
/net/broadcasthost (Errno::ETIMEDOUT)
from /usr/local/lib/ruby/1.8/find.rb:43:in `find'
from /usr/local/lib/ruby/1.8/find.rb:38:in `catch'
from /usr/local/lib/ruby/1.8/find.rb:38:in `find'
from ruby_find3.rb:10

I'm a little confused as to why it would timeout if it's simply
traversing the local directory tree? I would be grateful if anyone would
be able to give any assistance - I'm sure it's probably a simple fix!

From what you write I assume you are on a Unix style operating system
(single root directory). As the stack trace seems to reveal there is
some network communication going on ("/net/broadcasthost"). Apparently
there are some network drives mounted. Since you start in the root
directory they will be visited eventually. You can check with "mount"
or "df" what file systems are mounted on your system.

Apologies if something like this has been discussed already...I have had
a look at some forum posts and have only been able to find
server-related timeout threads so far. I did learn about rescuing these
timeout errors from some threads, which I had a stab at implementing
briefly at the end.

Here's my code:

==========================
require 'find'
path = gets.chomp
search1 = gets.chomp
search_exp = Regexp.new(search1, Regexp::IGNORECASE)

puts "<<-- Beginning Search -->>"

begin
Find.find(path) do |p|
if FileTest.directory?(p)
if File.basename(p) =~ search_exp
puts "Dir: " + File.basename(p)

# Ignore directories beginning with "."

This is completely superfluous because Find.find takes care of that (if
it would not it would *always* enter an infinite loop).

elsif File.basename(p)[0] == ?.
Find.prune
end
end

if FileTest.file?(p)
if (File.basename(p) =~ search_exp)
puts File.basename(p) + " : " + File.expand_path(p)
elsecat
Find.prune

Why do you prune here? What this basically means is that traversal will
stop on every directory that contains at least one file name that does
not match your regexp. Is this really what you want?

end
end
end
rescue SystemCallError
puts "Timeout error :S"
end

puts "<<-- End of Search -->"

Is this really the exact code that produced the error you present above?
I ask because you do not print out the exception itself but rather
the fixed string "Timeout error :S".

Some additional remarks: I usually try to avoid local variable 'p'
because of the method 'p' (print out #inspect string of an object) which
I frequently use for debugging.

You can make your code a bit more efficient by not repeating tasks,
namely the basename extraction.

This is how I would probably do it - assuming you want to find all files
and directories where the basename matches the given regexp.

require 'find'
dir = ARGV.shift or raise "Need a directory name"
rx = Regexp.new((ARGV.shift or raise "Need a regexp"),
Regexp::IGNORECASE)

Find.find dir do |f|
if rx =~ File.basename(f)
print File.directory?(f) ? "Dir : " : "File: ", f, "\n"
end
end

Kind regards

robert

 

[Mike --]

Hi Robert,

Thanks for your reply, much appreciated! Firstly, I'm running the code
on a Mac at the moment, hence the single root directory as your pointed
out.

I'm not sure about the network drive situation...the script does seem to
timeout and terminate once it reaches the folder '/net/'. I tried
navigating into this folder and discovered two entries - 'broadcasthost'
and 'localhost', both of which result in a timeout in the terminal
window when I try to access them. I know this isn't really related to
Ruby, but again any info people might have would be very appreciated.

Again, Robert, thanks for the info about not needing to chomp the
returned string from the user when using it with the Find module.

From what you write I assume you are on a Unix style operating system
(single root directory). As the stack trace seems to reveal there is
some network communication going on ("/net/broadcasthost"). Apparently
there are some network drives mounted. Since you start in the root
directory they will be visited eventually. You can check with "mount"
or "df" what file systems are mounted on your system.

I'm not sure about the network drive situation...the script does seem to
timeout and terminate once it reaches the folder '/net/'. I tried
navigating into this folder and discovered two entries - 'broadcasthost'
and 'localhost', both of which result in a timeout in the terminal
window when I try to access them. I know this isn't really related to
Ruby, but again any info people might have would be very appreciated.

This is completely superfluous because Find.find takes care of that (if
it would not it would *always* enter an infinite loop).

Again, Robert, thanks for the info about not needing to chomp the
returned string from the user when using it with the Find module.

elsif File.basename(p)[0] == ?.
Find.prune
end
end

if FileTest.file?(p)
if (File.basename(p) =~ search_exp)
puts File.basename(p) + " : " + File.expand_path(p)
elseif
Find.prune

Why do you prune here? What this basically means is that traversal will
stop on every directory that contains at least one file name that does
not match your regexp. Is this really what you want?

Yea, I think I know what you mean. No need for it really. I was playing
with something I saw on another web page whereby the programmer was
telling the Find script not to bother looking into Mac folders that
began with a '.', and I guess I just didn't think to remove it.

Is this really the exact code that produced the error you present above?
I ask because you do not print out the exception itself but rather
the fixed string "Timeout error :S".

Yep, that's the exact code I was using at the time of posting. I haven't
gotten round to finding out how to print out the details of the
exception so I just got it to tell me if an error occurred. I figured

Some additional remarks: I usually try to avoid local variable 'p'
because of the method 'p' (print out #inspect string of an object) which
I frequently use for debugging.

You can make your code a bit more efficient by not repeating tasks,
namely the basename extraction.

This is how I would probably do it - assuming you want to find all files
and directories where the basename matches the given regexp.

require 'find'
dir = ARGV.shift or raise "Need a directory name"
rx = Regexp.new((ARGV.shift or raise "Need a regexp"),
Regexp::IGNORECASE)

Find.find dir do |f|
if rx =~ File.basename(f)
print File.directory?(f) ? "Dir : " : "File: ", f, "\n"
end
end

I hate you. You've managed to take my 30something lines and make
reproduce it in 10. Thanks, good for me to see this sort of thing in
action so I can learn from it. Although I'm not running my code from the
command line so I wouldn't use the ARGV commands, but again good to know
about them. I do appreciate the help, I'm still at novice level of
programming so it's always good to see "best practice" code.

I reproduced the same code in Perl to perform the same search function
which didn't produce the same search error, so I guess that Perl does
not have an issue with the network mounts? Also, the Perl code seemed to
return significantly more results from the search when performed on my
Vista machine (used alongside the Ruby code), which seemed odd to me?
Not a huge issue, but still...

Thanks, all the best,

Mike

 

[Robert Klemme]

I'm not sure about the network drive situation...the script does seem to
timeout and terminate once it reaches the folder '/net/'. I tried
navigating into this folder and discovered two entries - 'broadcasthost'
and 'localhost', both of which result in a timeout in the terminal
window when I try to access them. I know this isn't really related to
Ruby, but again any info people might have would be very appreciated.

Yeah, some Mac wizzard should be able to answer you this. Since I'm
not a Mac user...

Again, Robert, thanks for the info about not needing to chomp the
returned string from the user when using it with the Find module.

Where exactly did I say this? I believe I did not and if, it would be wrong:

$ ruby -e 'p gets'
foo
"foo\n"

Yea, I think I know what you mean. No need for it really. I was playing
with something I saw on another web page whereby the programmer was
telling the Find script not to bother looking into Mac folders that
began with a '.', and I guess I just didn't think to remove it.

Ah, ok.

Yep, that's the exact code I was using at the time of posting. I haven't
gotten round to finding out how to print out the details of the
exception so I just got it to tell me if an error occurred. I figured

Hm, strange. Maybe the timeout is caught somewhere in #find and
printed to $stderr before another exception is thrown (if there is).

Btw, if you want to access the error you can do

rescue SystemCallError => e
puts e # or whatever, see docs

I hate you. You've managed to take my 30something lines and make
reproduce it in 10.

Well, not exactly. For one, the printout will be different.

Btw, if you do not need full regular expression support you can even
make it a near one liner:

puts Dir["#{base}/**/#{glob}"]

Thanks, good for me to see this sort of thing in
action so I can learn from it. Although I'm not running my code from the
command line so I wouldn't use the ARGV commands, but again good to know
about them. I do appreciate the help, I'm still at novice level of
programming so it's always good to see "best practice" code.

You're welcome! You'll have plenty of fun with Ruby, I'm sure.

I reproduced the same code in Perl to perform the same search function
which didn't produce the same search error, so I guess that Perl does
not have an issue with the network mounts?

Maybe Perl does some optimization (i.e. not reading those directories
on a Mac) or simply ignores the timeout.

Also, the Perl code seemed to
return significantly more results from the search when performed on my
Vista machine (used alongside the Ruby code), which seemed odd to me?
Not a huge issue, but still...

Are you sure both machines contain the same amount of files?

Kind regards

robert