Help: File.file?, ftype fails oddly

[Chris]

Hello all,

I've been working on a class+script to crawl through my directories and
thought I had it working just right, when it crashed some 13 minutes into
running. I included the stack at the bottom, in case it gives a better
hint to anyone here.

The code where it fails is this method - the if checks fail to see that
filename is a file, and even errors out on the else's File.ftype(filename=
)
call:

def get_dir_info_recursive change_into_dir, write_to_file, put_in_contain=
er
Dir.chdir( change_into_dir )
Dir.foreach(".") {|filename|
if File.directory?(filename) then
# write_to_file.puts stuff
# .. recursively call this method
elsif File.file?(filename) then
# write_to_file.puts stuff
# .. simple gather info and add to collection
else
puts "ERROR: TYPE=3D#{File.ftype(filename)} for #{filename} in
#{put_in_container.path}"
end
}
Dir.chdir('..')
end


My problem solving skills came up with a few alternatives:

1. Maybe the filename/directory path is too long. After all the path +
filename end up being right about 255 characters long.
NOPE: Tried starting the loop right in the directory where the file
errors out, even at earlier directories, the error doesn't happen -
the file is correctly identified as a file!

2. Maybe there's a problem with the output file? It is 11_892_316 bytes
at the error.
NOPE: Used irb to append another several k to the file with seek then
putc - no problem there.

3. Is there a hidden runtime problem?
RAM consumption? NOPE, I've run the script on larger, though not
deeper directories without it crashing. Though I would like to know
how to set the RAM usage. Anyone?

Stack problems? NOT purely stack level: simple recursive program
bombs at level 760 with 'stack level too deep'.
Other stack problems? Alternating recursion with each loops?
Maybe, since 20 directories deep causing 50+ levels in the stack.=20
And I am making either 1 or 2 new objects per call.

Ruby + NTFS + WinXP =3D ?occasional problem?


Any help and ideas are much appreciated.

Thanks,

-Chris

-----------------
Try #1: IF file? ELSE treat it as a directory
Starting file fdata_Q Mon Feb 06 22:40:22 Central Standard Time 2006
drive7.rb:118:in `chdir': Invalid argument -
_EJSRemoteStatelessCrystalItemReconControllerHome_Tie.java (Errno::EINVAL=
)
...
... 55 levels...
...
Try #2 with drive8: do directory? THEN file? THEN try printing File.ftype=
:
C:\CHRIS\ruby\MyFiles>ruby drive8.rb Q
Starting file fdata_Q Mon Feb 06 23:45:27 Central Standard Time 2006
Finished file fdata_Q Mon Feb 06 23:58:37 Central Standard Time 2006
drive8.rb:139:in `ftype': No such file or directory -
_EJSRemoteStatelessCrystalItemReconControllerHome_Tie.java (Errno:
:ENOENT)
from drive8.rb:139:in `get_dir_info_recursive'
from drive8.rb:122:in `foreach'
from drive8.rb:122:in `get_dir_info_recursive'
from drive8.rb:128:in `get_dir_info_recursive'
from drive8.rb:122:in `foreach'
from drive8.rb:122:in `get_dir_info_recursive'
from drive8.rb:128:in `get_dir_info_recursive'
from drive8.rb:122:in `foreach'
... 54 levels...
from drive8.rb:122:in `get_dir_info_recursive'
from drive8.rb:163
from drive8.rb:148:in `each'
from drive8.rb:148

 

[David Vallner]

D=C5=88a Utorok 07 Febru=C3=A1r 2006 18:15 Chris nap=C3=ADsal:
=2D-snip--

1. Maybe the filename/directory path is too long. After all the path +
filename end up being right about 255 characters long.
NOPE: Tried starting the loop right in the directory where the file
errors out, even at earlier directories, the error doesn't happen -
the file is correctly identified as a file!
=2D-snip--

Ruby + NTFS + WinXP =3D ?occasional problem?

Actually, I think NTFS does have some arbitrary limit on the path length. O=
r=20
the filename length. Or something. That said, I don't think it's quite=20
related, since you are #chdir-ing around, but who knows.

Try to rework the script to see if you occur it even if using File.find or=
=20
feeding the filenames via lessay Cygwin find, as opposed to recursing the=20
directories by hand?

David Vallner

David Vallner

 

[Daniel Berger]

Hello all,

I've been working on a class+script to crawl through my directories and
thought I had it working just right, when it crashed some 13 minutes into
running. I included the stack at the bottom, in case it gives a better
hint to anyone here.

The code where it fails is this method - the if checks fail to see that
filename is a file, and even errors out on the else's File.ftype(filename)
call:

def get_dir_info_recursive change_into_dir, write_to_file, put_in_container
Dir.chdir( change_into_dir )
Dir.foreach(".") {|filename|
if File.directory?(filename) then
# write_to_file.puts stuff
# .. recursively call this method
elsif File.file?(filename) then
# write_to_file.puts stuff
# .. simple gather info and add to collection
else
puts "ERROR: TYPE=#{File.ftype(filename)} for #{filename} in
#{put_in_container.path}"
end
}
Dir.chdir('..')
end


My problem solving skills came up with a few alternatives:

1. Maybe the filename/directory path is too long. After all the path +
filename end up being right about 255 characters long.
NOPE: Tried starting the loop right in the directory where the file
errors out, even at earlier directories, the error doesn't happen -
the file is correctly identified as a file!

YEP. You're hitting the MAX_PATH character limit. Try the following
sample code to see what I mean:

#initial = "\\\\\?\\C:\\really_long_test_path_to_be_deleted"
initial = "C:\\really_long_test_path_to_be_deleted"
Dir.mkdir(initial)
Dir.chdir(initial)

1.upto(40){ |n|
dir = "this_is_long_dir_number_#{n}"
file = "_some_really_long_file_#{n}.bogus"

File.open(file, "w+"){} # touch
puts "Created file: #{file}"

Dir.mkdir(dir)
puts "Created dir: #{dir}"

Dir.chdir(dir)
puts "Changed into dir: #{dir}"

puts "Current dir length: " + Dir.pwd.length.to_s
puts
}

And here's the output:

C:\eclipse\workspace\ruby-foo>ruby deepdir.rb
Created file: _some_really_long_file_1.bogus
Created dir: this_is_long_dir_number_1
Changed into dir: this_is_long_dir_number_1
Current dir length: 68

Created file: _some_really_long_file_2.bogus
Created dir: this_is_long_dir_number_2
Changed into dir: this_is_long_dir_number_2
Current dir length: 94

Created file: _some_really_long_file_3.bogus
Created dir: this_is_long_dir_number_3
Changed into dir: this_is_long_dir_number_3
Current dir length: 120

Created file: _some_really_long_file_4.bogus
Created dir: this_is_long_dir_number_4
Changed into dir: this_is_long_dir_number_4
Current dir length: 146

Created file: _some_really_long_file_5.bogus
Created dir: this_is_long_dir_number_5
Changed into dir: this_is_long_dir_number_5
Current dir length: 172

Created file: _some_really_long_file_6.bogus
Created dir: this_is_long_dir_number_6
Changed into dir: this_is_long_dir_number_6
Current dir length: 198

Created file: _some_really_long_file_7.bogus
Created dir: this_is_long_dir_number_7
Changed into dir: this_is_long_dir_number_7
Current dir length: 224

Created file: _some_really_long_file_8.bogus
deepdir.rb:14:in `mkdir': No such file or directory -
this_is_long_dir_number_8 (Errno::ENOENT)
from deepdir.rb:14
from deepdir.rb:7:in `upto'
from deepdir.rb:7

Note that even adding the leading "\\?\" to the directory doesn't seem
to help (though it should).

This isn't the best error message, but that could be Microsoft's fault
- I haven't checked to see what error is returned for this case using C
and GetLastError().

Regards,

Dan