Managing "requires" in projects with many subdirectories

[jeffz_2002]

I'd like to know how people people managing their file dependencies
with projects with several directories. Described below is the
problem, and a demo of a possible rudimentary solution for which I'd
like feedback/discussion. (I've searched and can't find any notes for
handling requires elegantly. It could be that I'm searching in the
wrong place ... please point me in the right direction if yes)

I've been working on a small project with several directories. Using
the $:.unshift File.join( ( File.expand_path( File.dirname( __FILE__ )
) ), '..', '..', 'blah' ) trick gets pretty boring to type, doesn't do
much for the code readability, and doesn't help when files get moved
around.

Side note: As a relic of my C++ interest, I also like my requires to be
self-contained so my files are independent (e.g., if A uses B and C,
I'll require B and C in A, even if B requires C ... that way, if the
require 'C' is removed from B, A will still work).


Here's the quick demo of one possible solution for handling requires
for a project with multiple subdirectories:

in root:
--------

** File add_dirs_to_search_path.rb:

require 'find'
Find.find( File.expand_path( File.dirname( __FILE__ ) ) ) do |path|
if FileTest.directory?(path)
$:.unshift path
end
end

in root/A:
----------

** File a_thing.rb:

module A
class AThing; def go() "A go!"; end; end
end

in root/B:
----------

** File a_client.rb:

require 'a_thing'
module B
class BThing; def go() "B go: #{A::AThing.new.go}"; end; end
end


Back in root:
-------------

** File main.rb:

$:.unshift File.expand_path( File.dirname( __FILE__ ))
require 'add_dirs_to_search_path'
require 'a_client'

b = B::BThing.new
puts b.go


Result when called from root:

$ ruby main.rb
B go: A go!


Notes, advantages, drawbacks:

- Module names mimic the directory structure (per the code conventions
at RWiki)

- B only has to use "require 'a_thing', since a_thing's directory was
pushed into the include search path

- Any code that uses B must now have specified the search path to
a_thing.rb somehow, either using the -I command-line directive, an
implicit require (requiring some code that requires A), or by requiring
add_dirs_to_search_path.rb

- main has to have "$:.unshift File.expand_path( File.dirname( __FILE__
)); require 'add_dirs_to_search_path'". So would any test suite, etc.

- Directory changes are easily handled. You can move A::AThing to a
different directory (note that this change should probably be
accompanied by a change in module name, per RWiki code conventions)

- To disambiguate the desired file, include the directory. For
example, if root/C also contains a file a_thing.rb, and you wanted to
use *that* file in main, you'd change the require in main to "require
File.join( 'C', 'a_thing.rb' )".

- The require gets mixed up if different folders with the same name
contain files with the same name. For example, if there's another file
A/C/a_thing.rb, and the module definition matches the directory
structure (module A; module C; class AThing; ...), the require *can*
bomb since the wrong file might be included.

- in add_dirs_to_search_path.rb, I'm dynamically finding the files, but
you could also hardcode them:

dirs = [ 'a',
'a|suba',
'b',
'b|subb',
'c' ]
root = File.expand_path( File.dirname( __FILE__ ) )
dirs.each do |d|
$:.unshift "#{root}|#{dirs}".gsub( '|', File::SEPARATOR )
end


This whole idea might be kind of silly ... it might be better to be
explicit when including files (like you have to do in Java ... "import
com.x.my.utilities.rock.Thingy"), and to always fully specify modules
whenever you're referring to a class.

Sorry for the long post - but I'd like to hear any good strategies for
managing requires. Thanks,

Jeff

 

[Eric Hodel]

I'd like to know how people people managing their file dependencies
with projects with several directories. [...]

I've been working on a small project with several directories. Using
the $:.unshift File.join( ( File.expand_path( File.dirname( __FILE__ )
) ), '..', '..', 'blah' ) trick gets pretty boring to type, doesn't do
much for the code readability, and doesn't help when files get moved
around.

require 'find'
Find.find( File.expand_path( File.dirname( __FILE__ ) ) ) do |
path|
if FileTest.directory?(path)
$:.unshift path
end
end

This'll work great right up until you have something like myproj/
time.rb and you also need time.rb from stdlib.

Instead, use explicit paths like everybody else does, it'll be less
confusing.

 

[Trans]

I'd like to know how people people managing their file dependencies
with projects with several directories. Described below is the
problem, and a demo of a possible rudimentary solution for which I'd
like feedback/discussion. (I've searched and can't find any notes for
handling requires elegantly. It could be that I'm searching in the
wrong place ... please point me in the right direction if yes)

I've been working on a small project with several directories. Using
the $:.unshift File.join( ( File.expand_path( File.dirname( __FILE__ )
) ), '..', '..', 'blah' ) trick gets pretty boring to type, doesn't do
much for the code readability, and doesn't help when files get moved
around.

Side note: As a relic of my C++ interest, I also like my requires to be
self-contained so my files are independent (e.g., if A uses B and C,
I'll require B and C in A, even if B requires C ... that way, if the
require 'C' is removed from B, A will still work).


Here's the quick demo of one possible solution for handling requires
for a project with multiple subdirectories:

in root:
--------

** File add_dirs_to_search_path.rb:

require 'find'
Find.find( File.expand_path( File.dirname( __FILE__ ) ) ) do |path|
if FileTest.directory?(path)
$:.unshift path
end
end

in root/A:
----------

** File a_thing.rb:

module A
class AThing; def go() "A go!"; end; end
end

in root/B:
----------

** File a_client.rb:

require 'a_thing'
module B
class BThing; def go() "B go: #{A::AThing.new.go}"; end; end
end


Back in root:
-------------

** File main.rb:

$:.unshift File.expand_path( File.dirname( __FILE__ ))
require 'add_dirs_to_search_path'
require 'a_client'

b = B::BThing.new
puts b.go


Result when called from root:

$ ruby main.rb
B go: A go!


Notes, advantages, drawbacks:

- Module names mimic the directory structure (per the code conventions
at RWiki)

- B only has to use "require 'a_thing', since a_thing's directory was
pushed into the include search path

- Any code that uses B must now have specified the search path to
a_thing.rb somehow, either using the -I command-line directive, an
implicit require (requiring some code that requires A), or by requiring
add_dirs_to_search_path.rb

- main has to have "$:.unshift File.expand_path( File.dirname( __FILE__
)); require 'add_dirs_to_search_path'". So would any test suite, etc.

- Directory changes are easily handled. You can move A::AThing to a
different directory (note that this change should probably be
accompanied by a change in module name, per RWiki code conventions)

- To disambiguate the desired file, include the directory. For
example, if root/C also contains a file a_thing.rb, and you wanted to
use *that* file in main, you'd change the require in main to "require
File.join( 'C', 'a_thing.rb' )".

- The require gets mixed up if different folders with the same name
contain files with the same name. For example, if there's another file
A/C/a_thing.rb, and the module definition matches the directory
structure (module A; module C; class AThing; ...), the require *can*
bomb since the wrong file might be included.

- in add_dirs_to_search_path.rb, I'm dynamically finding the files, but
you could also hardcode them:

dirs = [ 'a',
'a|suba',
'b',
'b|subb',
'c' ]
root = File.expand_path( File.dirname( __FILE__ ) )
dirs.each do |d|
$:.unshift "#{root}|#{dirs}".gsub( '|', File::SEPARATOR )
end


This whole idea might be kind of silly ... it might be better to be
explicit when including files (like you have to do in Java ... "import
com.x.my.utilities.rock.Thingy"), and to always fully specify modules
whenever you're referring to a class.

Sorry for the long post - but I'd like to hear any good strategies for
managing requires. Thanks,

You mention a root/ so that makes me think you're running directly from
your work directory. It's much easier to organize your project into a
standard layout and then use setup.rb to reinstall between coding
cycles. See http://i.loveruby.net/en/projects/setup/doc/. You could
also use RubyGems in this manner, but it is not as convenient when
coding.

Having said that I do have a lib that does the trick -- I've had it for
a while, but I haven't quite finsihed tweaking it out for release (I'm
adding a remote require feature to it). But it works fine for the most
part, and I sure could use another's input to get it to final release
state. If you (or anyone else) is interested let me know.

T.

 

[Daniel DeLorme]

This'll work great right up until you have something like myproj/time.rb
and you also need time.rb from stdlib.

Instead, use explicit paths like everybody else does, it'll be less
confusing.

Like everybody else except rails... which adds about 41 entries to the
$LOAD_PATH

Daniel

 

[Jan Svitok]

I'd like to know how people people managing their file dependencies
with projects with several directories. [...]

I've been working on a small project with several directories. Using
the $:.unshift File.join( ( File.expand_path( File.dirname( __FILE__ )
) ), '..', '..', 'blah' ) trick gets pretty boring to type, doesn't do
much for the code readability, and doesn't help when files get moved
around.

require 'find'
Find.find( File.expand_path( File.dirname( __FILE__ ) ) ) do |
path|
if FileTest.directory?(path)
$:.unshift path
end
end

This'll work great right up until you have something like myproj/
time.rb and you also need time.rb from stdlib.

Instead, use explicit paths like everybody else does, it'll be less
confusing.

I agree with Eric. We used to add all directories recusively to the
search path, and used relative paths without directories. It was nice
and all but:

- you need to keep the filenames unique. I was bitten by this when
suddenly I used a filename of a file from unrelated project, that got
on the search path somehow (it was located in the parent directory of
my stuff). It took me some time until I realized what's going on.

- if you give the code to another person, he/she can't tell from the
source where is the required file located. he/she has to figure out by
serching the whole tree.

- if you mix these two approches, you may and up with some files
required more times, which may or may not be good.

Now I'm slowly going back to requires with relative paths to 'lib'
directory, that gets on the search path from the main file or command
line.

NB: in the
$:.unshift File.join( File.expand_path( File.dirname( __FILE__ )) ,
'..', '..', 'blah' )
I would reverse expand_path and join, to remove those .. elements, i.e.
$:.unshift File.expand_path( File.join( File.dirname( __FILE__ ),
'..', '..', 'blah' ))

 

[James Edward Gray II]

Like everybody else except rails... which adds about 41 entries to
the $LOAD_PATH

And I believe the salted login generator for Rails once had the exact
conflict Eric describes.

James Edward Gray II

 

[jeffz_2002]

Thanks for the notes ... one final question:

Now I'm slowly going back to requires with relative paths to 'lib'
directory, that gets on the search path from the main file or command
line.

Does this mean that your files would, say, look like this:

(file a.rb)
require 'some/thing'

and then the command line becomes:

$ ruby -Ipath/to/lib my_file.rb ?

The reason I ask is that you still need to pull in the path/to/lib
somewhere, and including it in the -I param means that subsequent devs
will need to know this (yes, I know it's minor, but I'd want others to
be able to cd to /tests and just write "ruby ts_all.rb", or "ruby
tc_specific_test_case.rb" in any subdirectory ... trying to make life
easy for them, cause I'm such a great guy).

Also, minor point, but path separators are different in different OSs,
aren't they? So don't you need to do something like:

(file a.rb)
require File.join('some', 'thing')

which also becomes tiresome? Or does the ruby interpreter take care of
that? Sorry, I don't have another machine to try this out on, I only
have 'doze.

NB: in the
$:.unshift File.join( File.expand_path( File.dirname( __FILE__ )) ,
'..', '..', 'blah' )
I would reverse expand_path and join, to remove those .. elements, i.e.
$:.unshift File.expand_path( File.join( File.dirname( __FILE__ ),
'..', '..', 'blah' ))

Yes, that's better, thanks.


Thanks for putting up with my inane questions.

jz

 

[Eric Hodel]

Thanks for the notes ... one final question:


Does this mean that your files would, say, look like this:

(file a.rb)
require 'some/thing'

and then the command line becomes:

$ ruby -Ipath/to/lib my_file.rb ?

At worst I use ruby -Ilib my_file. Rarely is it way over there in
path/to. (When it is, I set up a rake rule.)

The reason I ask is that you still need to pull in the path/to/lib
somewhere, and including it in the -I param means that subsequent devs
will need to know this (yes, I know it's minor, but I'd want others to
be able to cd to /tests and just write "ruby ts_all.rb", or "ruby
tc_specific_test_case.rb" in any subdirectory ... trying to make life
easy for them, cause I'm such a great guy).

Nobody needs to remember anything if you write it down in a rake
rule. Automate, automate, automate.

PS: the Test::Unit convention is tests live in test/ and
implementations live in lib/ and test files start with test_. This
makes working with the standard tools (like testrb) and
interoperating with other tools (like rake and ZenTest) a breeze.

Also, minor point, but path separators are different in different OSs,
aren't they?

Ruby is smart enough to do what you mean.