Automatically Determining "Requires" and "Provides" information for a Ruby script or library

Discussion in 'Ruby' started by Scott Parkerson, Jan 19, 2007.

  1. I'm looking to write a script that examines one or more scripts
    written in Ruby and programatically determine the following:

    * What does each script provide? Specifically, what classes are
    defined, whether they are in a module namespace, etc.

    * What does the script require to operate (i.e. dependencies on other
    Ruby scripts)

    I need this because I am looking into writing Ruby support into the
    Conary packaging system (c.f. http://wiki.rpath.com/wiki/Conary).
    Conary already automatically generates provides/requires information
    when packaging scripts for other popular languages (e.g. Perl, Python,
    Java), so I think that having Ruby support would be A Good Thing.

    Ideally, I should be able to 100% accurately identify what the script
    provides, and *mostly* identify requires for dependencies. I say
    mostly because many scripts use tricks to load plugins that involve
    using the module's filename and path examined at runtime. The main
    thing is that there are no "false provides/requires" returned by my
    script.

    I would like the solution to not require examining gemspecs, or even
    require gems at all.

    If anyone has any ideas about how to go about doing this, please
    respond here. Let's talk!

    Thanks in advance,
    Scott
    --
    scott parkerson. geek, erstwhile prophet and fool.
    just a cog in the machinery: http://www.smerpology.org/sprocket/
     
    Scott Parkerson, Jan 19, 2007
    #1
    1. Advertising

  2. Scott Parkerson

    Guest

    On Jan 19, 2007, at 9:57 AM, Scott Parkerson wrote:
    > Ideally, I should be able to 100% accurately identify what the script
    > provides, and *mostly* identify requires for dependencies.


    I don't think this is technically possible because of the dynamic
    nature of Ruby. That is to say that I think the task you describe
    is equivalent to the halting problem as it requires the ability to
    divine the intent of executable code by simply examining the code
    as opposed to running it and seeing what happens.

    There are certainly heuristics you could use to get pretty close to
    100% for 'normal' code (searching for 'class X' and for 'require...'),
    but you'll never get to 100% and you'll probably have to re-implement
    a good portion of the Ruby parser in the process (i.e. to deal with
    nested classes/modules). That may be good enough for your needs though.

    Maybe there is something that already does that out there? Googling...

    Check out
    http://www.zenspider.com/ZSS/Products/ParseTree/index.html and
    http://www.zenspider.com/ZSS/Products/ParseTree/Examples/
    Dependencies.html
    maybe that will give you some ideas.


    Gary Wright
     
    , Jan 19, 2007
    #2
    1. Advertising

  3. On 1/19/07, <> wrote:

    > I don't think this is technically possible because of the dynamic
    > nature of Ruby. That is to say that I think the task you describe
    > is equivalent to the halting problem as it requires the ability to
    > divine the intent of executable code by simply examining the code
    > as opposed to running it and seeing what happens.


    I think you are right, so let me amend my request a bit.

    * I don't need to get the exact signature of every method provided or
    required. File-based provides should be enough for what we need.
    Anything more complex can be synthesized manually. In most cases, Ruby
    provide would essentially be the filepath, with the $LOAD_PATH chopped
    off the front and the extension removed. Thus,
    /usr/lib/ruby/1.8/yaml.rb provides 'yaml'.

    * Requires could be whatever was required at require time. I wrote a
    quick and dirty requires generator that essentially overrode
    Kernel.require to stuff the argument to require into a Set. Thus, for
    yaml:

    /usr/lib/ruby/1.8/yaml.rb requires the following modules:
    "yaml/constants"
    "yaml/ypath"
    "yaml/error"
    "yaml/rubytypes"
    "stringio"
    "date"
    "rational"
    "syck"
    "yaml/syck"
    "yaml/basenode"
    "date/format"
    "yaml/tag"
    "yaml/stream"
    "yaml/types"

    The big question is whether Ruby C extensions are always required by
    filename (i.e. if you have a C extension called big/foo.so, require
    'big/foo' will load it). In Python, this is tricky, as the shared
    library name may not be the thing you use with import at all.

    The bottom line is to have a "good enough" provides/requires mechanism
    that automates packaging information. It's obviously not perfect, as
    two foo.rb's might do wildly different things.

    Here's the code snippet, so far (quick and dirty is probably a vast
    understatement):

    require 'set'
    $required = Set.new

    module Kernel
    alias_method :eek:ld_require, :require
    def require(m)
    begin
    result = old_require(m)
    rescue LoadError => blargh
    print "warning: #{blargh}\n"
    rescue NameError
    true
    end
    $required = $required.add(m)
    result
    end
    end

    require(ARGV[0])

    print "#{ARGV[0]} requires the following modules:\n"
    $required.each { |file| p file }


    --
    scott parkerson. geek, erstwhile prophet and fool.
    just a cog in the machinery: http://www.smerpology.org/sprocket/
     
    Scott Parkerson, Jan 19, 2007
    #3
  4. Scott Parkerson

    Zev Blut Guest

    Hello,

    On Fri, 19 Jan 2007 23:57:54 +0900, Scott Parkerson
    <> wrote:

    > I'm looking to write a script that examines one or more scripts
    > written in Ruby and programatically determine the following:


    > * What does the script require to operate (i.e. dependencies on other
    > Ruby scripts)


    I have a tool in the kwala project on rubyforge that attempts to
    determine this. It uses the Java prefuse library to display a dynamic
    graph for inspection, or if you don't want the Java dependency you can
    have it output a static graphviz graph. It also does a few other
    things like find require cycles. If you are interested take a look in
    the cycle_detector.rb file.

    You can find it here:
    http://kwala.rubyforge.org/

    I hope that helps,
    Zev
     
    Zev Blut, Jan 22, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gene
    Replies:
    2
    Views:
    2,889
    Saravana [MVP]
    Apr 19, 2004
  2. Damodar Periwal
    Replies:
    0
    Views:
    313
    Damodar Periwal
    Feb 29, 2004
  3. lovecreatesbeauty
    Replies:
    2
    Views:
    423
    codigo
    Apr 25, 2005
  4. Anna Sunny
    Replies:
    0
    Views:
    4,966
    Anna Sunny
    Aug 20, 2006
  5. shinya
    Replies:
    1
    Views:
    620
    Stefan Ram
    Apr 21, 2008
Loading...

Share This Page