[ANN] Ruth 0.10

[Robert Feldt]

Here is a new version of ruth. Please
see README below for more info.

Regards,

--
Robert Feldt


==============================
= Ruth - Ruby Under The Hood =
= version 0.10 =
==============================

Date: $Date: 2003-08-22 11:17:26 +0200 (Fri, 22 Aug 2003) $
Version: 0.10
Svn revision: $Rev: 36 $
Available from: http://pronovomundo.dyndns.org/temp/ruth
Author: Robert Feldt, (e-mail address removed)

NOTE! The link is temporary until I get my server setup
properly.

NOTE! I don't use the CVS on sourceforge anymore so the code in the CVS
repository there is OBSOLETE! Since the version on sf has wrongly been
assigned the version number
0.8.0 I jumped to version 0.9 at one point even though its not
yet very close to a 1.0 release.

What is it?
-----------
Ruby extension giving classes and functions to access Ruby internals.

Currently includes:
* ruby/interpreter/parse, Ruby::Interpreter.parse Gives a tree
representing the internal parse tree used to execute
a Ruby program.
* ruby/interpreter/parse, Ruby::Interpreter.method_body
Returns the tree for the body of a method.

And an incomplete:
* ruby/parse, Ruby.parse
Parse Ruby source code into an abstract syntax tree.
Its a cheato-RubyInRuby parser since it looks like one but actually
uses Ruby::Interpreter.parse and then simply transforms the output.
Advantage of this is that it parses exactly the same programs as matz Ruby
interpreter parses (since it uses the same parser). Disadvantage is that it
relies on C code, returns nil on parse error,
isn't customizable etc.

Note that the Ruby.parse can never be "complete" since irrelevant info
is dropped by matz interpreter and is not available in the internal
parse tree (and thus cannot be "resurrected" later). An example is

def m(a)
1
end

in which the internal parse tree does not contain any reference to the
argument named "a" since it is never used (at least this was the behavior
least time I looked which was a couple of months back so it might have
changed...). All information that is relevant to executing the code must be
available though so should be enough for many uses.

Why?
----
To learn about MRI (Matz Ruby Interpreter) internals its good to be able to
dump and view its internal representation of Ruby programs.

To write tools that work with Ruby source code its good to have a parser.
To develop a parser its good to have a reference to compare to; Ruby.parse
is a good such reference (once its completed).

Installation?
-------------
1. unpack tarball (if you haven't already)
2. ruby helpers/make.rb
3. ruby helpers/install.rb
4. ruby tests/run_all_tests.rb (optional)

Example of use?
---------------
require 'ruby/parse'
p Ruby.parse("a = 1") # => LocalAssign[:a, IntegerLiteral[1]]

Requirements?
-------------
ANSI C compiler and Ruby.

I've successfully compiled and used Ruth with * Ruby 1.8.0 (2003-08-04) +
cygwin 1.3.22 (gcc version 3.2 20020927) on Windows XP Pro.
* Ruby 1.8.0 (2003-08-04) + gcc 3.2.3 20030422 on Linux 2.4.22 (Gentoo)

If it works for you on other platforms/setups I'd appreciate if you drop me
a note. However, it should work on most Ruby-enabled platforms having a C
compiler.

NOTE THAT THIS IS AN ALPHA RELEASE SO THERE WILL LIKELY BE BUGS.

Please report bugs to (e-mail address removed)

Documentation?
--------------
None yet.

License and legal issues?
-------------------------
All files in this package are coyrighted free software

Copyright (c) 2001, 2002, 2003 Robert Feldt, (e-mail address removed)
All rights reserved.

and they are distributed under LGPL. See LICENSE.

Acknowledgements for ruth/mri should go to Dave Thomas and Andy Hunt since
I've basically generalized their NodeDump extension.

I'd also like to thank George Marrows and Rich Kilmer for patches and
encouragement.

Special things to note?
-----------------------
Most of this package is highly implementation specific. If matz changes the
internals of the interpreter things might stop working. Beware!

There might be safety issues with accessing the internals...

There are known bugs (for example the line number counting is faulty)
and the tests are *VERY* incomplete. This was a hack that has grown so
(unfortunately) I didn't do any test-first development. I've since
learned that its always a good idea to write things test-first...

Plans for the future?
---------------------
See TODO. This is an alpha release so there will probably be changes to the
API.

Do you have comments or questions?
----------------------------------
I'd appreciate if you drop me a note if you're successfully using Ruth. If
there are some known users I'll be more motivated to packing up additions /
new versions and post them to RAA.

Happy coding!

Robert Feldt, (e-mail address removed)

 

[ts]

R> Note that the Ruby.parse can never be "complete" since irrelevant info
R> is dropped by matz interpreter and is not available in the internal
R> parse tree (and thus cannot be "resurrected" later). An example is

R> def m(a)
R> 1
R> end

I've not understood

svg% ruby -riis -e 'puts dump; def m(a) 1 end'
puts(dump)

def m(a)
1
end
svg%

I was able to re-build `def m'


Guy Decoux

 

[Robert Feldt]

I've not understood
svg% ruby -riis -e 'puts dump; def m(a) 1 end'
puts(dump)
def m(a)
1
end
svg% I was able to re-build `def m'

Yes, this is from an older version of ruth. Maybe it had
a bug back then or I didn't understand it. Now it works:

feldt@novomundo1 /tmp/development/ruth/trunk
$ ruby tests/parse_ruby_file.rb 'def m(a); 1; end'
Def
:m
[:a]
IntegerLiteral
1

Regards,

Robert

 

[MikkelFJ]

And an incomplete:
* ruby/parse, Ruby.parse
Parse Ruby source code into an abstract syntax tree.
Its a cheato-RubyInRuby parser since it looks like one but actually
uses Ruby::Interpreter.parse and then simply transforms the output.
Advantage of this is that it parses exactly the same programs as matz Ruby
interpreter parses (since it uses the same parser). Disadvantage is that it
relies on C code, returns nil on parse error,
isn't customizable etc.

This could be useful for prototyping byte code generators in Ruby for Parrot
and similar targets.


Mikkel

 

[mark]

I've not understood
svg% ruby -riis -e 'puts dump; def m(a) 1 end'
puts(dump)
def m(a)
1
end
svg% I was able to re-build `def m'

Yes, this is from an older version of ruth. Maybe it had
a bug back then or I didn't understand it. Now it works:

feldt@novomundo1 /tmp/development/ruth/trunk
$ ruby tests/parse_ruby_file.rb 'def m(a); 1; end'
Def
:m
[:a]
IntegerLiteral
1

However if you try something like

[mark@laptop ruth-0.10]$ ruby tests/parse_ruby_file.rb 'def m(a); x=1; end'
Def
:m
[:a, :x]
LocalAssign
:x
IntegerLiteral
1

So it seems that the array [:a, :x] is a list of all local variables used in
the function.

Is there any way to find out what parameters the function takes?

Regards,

Robert

Best Regards

Mark Sparshatt

 

[Robert Feldt]

However if you try something like

[mark@laptop ruth-0.10]$ ruby tests/parse_ruby_file.rb 'def m(a); x=1;
end'
Def
:m
[:a, :x]
LocalAssign
:x
IntegerLiteral
1

So it seems that the array [:a, :x] is a list of all local variables used
in the function.

Is there any way to find out what parameters the function takes?

There transformation from internal to AST is very sketchy/incomplete
as of now but if you check the internal tree:

$ ruby tests/parse_ruby_file.rb 'def m(a); x = 1; end' internal
Internal tree:
Newline(Defnm
Scope([:a, :x]
Block(Args(1
-1
nil)
Block(Newline(Lasgn(3
:x
Lit(1)))
nil)))))

You can see the info is there (1 arg to the block that is
the body => first in the scope => a) so its a bug
in the transformation.

anyway, I'm offline for some time now so will be back
with fixes at a later time.

Regards,

Robert