Ruby to Parrot compiler

[Mark]

Is there anyone working on a compiler for compiling Ruby code to work on
parrot?

The latest effort Cardinal seemed to die off before it got started,
presumably due to the immaturity of Parrot.

However version 0.1.0 of Parrot is set to be released on the 29th of
this month wich should include support for objects, so this seems to be
the ideal time to get a compiler project started up.

Cardinal was set up on Savannah but this seems to be a perfect project
to start up on Rubyforge.

Is anyone else interested in this?

Best Regards

Mark Sparshatt

 

[gabriele renzi]

il Fri, 20 Feb 2004 04:45:37 +0900, Mark <[email protected]> ha
scritto::

Is anyone else interested in this?

I'd love to see that, but I really don't think I can help ,even if dan
sugalsky says that compilers are easy[1]
PDD 15 (objects) is done IIRC , so a ruby compiler could be started
prolly


http://www.sidhe.org/~dan/blog/archives/000304.html

 

[Phil Tomson]

Is there anyone working on a compiler for compiling Ruby code to work on
parrot?

The latest effort Cardinal seemed to die off before it got started,
presumably due to the immaturity of Parrot.

However version 0.1.0 of Parrot is set to be released on the 29th of
this month wich should include support for objects, so this seems to be
the ideal time to get a compiler project started up.

So they're promising object support at the end of the month? I've
heard this for almost two years now. I'll believe it when I see it.

Cardinal was set up on Savannah but this seems to be a perfect project
to start up on Rubyforge.

Is anyone else interested in this?

I'm very interested in this. I was the one who setup the project at
Savannah (now I don't even know where I put the passwords for it, it's
been so long ago). I definately want to see a Ruby front-end for
Parrot. Back when I started the project I wasn't working. Since then
I've been going to grad school and more recently I've gotten a job and
I'm still going to grad school so I really don't have a lot of time to
commit to this.

Please feel free start a Cardinal project on Rubyforge. Even if
Object support in Parrot doesn't come out this month, it will
_eventually_ happen(at least I hope so . In the meantime there are
lots of other tasks that could be done. The foremost being that we
need a good parser for Ruby (Cardinal isn't the only project that
needs this). I just saw another post that mentioned Ripper again.
Might be worth a try. There's also Ruth. At any rate, I think the
main focus right now needs to be on the frontend until the backend
issues (object support in Parrot) are ironed out.

Phil

 

[Mark]

So they're promising object support at the end of the month? I've
heard this for almost two years now. I'll believe it when I see it.

AFAIK, The spec for the object support has been written and Dan has
promised to check a basic implementation in to the source tree in time
for the feature freeze monday.

I'm very interested in this. I was the one who setup the project at
Savannah (now I don't even know where I put the passwords for it, it's
been so long ago). I definately want to see a Ruby front-end for
Parrot. Back when I started the project I wasn't working. Since then
I've been going to grad school and more recently I've gotten a job and
I'm still going to grad school so I really don't have a lot of time to
commit to this.

Please feel free start a Cardinal project on Rubyforge. Even if
Object support in Parrot doesn't come out this month, it will
_eventually_ happen(at least I hope so . In the meantime there are
lots of other tasks that could be done. The foremost being that we
need a good parser for Ruby (Cardinal isn't the only project that
needs this). I just saw another post that mentioned Ripper again.
Might be worth a try. There's also Ruth. At any rate, I think the
main focus right now needs to be on the frontend until the backend
issues (object support in Parrot) are ironed out.

I've tried out Ripper and it seems to be very close to what we need for
a front end. Though I'll also take a look at Ruth.

There are AFAIK at least two projects that convert Ruby code into
bytecode, I wonder how easy it would be to modify them so they emit
parrot bytecode? I admit I'm not clear about how they work.

I agree that the front end is the best place to start working on the
project.

 

[Mark]

AFAIK, The spec for the object support has been written and Dan has
promised to check a basic implementation in to the source tree in time
for the feature freeze monday.

I've tried out Ripper and it seems to be very close to what we need
for a front end. Though I'll also take a look at Ruth.

Does anyone know where I can find Ruth? I've tried looking on Rubyforge
and RAA but haven't been able to find it.

Thanks in Advance.

 

[Mark Hubbart]

Does anyone know where I can find Ruth? I've tried looking on
Rubyforge and RAA but haven't been able to find it.
Thanks in Advance.

It is part of the RubyVM project, on sourceforge.
http://sourceforge.net/project/showfiles.php?group_id=26000

 

[Phil Tomson]

It is part of the RubyVM project, on sourceforge.
http://sourceforge.net/project/showfiles.php?group_id=26000

But that appears to be an old version. A newer version was announced last
August (http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/79936)
but it's not available at the location given in that link.

Phil

 

[daz]

But that appears to be an old version. A newer version was announced last
August (http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/79936)
but it's not available at the location given in that link.

Phil

Still temporary, but this worked just now.

V 0.10

http://www.pronovomundo.com/ruth/


daz

 

[Mark]

:



Still temporary, but this worked just now.

V 0.10

http://www.pronovomundo.com/ruth/

Thanks, that link works. Though the file linked there is misnamed. It
has the extension .tar.gz even though it's just a tar file.

It seems that the choice for a front end is between Ripper and Ruth.
Both'll need a bit of development before they can be used, and neither
is documented.

Personally I think Ruth is the better choice since it has fewer
dependencies (Ripper needs bison and gperf) and it's interface seems to
be simpler.

Though it has the problem that when parsing a method definition using
the standard parse method there's no way to find out what parameters the
methods takes. However I think there might be lower level methods which
can be used in this case.


ISTM the best bet would be to design the compiler so that the parser can
be swapped easily. Looking at the archives for Cardinal I noticed there
was talk of Rite having a seperate parser component. Being able to use
that as a parser would be very useful.


I'm going to set up a project on Rubyforge in order to work on this.

 

[Alexander Kellett]

Still temporary, but this worked just now.

V 0.10

http://www.pronovomundo.com/ruth/


daz

patch to make it compile for my 1.8.1.

diff -uir ruth-0.10/ext/ruby/interpreter/node_specifications.rb
ruth-0.10-mod/ext/ruby/interpreter/node_specifications.rb
--- ruth-0.10/ext/ruby/interpreter/node_specifications.rb 2003-08-22
14:37:49.000000000 +0200
+++ ruth-0.10-mod/ext/ruby/interpreter/node_specifications.rb 2004-02-21
12:58:25.000000000 +0100
@@ -121,8 +121,8 @@
if RUBY_VERSION >= "1.7"
# REXPAND seems not to be in Ruby 1.8 anymore as of 2003-04-15
# NODE_SPECIFICATIONS["REXPAND"] = [[:head, :RNode]]
- NODE_SPECIFICATIONS["RESTARY"] = [[:head, :RNode]]
- NODE_SPECIFICATIONS["RESTARY2"] = [[:head, :RNode]]
+ # NODE_SPECIFICATIONS["RESTARY"] = [[:head, :RNode]]
+ # NODE_SPECIFICATIONS["RESTARY2"] = [[:head, :RNode]]
NODE_SPECIFICATIONS["SVALUE"] = [[:head, :RNode]]
NODE_SPECIFICATIONS.delete "CVAR2"
NODE_SPECIFICATIONS.delete "RESTARGS"

Alex

 

[Robert Feldt]

Still temporary, but this worked just now.

V 0.10

http://www.pronovomundo.com/ruth/

Oops, I never updated that page. Anyway that's the latest released
version. I've had a couple of bug reports which are in the pipe.

Regards,

Robert

 

[Robert Feldt]

Thanks, that link works. Though the file linked there is misnamed. It
has the extension .tar.gz even though it's just a tar file.

It seems that the choice for a front end is between Ripper and Ruth.
Both'll need a bit of development before they can be used, and neither
is documented.

Personally I think Ruth is the better choice since it has fewer
dependencies (Ripper needs bison and gperf) and it's interface seems
to be simpler.

Though it has the problem that when parsing a method definition using
the standard parse method there's no way to find out what parameters
the methods takes. However I think there might be lower level methods
which can be used in this case.


ISTM the best bet would be to design the compiler so that the parser
can be swapped easily. Looking at the archives for Cardinal I noticed
there was talk of Rite having a seperate parser component. Being able
to use that as a parser would be very useful.


I'm going to set up a project on Rubyforge in order to work on this.

On Ruby parsers I might also mention that the new version of Rockit
(that I've used internally in my research for some time) has a Ruby
parser. Rockit's new model is to automatically generate an AST-builder
class that builds an AST for you when parsing. If you want your own
callbacks without building the AST you can just subclass or write your
own builder class and initialize the parser class with that. It seems to
be a fairly good compromise between the different parsing "styles". I
have planned to release it for long but it has some non-open code that I
need to get rid of. I've sat dates before so I won't do that again
though so you might be better off with something else for now...

Anyway, the idea is that both Ruth and Rockit's Ruby parser should
generate the same AST so that they can be tested against each other for
conformance.

/Robert

 

[Sean Ross]

Hi.
This is probably naive, but I was wondering why you couldn't just use Ruby's
existing front-end implementation to parse and build an AST, and then build
a seperate generator that takes the AST as input to compile and emit Parrot
VM code? I haven't looked at Ruby's implementation, so I don't know how it
was designed, but if it was designed to seperate concerns: tokenizing,
parsing, semantic checking, optimization, etc., then as long as you know
what each part returns, you should be able to build new components for each
stage. In this case, you'd want to build a new Parrot code generator that
takes the AST that Ruby currently creates when interpreting a program
(assuming Ruby creates an AST, of course). You might also be able to make a
generator for .NET in this way ... Anyway. I was just thinking you could
leverage the existing code, and save some time and effort. But, perhaps what
I'm saying can't be done - or can't be done using Ruby's existing C
implementation. Just an idea.

Sean

 

[Simon Strandgaard]

This is probably naive, but I was wondering why you couldn't just use Ruby's
existing front-end implementation to parse and build an AST, and then build
a seperate generator that takes the AST as input to compile and emit

[snip]

Cool.. a ruby2ast converter.

Question: is the AST a bunch of ruby classes, which can be
manipulated/navigated through Ruby?


One day, I hope, to extend my editor so it can interact with Rubys AST,
then some really nice things becomes possible:

* advanced syntax coloring which can identify syntax-errors.
* obtain advanced help about the code under the cursor.
* advanced code completion.

 

[Charles Comstock]

On Ruby parsers I might also mention that the new version of Rockit
(that I've used internally in my research for some time) has a Ruby
parser. Rockit's new model is to automatically generate an AST-builder
class that builds an AST for you when parsing. If you want your own
callbacks without building the AST you can just subclass or write your
own builder class and initialize the parser class with that. It seems to
be a fairly good compromise between the different parsing "styles". I
have planned to release it for long but it has some non-open code that I
need to get rid of. I've sat dates before so I won't do that again
though so you might be better off with something else for now...

Anyway, the idea is that both Ruth and Rockit's Ruby parser should
generate the same AST so that they can be tested against each other for
conformance.

/Robert

How hard is it to just piggyback on top of parse.y from the ruby source?
Or does a lot of evaluation take place still in that file. You could
however just change what each of the actions to do to match your
backend. Unless you want the byte-code compiler to be entirely self
hosting of course. But if Rocket depends on bison anyhow you don't get
anyway.

I guess the problem I am seeing with the parrot compiler is all the code
that is C-calls into the interpreter. How do you plan on keeping that
ability, while adding the benefit of parrot? I'm curious how the
interop works with parrot.

Another point is that while I think this is a great idea, maybe we
should push to get a Rite compiler working, but make it have some sort
of pluggable middle/backend so that we can drop out bytecode or just
interpret in C. It just seems awkward to rush ahead and get a bytecode
compiler for a bytecode platform that hasn't completely standardized,
just before the language your compiling is going through a standards
change that may break code.

Just my 2 cents,
Charles Comstock

 

[Charles Comstock]

This is probably naive, but I was wondering why you couldn't just use Ruby's
existing front-end implementation to parse and build an AST, and then build
a seperate generator that takes the AST as input to compile and emit

[snip]

Cool.. a ruby2ast converter.

Question: is the AST a bunch of ruby classes, which can be
manipulated/navigated through Ruby?


One day, I hope, to extend my editor so it can interact with Rubys AST,
then some really nice things becomes possible:

* advanced syntax coloring which can identify syntax-errors.
* obtain advanced help about the code under the cursor.
* advanced code completion.

I think we definitely need a meta-aware ruby interpreter that you can
hook some stuff when it's parsing it's own file. Or even just an
interpreter switch that dumps the parse tree in a couple of different
formats, if it's too hard to get it at runtime. This would make it much
easier for alot of things.

Charles Comstock

 

[Mark]

How hard is it to just piggyback on top of parse.y from the ruby
source? Or does a lot of evaluation take place still in that file.
You could however just change what each of the actions to do to match
your backend. Unless you want the byte-code compiler to be entirely
self hosting of course. But if Rocket depends on bison anyhow you
don't get anyway.

Ripper uses parse.y to generate it parser, so it is possible.

I guess the problem I am seeing with the parrot compiler is all the
code that is C-calls into the interpreter. How do you plan on keeping
that ability, while adding the benefit of parrot? I'm curious how the
interop works with parrot.

I'm not sure that I follow you here.

Do you mean how to handle the eval instruction? In parrot there's an
opcode that registers a function as being a compiler. then another
opcode is used to compile a string to bytecode.

If you mean how to call code written in c there's the NCI interface
which so far has been used to create parrot bindings to PostgreSQL and SDL

Another point is that while I think this is a great idea, maybe we
should push to get a Rite compiler working, but make it have some sort
of pluggable middle/backend so that we can drop out bytecode or just
interpret in C. It just seems awkward to rush ahead and get a
bytecode compiler for a bytecode platform that hasn't completely
standardized, just before the language your compiling is going through
a standards change that may break code.

My understanding is that Matz isn't going to release Rite until it is at
a stage where it is running reasonably. I'm not sure how we can push to
get Rite working till then.

I like the idea of having a plugable back end to the Rite compiler so it
would be possible to get it to target the Rite Vm, Parrot, .net or the JVM.

Still even if this project is supersceded by Rite, I think it would at
least make a good prototype.

 

[Mark]

This is probably naive, but I was wondering why you couldn't just
use Ruby's
existing front-end implementation to parse and build an AST, and
then build
a seperate generator that takes the AST as input to compile and emit

[snip]

Cool.. a ruby2ast converter.

Question: is the AST a bunch of ruby classes, which can be
manipulated/navigated through Ruby?


One day, I hope, to extend my editor so it can interact with Rubys
AST, then some really nice things becomes possible:

* advanced syntax coloring which can identify syntax-errors.
* obtain advanced help about the code under the cursor.
* advanced code completion.

I think we definitely need a meta-aware ruby interpreter that you can
hook some stuff when it's parsing it's own file. Or even just an
interpreter switch that dumps the parse tree in a couple of different
formats, if it's too hard to get it at runtime. This would make it
much easier for alot of things.

This sounds like a Ruby version of Perls B::** modules, which I agree
would be a very good thing.

Though would it be best to do this by modifying the interpreter, or by
using one of Ripper, Ruth or Rocket?

 

[gabriele renzi]

il Sat, 21 Feb 2004 09:54:25 -0500, "Sean Ross"

Hi.
This is probably naive, but I was wondering why you couldn't just use Ruby's
existing front-end implementation to parse and build an AST,

I think this is what ruth , bytecoderuby and YAPV do.
ripper, OTOH, uses ruby's parse.y

 

[Robert Feldt]

My view on Ruby parsers below.

il Sat, 21 Feb 2004 09:54:25 -0500, "Sean Ross"
<[email protected]> ha scritto::




I think this is what ruth , bytecoderuby and YAPV do.

Yes, ruth and bytecoderuby (since it uses ruth!?) use the parser in your
current ruby, and with almost certainty that also goes for YAPV. Good
thing is that it really parses exactly the same language and you don't
have to do much yourself. Bad thing is not all info is there.

ripper, OTOH, uses ruby's parse.y

In theory should be in sync with Ruby's parser since it is based on it
but might lag if there are changes to Ruby's (although I think changes
are minor in practice). Downside: more work.

And then Rockit/Ruby: Rockit is a stand-alone parser generator with
clean grammar grammar. Upside: more easily readable Ruby grammar.
Actions can be plugged in. Generates AST-building actions from grammar.
Rockit supports tree traversal, pretty-printing etc for all its
AST's/grammars. Downside: Not yet released . Can get out of sync/might
not parse exactly same language.

/Robert