Ripper and Ruby 1.8

[Jonathan Maasland]

Hi all,

For the past two days I've been trying to get Ripper to compile with
Ruby 1.8
After searching and semi-translating some Japanese messages [1] with
Babelfish I managed to compile Ripper for 1.8 but when using it I get an
error-message: [Ripper - FATAL] : Unknown token 357 (or something like that)

Does anyone here have experience with getting Ripper working under 1.8?
Does anyone know the specifics as to why Ripper states it needs 1.9?
Does it have to do with parse.y?

Any information on the subject is really really appreciated.

With kind regards,
Jonathan


[1] : http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-dev/26851

 

[nobu]

Hi,

At Sun, 20 Aug 2006 20:07:16 +0900,
Jonathan Maasland wrote in [ruby-talk:209477]:

Does anyone know the specifics as to why Ripper states it needs 1.9?
Does it have to do with parse.y?

Ripper needs bison, but 1.8 can be compiled with other yaccs.

 

[Jonathan Maasland]

Hi,


Ripper needs bison, but 1.8 can be compiled with other yaccs.

Thanks for the reply but I'm a bit at a loss here. I'm a total noob
regarding Ruby's compilation process so I'm sorry if this is a stupid
question but if I have bison available why should I not be able to
compile Ripper against 1.8?

Thanks,
Jonathan

 

[Dominik Bathon]

Hi,

Thanks for the reply but I'm a bit at a loss here. I'm a total noob =

regarding Ruby's compilation process so I'm sorry if this is a stupid = =

question but if I have bison available why should I not be able to =

compile Ripper against 1.8?

I don't know how to get Ripper working, but depending on what you want t=
o =

do, there might be better solutions like RubyNode or ParseTree.

For example with RubyNode (http://rubynode.rubyforge.org/) you can do:
[:iter,
{:var=3D>false,
:iter=3D>[:call, {:args=3D>false, :mid=3D>:times, :recv=3D>[:lit, {:l=
it=3D>3}]}],
:body=3D>[:fcall, {:args=3D>[:array, [[:str, {:lit=3D>"Ruby"}]]], =

:mid=3D>uts}]}]
=3D> nil

The only thing that RubyNode or ParseTree won't give you are the comment=
s =

and whitespace in the code.


Dominik

 

[Jonathan Maasland]

I don't know how to get Ripper working, but depending on what you want
to do, there might be better solutions like RubyNode or ParseTree.

--snip--

It's what we currently use in FreeRIDE and I kinda like the way Ripper
is used. Currently we only use it to aquire all methods/modules/classes.
I guess I could look into working with a different parser but I'd prefer
Ripper.

The only thing that RubyNode or ParseTree won't give you are the
comments and whitespace in the code.

That's a bit of a letdown, nothing too serious. Probably easy to work
around by wrapping the IO-source.
Do you have any idea as to how memory-intensive either are?

Anyhow, thanks for the suggestion. I'll look into it if Ripper takes too
long to get running again.

Jonathan

 

[Dominik Bathon]

t =

It's what we currently use in FreeRIDE and I kinda like the way Ripper= =

is used. Currently we only use it to aquire all methods/modules/classe=

s. =

I guess I could look into working with a different parser but I'd pref=

er =

Ripper.

Ah okay, I thought you were trying to get Ripper running for the first =

time, for a new project. If you already have code then it might be easie=
r =

to get Ripper working.

Anyway, here is some basic code to get methods and classes with RubyNode=
=

(it's really basic and doesn't work with nested classes, just to get you=
=

started):

Lets say we have this file:

$ cat test.rb
class A
def a
1
end
def b
2
end
end

class B
def c
3
end
end

Then this code:

require "rubynode"
require "pp"

tree =3D IO.read("test.rb").parse_to_nodes.transform

def statements(block_or_single_node)
if block_or_single_node.first =3D=3D :block
block_or_single_node.last
else
[block_or_single_node] # only one statement
end
end

statements(tree).each { |s|
if s.first =3D=3D :class
p s.last[:cpath]
statements(s.last[:body].last[:next]).each { |d|
if d.first =3D=3D :defn
p d.last[:mid]
pp d.last[:defn]
end
}
end
}

outputs:

[:colon2, {:mid=3D>:A, :head=3D>false}]
:a
[:scope,
{:rval=3D>false,
:tbl=3D>nil,
:next=3D>
[:block, [[:args, {:cnt=3D>0, pt=3D>false, :rest=3D>-1}], [:lit, =

{:lit=3D>1}]]]}]
:b
[:scope,
{:rval=3D>false,
:tbl=3D>nil,
:next=3D>
[:block, [[:args, {:cnt=3D>0, pt=3D>false, :rest=3D>-1}], [:lit, =

{:lit=3D>2}]]]}]
[:colon2, {:mid=3D>:B, :head=3D>false}]
:c
[:scope,
{:rval=3D>false,
:tbl=3D>nil,
:next=3D>
[:block, [[:args, {:cnt=3D>0, pt=3D>false, :rest=3D>-1}], [:lit, =

{:lit=3D>3}]]]}]

That's a bit of a letdown, nothing too serious. Probably easy to work = =

around by wrapping the IO-source.
Do you have any idea as to how memory-intensive either are?

They both wrap Ruby's internal NODEs. ParseTree converts them into =

s-expressions (nested arrays), RubyNode's transform converts them to =

something similar (see above, a mix of arrays and hashes). So they use a=
s =

much memory as their output needs.

If you really want to avoid the transformation to s-expressions then you=
=

can use RubyNodes directly without calling transform (see documentation)=
, =

but that usually shouldn't be necessary.

And btw. if you want to get the nodes without evaling the code then you =
=

currently have to use RubyNode, ParseTree doesn't support that (yet).

Dominik

 

[nobu]

Hi,

At Mon, 21 Aug 2006 00:39:40 +0900,
Jonathan Maasland wrote in [ruby-talk:209505]:

Thanks for the reply but I'm a bit at a loss here. I'm a total noob
regarding Ruby's compilation process so I'm sorry if this is a stupid
question but if I have bison available why should I not be able to
compile Ripper against 1.8?

Ripper needs a bison specific feature, but parse.y in 1.8 can't
and doesn't use it due to the compatibility.