Syntax extensions - question

[samuel_x_winters_x]

Context: using Ruby for domain-specific languages. Ruby's properties of
dynamic + reflective + blocks are GREAT for the DSL semantics, so this
is about allowing DSL syntax.

I have written a few "macro" like methods in the Ruby way, usually
against some meta-object such as Class or Module. Blocks are hugely
helpful, in large part because they are not evaluated right away. But
everything else is evaluated, so I need to do things like write :abc or
"abc" in places where it would be more natural to just write abc. Plus
it is hard to do things like manipulate variable bindings from such
macros.

At the same time there are various convenient syntax 'extensions' like
%w{...}. And there is really nice work on Syntax and Ripper to make
ruby ASTs accessible.

I was wondering: what would be the interest, value, and feasibility of
something like this:

Suppose there was a Grammer type (it can parse, returns something like
an AST). And some distinguished syntax to introduce a grammar object
(I'll use "!!" here, some variant of %w might be better) so that an
expression like this:

X!!{ ..... }

would result at syntax analysis time in:
1. call X.grammer # expect a Grammar object, g
2. pass everything between {...} to g.parse
3. expect (a fragment of) a Ruby AST back, t
4. call t.process (or whatever is Ruby's way to combine and analyze its
AST fragments and eventually produce classes, methods, etc. Probably a
richer protocol than a single method)
5. keep going from just after "}"

Note that step 3 need not return a "standard" AST, it could just return
something that responded appropriately to "process" in #4, so this need
not be standard "macro" like expansion. However, I would expect to very
often use a standard AST.

Most likely the "X" will have to be a constant. And most likely this
will require some standardization of Ruby's (duck-typed) ways of
handling ASTs.

Result: Can define, use, mix, and combine (fragments of)
domain-specific language with domain-specific syntax.

Thoughts?

 

[Phil Tomson]

Context: using Ruby for domain-specific languages. Ruby's properties of
dynamic + reflective + blocks are GREAT for the DSL semantics, so this
is about allowing DSL syntax.

I have written a few "macro" like methods in the Ruby way, usually
against some meta-object such as Class or Module. Blocks are hugely
helpful, in large part because they are not evaluated right away. But
everything else is evaluated, so I need to do things like write :abc or
"abc" in places where it would be more natural to just write abc. Plus
it is hard to do things like manipulate variable bindings from such
macros.

At the same time there are various convenient syntax 'extensions' like
%w{...}. And there is really nice work on Syntax and Ripper to make
ruby ASTs accessible.

I was wondering: what would be the interest, value, and feasibility of
something like this:

Suppose there was a Grammer type (it can parse, returns something like
an AST). And some distinguished syntax to introduce a grammar object
(I'll use "!!" here, some variant of %w might be better) so that an
expression like this:

X!!{ ..... }

would result at syntax analysis time in:
1. call X.grammer # expect a Grammar object, g
2. pass everything between {...} to g.parse
3. expect (a fragment of) a Ruby AST back, t
4. call t.process (or whatever is Ruby's way to combine and analyze its
AST fragments and eventually produce classes, methods, etc. Probably a
richer protocol than a single method)
5. keep going from just after "}"

Note that step 3 need not return a "standard" AST, it could just return
something that responded appropriately to "process" in #4, so this need
not be standard "macro" like expansion. However, I would expect to very
often use a standard AST.

Most likely the "X" will have to be a constant. And most likely this
will require some standardization of Ruby's (duck-typed) ways of
handling ASTs.

Result: Can define, use, mix, and combine (fragments of)
domain-specific language with domain-specific syntax.

Thoughts?

Interesting ideas. It would be very nice to be able to access the AST as
an object and have this builtin to Ruby. I would certainly have uses for
this sort of thing.

....however, I suspect that this might smack of macros (as in Lisp) in
Ruby and Matz tends to frown upon that sort of thing. I think there has
been some talk of allowing greater access to the AST in Ruby 2.0 (Rite),
though.

BTW: have you taken a look at ParseTree?
http://blog.zenspider.com/archives/parsetree/index.html

Phil

 

[gabriele renzi]

(e-mail address removed) ha scritto:

<snipall>
What's wrong with:

Grammar.new %{
foo: bau
bar: miao-102
}

the %{} literal gives you a string, and you can parse it with the full
power of regexp/racc/cocorb/whatever in Grammar#initialize.

 

[samuel_x_winters_x]

Yes, that is a reasonable way to get the string into the grammar. Could
you explain if / how it would deal with #3, #4 (and hence #5)?

3. expect (a fragment of) a Ruby AST back, t
4. call t.process (or whatever is Ruby's way to combine and analyze its
AST fragments and eventually produce classes, methods, etc. Probably a
richer protocol than a single method)
5. keep going from just after "}"

I want the DSL fragment to contribute to building the overall AST and
subsequently processing it. e.g. even use a DSL fragment inside a
normal Ruby method. Any suggestions in that direction?