r4 - the simplest ruby pre-processor

A

Ara.T.Howard

all this talk about pre-processors and erb got me thinking : what would it
take to code up a ruby pre-procssors that could be used for any laguange. my
first crack is 38 lines of ruby that pre-process any files on the command line
(or stdin) using ruby as the macro language and a convenience function 'macro'
which can be used to define ruby methods which return test that is used as
output. functions declared this way need not do any explicit io. here's some
examples:

harp:~ > cat input.r4
|
| # any valid ruby code can go here
|
| x = 42
|
| # any output produced is used in the generated output
|
| p x
|

any lines which are not marked with a leading '|' are copied verbetim to the
output

|
| # since the pre-processor language is ruby it can do anything
|
| 10.times{|i| p i}
|


harp:~ > r4 input.r4
42

any lines which are not marked with a leading '|' are copied verbetim to the
output

0
1
2
3
4
5
6
7
8
9



and, using r4's macro shortcut to generate c code:


harp:~ > cat input.c
|
| fields = %w( foo bar foobar barfoo )
|
| macro('field'){|name| "int #{ name };" }
|
| macro('setter') do |name|
| <<-c
| int set_#{ name }(self, value)
| object * self;
| int value;
| {
| return( self->#{ name } = value );
| }
| c
| end
|
| macro('getter') do |name|
| <<-c
| int get_#{ name }(self)
| object * self;
| {
| return( self->#{ name } );
| }
| c
| end

struct object {
|
| fields.each{|f| field f}
|
};

|
| fields.each{|f| setter(f); getter(f); }
|


harp:~ > r4 input.c

struct object {
int foo;
int bar;
int foobar;
int barfoo;
};

int set_foo(self, value)
object * self;
int value;
{
return( self->foo = value );
}
int get_foo(self)
object * self;
{
return( self->foo );
}
int set_bar(self, value)
object * self;
int value;
{
return( self->bar = value );
}
int get_bar(self)
object * self;
{
return( self->bar );
}
int set_foobar(self, value)
object * self;
int value;
{
return( self->foobar = value );
}
int get_foobar(self)
object * self;
{
return( self->foobar );
}
int set_barfoo(self, value)
object * self;
int value;
{
return( self->barfoo = value );
}
int get_barfoo(self)
object * self;
{
return( self->barfoo );
}


and, finally, the source for r4:

#! /usr/bin/env ruby
require 'tempfile'

script = Tempfile::new(File::basename(__FILE__))
script << DATA.read

pat = %r/^\s*\|(.*)$/

hdoc = []
start_hdoc = lambda do
if hdoc.empty?
script << "puts <<-'" << hdoc.push("___code_#{ rand(2 ** 42) }___") << "'" << "\n"
end
end
end_hdoc = lambda do
unless hdoc.empty?
script << hdoc.pop << "\n"
end
end
ARGF.each do |line|
m = pat.match line
if m
code = m[1]
end_hdoc[]
script << code << "\n"
else
start_hdoc[]
script << line
end
end
end_hdoc[]

script.close
load script.path

__END__
class Object
def macro(m, &b)
klass = (Class === self ? self : (class << self; self; end))
klass.module_eval do
define_method(m){|*a| puts b.call(*a)}
end
end
end


obivious the char marking pre-processor lines could be anything - but other
than that there is special markup.

thoughts/comments/flames?

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================
 
A

Ara.T.Howard

obivious the char marking pre-processor lines could be anything - but other
than that there is special markup.
^^
^^
no

sorry.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================
 
J

Johan Toki Persson

Short and sweet. :)
Reminds me of my BrainF*ck variant implemented in 17 lines of readable
Ruby-code.
 
B

Bob Hutchison

Hi,

With a tiny change to this you can also use ruby's embedded value
substitution (interpolation), like:

|
| some_number = 42
|

Hello there!

some number: [[[#{some_number}]]]

to get output like:

Hello there!

some number: [[[42]]]

That one change is from:

"puts #{line.chomp.dump}\n"

to:

"puts \"#{line.chomp}\"\n"

(just added the quotes).

This is a very interesting little toy. Maybe not a toy at all.

Cheers,
Bob

all this talk about pre-processors and erb got me thinking : what
would it
take to code up a ruby pre-procssors that could be used for any
laguange. my
first crack is 38 lines of ruby that pre-process any files on the
command line
(or stdin) using ruby as the macro language and a convenience
function 'macro'
which can be used to define ruby methods which return test that is
used as
output. functions declared this way need not do any explicit io.

Alternative implementation ("r5"):

def macro(name, &block)
Kernel.send:)define_method, name) { |*args|
puts block.call(args)
}
end

send(($DEBUG ? "puts" : "eval"), ARGF.readlines.map { |line|
if line[0] == ?|
line[1..-1]
else
"puts #{line.chomp.dump}\n"
end
}.join)

Happy hacking,
 
B

Bob Hutchison

Bob Hutchison wrote:



And broke it.
| blah = 42
Hello. I'm an evil template. ", `rm -rf /`, "I advise against
running me as root.

Devin

Broke it? Nah, it was already 'broken'. And anyway, that's what I
wanted to do, and really would've 'broken' it to achieve that :)

If you want safe, you can't be executing code in a template at all.
That includes the lines beginning with "|". For example:

| system("ls -lt") # that could be "rm -rf /"

It gives you a warning, but you'll be reading that just a little late.

Seriously though, this is exactly what I want to be able to do.
Personally, I don't need something safe. The same people using the
software I'm working on have access to a command line, they don't
need to go through all that trouble to screw themselves up.

I have a code generation problem, and this looks to address my
requirements very well.

I also have general template requirements, I won't be using this
because those users don't necessarily know a lot about programming
(and if they don't already know won't be looking to learn just to use
my software :) I'll look elsewhere for a solution to that.

Cheers,
Bob
 
B

Bob Hutchison

Alternative implementation ("r5"):

So, playing around a bit more, I've got something that solves a
problem I have right now (with code generation)... using this scheme
to add methods to an arbitrary class.

Keeping with the theme, maybe call this r6?

r6.rb -----------------------------
#! /usr/bin/env ruby
def macro(name, &block)
Kernel.send:)define_method, name) { |*args|
puts block.call(args)
}
end

def build_script(template_file_name, class_name, method_name)

# Build a method (called 'method_name') in the class 'class_name'
that
# will execute the template (in the file 'template_file_name').
There will
# be an optional argument that defaults to the empty string (and
so this
# method will, by default, build a new string representing the
result). If
# the argument is supplied, it must respond to the "<<" (append)
method.
#
# The result variable is available in the template. To write to
the result
# from Ruby code, result << sprintf("hello %s", "world") in the
template
# will get its output where expected.

File.open(template_file_name) do | file |
r = "
class #{class_name}
def #{method_name}(result=\"\")
result << \"\"
"
while line = file.gets
if line[0] == ?|
r << " #{line[1..-1]}"
else
r << " result << \"#{line.chomp}\\n\"\n"
end
end
r << "
result
end
end
"
end
end

###
### and this is how it can be used...
###

# Define a class to hold the template methods. There is an attribute
'message'
# that can be set by the program invoking the template, and referred
to by the
# templates. You can put any attributes you want in there.

class R6_Template
attr_accessor :message
end

# Assume that the command line arguments are all specifying the name
of a
# template file. Open each file and pass it to the build script
method. When
# the script is built, 'eval' it.

ARGV.each { | script_name |
method_name = File::basename(script_name, ".*")
the_script = build_script(script_name, "R6_Template", method_name)
#puts the_script ## if you want to see what it looks like,
uncomment this line
eval the_script
}

# For illustrative purposes, go over the command line arguments and
execute
# the corresponding method defined above.

template = R6_Template.new()
ARGV.each { | script_name |
method_name = File::basename(script_name, ".*")
template.message = sprintf("this is script '%s'", method_name)
puts "#{method_name}*******************"
what = template.send(method_name);
puts "{{{#{what}}}}*******************"
}

# This time call the play method directly. There must be a template
# called 'play' (sans extenstion) for this to work.

template.message = "this is script 'play' -- called explicitly"
puts "!!play!!*******************"
what = template.play()
puts "{{{#{what}}}}*******************"

# Now do the same thing as the illustrative loop above but writing to
a file
# with the script name and a ".out" extension.

ARGV.each { | script_name |
method_name = File::basename(script_name, ".*")
template.message = sprintf("this is script '%s'", method_name)
File.open(sprintf("%s.out", method_name), "w") { | file |
template.send(method_name, file);
}
}

# Write to a file called "play-x.out", again, there must be a play
template
# defined.

template.message = "this is script 'play' -- called explicitly"
File.open("play-x.out", "w") { | file |
template.play(file)
}

# Build up a single string by applying all the templates
long_string = ""
ARGV.each { | script_name |
method_name = File::basename(script_name, ".*")
template.message = sprintf("this is script '%s'", method_name)
what = template.send(method_name, long_string);
}
puts "!!{{{#{long_string}}}}!!*******************"

-----------------------------
play.r6 -----------------------------
|
| some_number = 42
| def gen_some_number
| 42 + rand
| end
|

Hello there (play)! #{@message}

some number: [[[#{some_number}]]]
gen some number: [[[#{gen_some_number}]]]
-----------------------------
play_more.r6 -----------------------------
|
| some_number = 42
| def gen_some_number
| 42 + rand
| end
|

Hello there(play_more)! #{@message}

|
| another_number = 99
| result << play()
|

some number: [[[#{some_number}]]]
gen some number: [[[#{gen_some_number}]]]
another number: [[[#{another_number}]]]

play again:... #{play}
-----------------------------

Note that the play_more template uses the play template (twice)

I've also taken to calling them 'templates'...

Cheers,
Bob
 
A

Ara.T.Howard

I have a code generation problem, and this looks to address my requirements
very well.

careful - if you go the

route an errant '#{' or '`' in your input will cause a syntax error. the
orginal method of

"puts "#{ line.chomp.dump }\n"

will always work though.

cheers.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================
 
B

Bob Hutchison

careful - if you go the



route an errant '#{' or '`' in your input will cause a syntax
error. the
orginal method of

"puts "#{ line.chomp.dump }\n"

will always work though.

Except that '#{' or '`' will be escaped and ignored. Still, point
taken. Maybe a 'be_careful' option? But as I said in a previous post,
a 'be_safe' option isn't really possible. Now, I've not used Ruby in
a while, maybe I'm missing something.

Cheers,
Bob
cheers.

-a
--
======================================================================
=========
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
======================================================================
=========
 
A

Ara.T.Howard

Ara.T.Howard said:
all this talk about pre-processors and erb got me thinking : what would it
take to code up a ruby pre-procssors that could be used for any laguange. my
first crack is 38 lines of ruby that pre-process any files on the command line
(or stdin) using ruby as the macro language and a convenience function 'macro'
which can be used to define ruby methods which return test that is used as
output. functions declared this way need not do any explicit io.

Alternative implementation ("r5"):

def macro(name, &block)
Kernel.send:)define_method, name) { |*args|
puts block.call(args)
}
end

send(($DEBUG ? "puts" : "eval"), ARGF.readlines.map { |line|
if line[0] == ?|
line[1..-1]
else
"puts #{line.chomp.dump}\n"
end
}.join)

Happy hacking,

a version which optionally interpolates - six lines, ergo r6

harp:~ > cat in
| x = 42

using the -i switch i can interpolate x as #{ x }


harp:~ > r6 in

using the -i switch i can interpolate x as #{ x }


harp:~ > r6 --interpolate in

using the -i switch i can interpolate x as 42


harp:~ > cat r6
#! /usr/bin/env ruby

def macro(m, &b); Kernel::send('define_method', m){|*a| puts b.call(a)}; end

debug = $DEBUG || ENV['DEBUG'] || ARGV.delete('-d') || ARGV.delete('--debug')

interp = $INTERPOLATE || ENV['INTERPOLATE'] || ARGV.delete('-i') || ARGV.delete('--interpolate')

meth = debug ? 'puts' : 'eval'

src = ARGF.readlines.map{|l| l =~ %r/^\s*\|(.*)/ ? $1 : (interp ? "puts \"#{ l.chomp }\"" : "puts #{ l.chomp.dump }")}

send meth, src.join("\n")

cheers.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================
 
A

Ara.T.Howard

You broke it because " can't appear as-is in the file anymore.
(Which is likely what I want when I code C or something.)

fixed:

harp:~ > cat in
| x = 42

using the -i switch i can interpolate x as #{ x }

quote " works

quote ' works


harp:~ > r7 in

using the -i switch i can interpolate x as #{ x }

quote " works

quote ' works


harp:~ > r7 --interpolate in

using the -i switch i can interpolate x as 42

quote " works

quote ' works


harp:~ > cat r7
#! /usr/bin/env ruby

def macro(m, &b); Kernel::send('define_method', m){|*a| puts b.call(a)}; end

d = $DEBUG || ENV['DEBUG'] || ARGV.delete('-d') || ARGV.delete('--debug')

i = $INTERPOLATE || ENV['INTERPOLATE'] || ARGV.delete('-i') || ARGV.delete('--interpolate')

h = '_' * 42

m = d ? 'puts' : 'eval'

s = ARGF.readlines.map{|l| l =~ %r/^\s*\|(.*)/ ? $1 : (i ? %Q(puts <<#{ h }\n#{ l.chomp }\n#{ h }) : %Q(puts #{ l.chomp.dump }))}

send m, s.join("\n")


-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================
 
P

Pit Capitain

Ara.T.Howard said:
a version which optionally interpolates - six lines, ergo r6
...
src = ARGF.readlines.map{|l| l =~ %r/^\s*\|(.*)/ ? $1 : (interp ?
"puts \"#{ l.chomp }\"" : "puts #{ l.chomp.dump }")}
...

Just a minor simplification: ARGF is kind of an Enumerable, so you can
omit the call to readlines.

BTW: nice idea.

Regards,
Pit
 
T

Trans

Very cool Ara.

What about making the '|' user-definable, as well as which it isolates:
the code or the text. Use '#' instead and reverse it, you'd have a ruby
program runner that spits out all its comments :)

T.
 
T

Trans

gee, I just read what I wrote and it borders on nonsense (I don't
exactly think in words, so its not alwasy easy to translate) BUt to
explain better:

struct object {
|
| fields.each{|f| field f}
|
};

change marker, becomes:

struct object {
#
# fields.each{|f| field f}
#
};

invert:

#struct object {
fields.each{|f| field f}
#};

So, using this setup a normal ruby program would run while printing out
it's comments. Not useful, but I just thought it was interesting.
 
D

Dave Burt

Trans said:
So, using this setup a normal ruby program would run while printing out
it's comments. Not useful, but I just thought it was interesting.

I think this is a very cool idea for an interesting debug mode.

Cheers,
Dave
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,770
Messages
2,569,583
Members
45,075
Latest member
MakersCBDBloodSupport

Latest Threads

Top