eval/binding question

[Stefan Kaes]

--------------050704080108030307000402
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I tried to create local variables from a name=>value hash passed as a
parameter, but I can't get it to work. The following test program shows
what is happening:

def test1
eval "x=25"
eval 'print "x=#{x}\n"'
end

test1

def test2
eval "x=25"
print "x=#{x}\n"
end

test2

produces:

x=25
test.rb:189:in `test2': undefined local variable or method `x' for
main:Object ( NameError)
from test.rb:192

It seems that changes to local variables are not reflected back to the
executing environment.

Does anyone out there know how to solve this problem?

-- stefan


--------------050704080108030307000402--

 

[Robert Klemme]

I tried to create local variables from a name=>value hash passed as a
parameter, but I can't get it to work. The following test program shows
what is happening:

def test1
eval "x=25"
eval 'print "x=#{x}\n"'
end

test1

def test2
eval "x=25"
print "x=#{x}\n"
end

test2

produces:

x=25
test.rb:189:in `test2': undefined local variable or method `x' for
main:Object ( NameError)
from test.rb:192

It seems that changes to local variables are not reflected back to the
executing environment.

Does anyone out there know how to solve this problem?

You can't. Local variables are determined during parse phase and you
cannot access local variables that you create via eval other than via
eval. Consider

16:11:59 [robert.klemme]: ruby -e 'eval "x=10";puts x'
-e:1: undefined local variable or method `x' for main:Object (NameError)
16:13:51 [robert.klemme]: ruby -e 'x=nil;eval "x=10";puts x'
10
16:13:57 [robert.klemme]: ruby -e 'eval "x=10";eval "puts x"'
10

IOW, you can manipulate predefined locals but not those that spring into
existence during an eval.

Kind regards

robert

 

[Yukihiro Matsumoto]

Hi,

In message "Re: eval/binding question"

|I tried to create local variables from a name=>value hash passed as a
|parameter, but I can't get it to work. The following test program shows
|what is happening:

local variables should be determined at compile time, thus local
variables defined first in the eval'ed string, can only be accessed from
other eval'ed strings. In addition, they will be more ephemeral in
Ruby2, so that these variables will not be accessed from outside.

In summary, I recommend you not to use local variables for your
purpose. They are wrong tool for it.

matz.

 

[Stefan Kaes]

Hi,

In message "Re: eval/binding question"

|I tried to create local variables from a name=>value hash passed as a
|parameter, but I can't get it to work. The following test program shows
|what is happening:

local variables should be determined at compile time, thus local
variables defined first in the eval'ed string, can only be accessed from
other eval'ed strings. In addition, they will be more ephemeral in
Ruby2, so that these variables will not be accessed from outside.

Well, I don't agree. From a language designers point of view x=5 and
eval "x=5" should do the same thing: modify the current binding by
introducing a new value-binding with name x and value 5. I don't know of
any language which behaves differently (e.g. LISP works like this
AFAIK). Of course, as a compiler writer, one might prefer to be able to
determine all local variables by looking at the source. But this is just
a wish to make the compiler simpler or enable better optimization.

In summary, I recommend you not to use local variables for your
purpose. They are wrong tool for it.

matz.

Maybe I should explain why I tried this:

when using ERB one can compile a template into source, which can then be
eval'ed using eval. Local variables for use in the template can be set
up using the sort of eval given in my example. However, in this case the
'compiled' template code gets reparsed on each evaluation of the
template code. I wanted to speed up this process by defining a function
(eval "def fun; #{src}; end", aBinding), thereby pasring the code only
once. This works pretty well and gives about 25% increase in speed. The
performance gain will be much bigger, once ruby gets a real JIT. But, as
it turned out, local variables are not accessible when the defined
function is executed.

I saw two solutions: First, change the local variables to instance
variables on the class executing the function. This works, but does not
feel right (name space pollution). Second, put all locals in an instance
hash and set up local functions inside the defined function from the
hash. Which does not work.

Any other ideas?

 

[Yukihiro Matsumoto]

Hi,

In message "Re: eval/binding question"

|>local variables should be determined at compile time, thus local
|>variables defined first in the eval'ed string, can only be accessed from
|>other eval'ed strings. In addition, they will be more ephemeral in
|>Ruby2, so that these variables will not be accessed from outside.
|
|Well, I don't agree. From a language designers point of view x=5 and
|eval "x=5" should do the same thing: modify the current binding by
|introducing a new value-binding with name x and value 5. I don't know of
|any language which behaves differently (e.g. LISP works like this
|AFAIK). Of course, as a compiler writer, one might prefer to be able to
|determine all local variables by looking at the source. But this is just
|a wish to make the compiler simpler or enable better optimization.

You don't have to agree here. Each language designer have different
point of view. I'm not going to change my mind by your "should".

|when using ERB one can compile a template into source, which can then be
|eval'ed using eval. Local variables for use in the template can be set
|up using the sort of eval given in my example. However, in this case the
|'compiled' template code gets reparsed on each evaluation of the
|template code. I wanted to speed up this process by defining a function
|(eval "def fun; #{src}; end", aBinding), thereby pasring the code only
|once. This works pretty well and gives about 25% increase in speed. The
|performance gain will be much bigger, once ruby gets a real JIT. But, as
|it turned out, local variables are not accessible when the defined
|function is executed.

I'm not sure I'm fully understand you. Can you show us your code?

matz.

 

[Stefan Kaes]

I'm not sure I'm fully understand you. Can you show us your code?

Yes, but I need to clean up first.

-- stefan

 

[George Moschovitis]

Maybe I should explain why I tried this:

when using ERB one can compile a template into source, which can then

be eval'ed using eval. Local variables for use in the template can be set
hash and set up local functions inside the defined function from the

Well, that's funny

I just finished working on a VERY similar bit of code.

FYI, I came up with a sligthly altered version of your first solution.
In my case, name space polution is not a problem.

The code will be available soon as part of Nitro's new Mailer
subsystem.

regards,
-g.

ps:

 

[Stefan Kaes]

--------------050504010907080609010900
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

code attached.

In the real environment, the names of the locally assigned variables can
not be determined in advance, of course.




--------------050504010907080609010900
Content-Type: text/plain;
name="evalbinding.rb"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="evalbinding.rb"

require 'erb'

class ERBrunner #all compiled template functions go here
def get_binding
binding
end
end

Erb_runner = ERBrunner.new

class Test

def initialize
@erb = ERB.new(File.read("test.rhtml"), nil, '%')
evalstr = "def run_erb\n" +
"bnd = binding\n" +
'@locals.each{|k,v| eval "#{k} = @locals[\"#{k}\"]", bnd }' +
"\n#{@erb.src}\n_erbout\nend\n"
print evalstr, "\n\n"
eval evalstr, Erb_runner.get_binding
end

def runit
Erb_runner.instance_variable_set "@mumu", "HALLO"
Erb_runner.instance_variable_set "@locals", {'mumu'=>"HALLO"}
bnd = Erb_runner.get_binding
eval "Erb_runner.run_erb", bnd
end

end

t = Test.new
print t.runit


--------------050504010907080609010900
Content-Type: text/plain;
name="test.rhtml"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="test.rhtml"

locals: <%= @locals.inspect %>
locals['mumu']: <%= @locals['mumu'] %>
mumu: <%= mumu rescue "not defined" %>
@mumu: <%= @mumu %>

--------------050504010907080609010900--

 

[Stefan Kaes]

You don't have to agree here. Each language designer have different
point of view. I'm not going to change my mind by your "should".

Let me rephrase: the semantics of eval should be defined in a way that
holds no surprises. I would expect that writing

...
eval "code"
...

and

...
code
...

should be identical.

-- stefan

 

[Yukihiro Matsumoto]

Hi,

In message "Re: eval/binding question"

|Let me rephrase: the semantics of eval should be defined in a way that
|holds no surprises.

I know what you mean. But it's not good strategy to persuade me to
use "your expectation" or "your surprise". This "limitation" has a
lot of good aspects, such as better performance, better error
detection, etc. I'd love to pay the cost of small restriction for
these benefits as a language designer.

OK, I think this is enough. Let us focus on solving "your problem".

matz.

 

[Yukihiro Matsumoto]

Hi,

In message "Re: eval/binding question"

|In the real environment, the names of the locally assigned variables can
|not be determined in advance, of course.

I'm sorry to say you can't pre-compile eRB allowing local variable set
not determined in advance. I know allowing this will make your
program 25% faster, but this also would make any other programs
slower.

If you have limited set of local variables, you can do something like:

require 'erb'

class Test
def initialize(path, *lv)
@erb = ERB.new(File.read(path), nil, '%')
@locals = {}
evalstr = "def run_erb\n" +
lv.collect{|k| "#{k} = @locals[\"#{k}\"]\n"}.join +
"#{@erb.src}\n_erbout\nend\n"
print evalstr, "\n\n"
eval evalstr, nil, path, 0
end
def values(hash)
@locals.update(hash)
hash.each {|k,v|
self.instance_variable_set("@#{k}", v)
}
end
end

t = Test.new("/tmp/test.rhtml", 'mumu')
t.values("mumu" => "HELLO")
print t.run_erb

matz.

 

[Eric Hodel]

--Apple-Mail-23--505849831
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; format=flowed

when using ERB one can compile a template into source, which can then
be eval'ed using eval. Local variables for use in the template can be
set up using the sort of eval given in my example. However, in this
case the 'compiled' template code gets reparsed on each evaluation of
the template code. I wanted to speed up this process by defining a
function (eval "def fun; #{src}; end", aBinding), thereby pasring the
code only once.

ERb already does this for you. See ERB:efmethod. Unfortunately,
there are no docs for it, but you can dig around in Rails to see how it
is used.

--
Eric Hodel - (e-mail address removed) - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

--Apple-Mail-23--505849831
content-type: application/pgp-signature; x-mac-type=70674453;
name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFCN644MypVHHlsnwQRAqVXAKDpMrfdCwEgWGxAZ7rrBC3nkCtdLwCfekGG
Qi0hXNNyvrKWpqLMFfXWpl4=
=19Tc
-----END PGP SIGNATURE-----

--Apple-Mail-23--505849831--

 

[Eric Hodel]

--Apple-Mail-1--502680050
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; format=flowed

ERb already does this for you. See ERB:efmethod. Unfortunately,
there are no docs for it, but you can dig around in Rails to see how
it is used.

Ok, I lied. ERB:efMethod is both not used by Rails (does something
else) but it is also unsuitable for the task.

Anyhow, as my penance, here's my implementation, I did use def_method,
though:

http://segment7.net/projects/ruby/snippets/erb_cache.rb

--
Eric Hodel - (e-mail address removed) - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

--Apple-Mail-1--502680050
content-type: application/pgp-signature; x-mac-type=70674453;
name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFCN7qaMypVHHlsnwQRAusxAJ91YtmZxwzqnOCCwWNlVYvVJMv/DwCeL7X2
7Zps0NGBjewBXoZI9Cy3f54=
=hVai
-----END PGP SIGNATURE-----

--Apple-Mail-1--502680050--

 

[Stefan Kaes]

ERb already does this for you. See ERB:efmethod. Unfortunately,
there are no docs for it, but you can dig around in Rails to see how
it is used.

That's not true. ERB:efmethod does not work this way. And Rails uses
the less efficient method, which I wanted to improve on (and did). And I
wanted the local varaiable solution mainly for backwards compatibility
with rendering partials in Rails. But the question about eval/binding
semantics was posted to this list because I thought (and still think)
that the semantics of eval is not what it should be.

 

[Stefan Kaes]

Ok, I lied. ERB:efMethod is both not used by Rails (does something
else) but it is also unsuitable for the task.

Why did you lie???

Anyhow, as my penance, here's my implementation, I did use def_method,
though:

http://segment7.net/projects/ruby/snippets/erb_cache.rb

Thanks for the code, but I already knew this would also be a possible
solution. But probably less efficient than using instance variables,
though I have not checked that yet. Did you?

 

[Stefan Kaes]

Hi,

In message "Re: eval/binding question"

|In the real environment, the names of the locally assigned variables can
|not be determined in advance, of course.

I'm sorry to say you can't pre-compile eRB allowing local variable set
not determined in advance. I know allowing this will make your
program 25% faster, but this also would make any other programs
slower.

I fail to see why it would make other code slower. I am convinced with a
proper compiler implementation there would'nt be any speed difference
for local variables introduced via explicit assignment. Access to
predeclared variables would use an optimized access method, whereas
access to variable names whose declaration point cannot be determined by
statical program analysis would use a less efficient access method.

If you have limited set of local variables, you can do something like:

require 'erb'

class Test
def initialize(path, *lv)
@erb = ERB.new(File.read(path), nil, '%')
@locals = {}
evalstr = "def run_erb\n" +
lv.collect{|k| "#{k} = @locals[\"#{k}\"]\n"}.join +
"#{@erb.src}\n_erbout\nend\n"
print evalstr, "\n\n"
eval evalstr, nil, path, 0
end
def values(hash)
@locals.update(hash)
hash.each {|k,v|
self.instance_variable_set("@#{k}", v)
}
end
end

t = Test.new("/tmp/test.rhtml", 'mumu')
t.values("mumu" => "HELLO")
print t.run_erb

matz.

That's simlar to the way I "solved" "my problem" already. Thanks.

-- stefan

 

[Robert Klemme]

Hi,

In message "Re: eval/binding question"

I fail to see why it would make other code slower. I am convinced with a
proper compiler implementation there would'nt be any speed difference
for local variables introduced via explicit assignment. Access to
predeclared variables would use an optimized access method, whereas
access to variable names whose declaration point cannot be determined by
statical program analysis would use a less efficient access method.

I wonder in which cases this would be useful. This code is artificial,
since one would never write test2 like you did:

def test2
eval "x=25"
print "x=#{x}\n"
end

If you write something similar but with a string coming from somewhere
else

def test2(s)
eval s
print "x=#{x}\n"
end

you know already that "s" must contain an assignment to "x". So it's not
a problem to predeclare it IMHO.

I don't really see which real world problems would be solved with a more
complex lookup for locals, which would make the second variant succeed
without the explicit declaration of "x".

Kind regards

robert

 

[Stefan Kaes]

--------------070406000007040004030302
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I wonder in which cases this would be useful. This code is artificial,
since one would never write test2 like you did:

def test2
eval "x=25"
print "x=#{x}\n"
end

This code is artificial because I only wrote it to demonstrate the
semantics of the current implementation.

If you write something similar but with a string coming from somewhere
else

def test2(s)
eval s
print "x=#{x}\n"
end

you know already that "s" must contain an assignment to "x". So it's not
a problem to predeclare it IMHO.

For this simple demonstration program you are absolutely right: there is
no reason to write it that way, other than to demonstrate current semantics.

I don't really see which real world problems would be solved with a more
complex lookup for locals, which would make the second variant succeed
without the explicit declaration of "x".

Kind regards

robert

The current semantics introduces an artificial barrier between code
executed in *eval "code"* vs. just writing *code*, which makes the
semantics more difficult to explain and understand. Ideally, these two
forms should have identical semantics. As I have already explained, the
complexity of variable access increases only for variables whose
declaration point cannot be determined statically and therfore would not
have a negative performance impact on most programs.

The real world example can be found in Rails. It has a function
render_partial, which takes an erb template name (tn), a value (v) and a
name=>value hash (local_assigns). It makes the value v accessible as a
local variable named "tn" and all name=>value pairs in local_assigns
accessible by their name. This works, because the whole template source
is evaluated using an eval, but requires reparsing of the erb-code each
time the template gets interpreted. And, of course, the code that
implements this function cannot know the names of the local varaiables
in advance.

As soon as one tries to abstract the erb code into a function the whole
scheme breaks down, because there is no way to dynamically declare local
variables. I solved this problem by resorting to instance variables.
That is okay for me, I just rewrote my template code, but it breaks
backwards compatibility. So this valuable optimisation cannot be
included in Rails, unless everyone rewrites their templates. For this
reason, I have not proposed it to the Rails community.

regards, stefan


--------------070406000007040004030302--

 

[Robert Klemme]

The current semantics introduces an artificial barrier between code
executed in *eval "code"* vs. just writing *code*, which makes the
semantics more difficult to explain and understand. Ideally, these two
forms should have identical semantics. As I have already explained, the
complexity of variable access increases only for variables whose
declaration point cannot be determined statically and therfore would not
have a negative performance impact on most programs.

The real world example can be found in Rails. It has a function
render_partial, which takes an erb template name (tn), a value (v) and a
name=>value hash (local_assigns). It makes the value v accessible as a
local variable named "tn" and all name=>value pairs in local_assigns
accessible by their name. This works, because the whole template source
is evaluated using an eval, but requires reparsing of the erb-code each
time the template gets interpreted. And, of course, the code that
implements this function cannot know the names of the local varaiables
in advance.

As soon as one tries to abstract the erb code into a function the whole
scheme breaks down, because there is no way to dynamically declare local
variables. I solved this problem by resorting to instance variables.
That is okay for me, I just rewrote my template code, but it breaks
backwards compatibility. So this valuable optimisation cannot be
included in Rails, unless everyone rewrites their templates. For this
reason, I have not proposed it to the Rails community.

Hm, I'm not 100% sure I modeled this properly, but does this solve the
problem?

I omitted the template name and the value as these are just special cases
of values with fixed names. But you get the picture. And there is no
reparsing needed.

def local_eval(_values, _code)
_val = nil
_bnd = binding
p local_variables
_values.each {|name, val| eval "#{name}=_val", _bnd}
p local_variables
eval _code, _bnd
end
["_values", "_code", "_val", "_bnd"]
["_values", "_code", "_val", "_bnd", "x"]
nil
=> nil

Of course you get some more local variables than you would like to but
with proper naming that should not be a problem in practice.

Kind regards

robert

 

[Stefan Kaes]

--------------060409080004040409060302
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hm, I'm not 100% sure I modeled this properly, but does this solve the
problem?

No. _code gets parsed on every call to local_eval.

I omitted the template name and the value as these are just special cases
of values with fixed names. But you get the picture. And there is no
reparsing needed.

def local_eval(_values, _code)
_val = nil
_bnd = binding
p local_variables
_values.each {|name, val| eval "#{name}=_val", _bnd}
p local_variables
eval _code, _bnd
end


["_values", "_code", "_val", "_bnd"]
["_values", "_code", "_val", "_bnd", "x"]
nil
=> nil

-- stefan

--------------060409080004040409060302--