How to access to local variables in enclosing scopes?

[Yuh-Ruey Chen]

I'm familiar with other popular scripting languages (JS and Python),
and I'm struggling to figure out Ruby's (inanely) complex scoping
mechanisms.

My questions are within the following example:

x = 10

def foo
# how to access x from here?
end

class Klass
# how to access x from here?
def bar
# how to access x from here?
end
end

lambda { x } # I can access x here but why not in foo or Klass or bar?

I sincerely hope I'm missing something here. To me, the inability to
access local variables in an enclosing scope is very counter-
intuitive. I thought Ruby was supposed to be elegant? At the very
least, I expect some sort of "nonlocal" statement similar to Python,
e.g.

def foo
nonlocal x
# I can access x in enclosing scope now
end

And no, I don't want to have to use global variables - that would just
be pollution and would be incredibly unwieldy in large projects.
Frankly, if there was a way to access local variables in enclosing
scopes, there wouldn't even need to be a special global variable
concept.

 

[David A. Black]

Hi --

I'm familiar with other popular scripting languages (JS and Python),
and I'm struggling to figure out Ruby's (inanely) complex scoping
mechanisms.

The rules are pretty simple. The def, class, and module keywords start
new local scopes. That's most of it.

My questions are within the following example:

x = 10

def foo
# how to access x from here?

end

class Klass
# how to access x from here?
def bar
# how to access x from here?
end
end

lambda { x } # I can access x here but why not in foo or Klass or bar?

I'm not sure what you mean by "why". Every language has scoping rules.
Ruby's rules include provision for closures, in that code blocks do
not start a new local scope and can be used as the bodies of anonymous
function objects.

There's no moral imperative, one way or the other. You're just
encountering and learning how Ruby handles these things. Relax

I sincerely hope I'm missing something here. To me, the inability to
access local variables in an enclosing scope is very counter-
intuitive. I thought Ruby was supposed to be elegant? At the very
least, I expect some sort of "nonlocal" statement similar to Python,
e.g.

def foo
nonlocal x
# I can access x in enclosing scope now
end

I thought you said you wanted elegance

And no, I don't want to have to use global variables - that would just
be pollution and would be incredibly unwieldy in large projects.
Frankly, if there was a way to access local variables in enclosing
scopes, there wouldn't even need to be a special global variable
concept.

Have you looked at class_eval and define_method?


class MyClass; end
m = "my_method"
MyClass.class_eval do
define_method(m) { puts "Hi from #{m}!" }
end

and similar.


David

--
Rails training from David A. Black and Ruby Power and Light:
Intro to Ruby on Rails January 12-15 Fort Lauderdale, FL
Advancing with Rails January 19-22 Fort Lauderdale, FL *
* Co-taught with Patrick Ewing!
See http://www.rubypal.com for details and updates!

 

[Robert Dober]

Ruby, in it's infinite wisdom; gives you a cake that you can have and eat.

x =3D 42
def a; x end # does not work!!
Why?
Because it is efficient, there is no closure and that means one
context less to look up at runtime.

If however you want the closure (as I often do) you can still do it

x =3D 42

class << self; self end.module_eval do
define_method :a do x end
define_method :a=3D do |new_x| x =3D new_x end
end

puts a
a =3D 43
puts a
#! The next one is important !
puts x

Note the class << self; self end.module_eval
but I guess you know why I am doing this...

... class << self
end

does not create a closure.

HTH
Robert



--=20
C'est v=E9ritablement utile puisque c'est joli.

Antoine de Saint Exup=E9ry

 

[Pit Capitain]

x = 10

def foo
# how to access x from here?
end

class Klass
# how to access x from here?
def bar
# how to access x from here?
end
end

And no, I don't want to have to use global variables - that would just
be pollution and would be incredibly unwieldy in large projects.

What is the difference between your "local" variable x, which should
be accessible from everywhere, and a global variable?

Regards,
Pit

 

[Yuh-Ruey Chen]

Thanks for the reply.

I'm not sure what you mean by "why". Every language has scoping rules.
Ruby's rules include provision for closures, in that code blocks do
not start a new local scope and can be used as the bodies of anonymous
function objects.

I understand that they create new scopes, but what I don't get is why
you can't access enclosing scopes! In most other languages, scopes are
hierarchical, in that nested scopes will include parent scopes.

JS:

var x = 10
function foo() {
x // I can still access x!
}

Python:

x = 10
def foo():
x # I can still access x!

Scheme:

(let ((x 10))
((lambda ()
x) ; I can still access x!
))

There's no moral imperative, one way or the other. You're just
encountering and learning how Ruby handles these things. Relax



I thought you said you wanted elegance

I did explicitly say "at the very least". I'd rather not have to
specify that "nonlocal x", but if it were necessary for the sake of
compatibility, then so be it. And that's another issue: if Ruby
provides all these methods that allow me to break encapsulation, I
don't see why there isn't a method that allows me to access local vars
in enclosing scopes. That's just inflexible.

With that said, I kinda understand why x can't simply refer to
enclosing scopes, because it would be fragile: a method x would be
added which would make x refer to that method instead of the var in
the enclosing scope. That's why I think there needs to be some
equivalent of "nonlocal x". For a more slightly more elegant example:

def foo
$nonlocal.x
end

or

def foo
::x # don't know if this would conflict with anything
end

Have you looked at class_eval and define_method?

   class MyClass; end
   m = "my_method"
   MyClass.class_eval do
     define_method(m) { puts "Hi from #{m}!" }
   end

and similar.

Yes I'm familiar with them, and I actually would like to use them, but
blocks can't have default arguments. Ideally, I would like to write
everything with blocks and avoid all this magical "new scope"
nonsense. It makes metaprogramming tricky in my experience.

 

[Yuh-Ruey Chen]

Ruby, in it's infinite wisdom; gives you a cake that you can have and eat..

x =  42
def a; x end # does not work!!
Why?
Because it is efficient, there is no closure and that means one
context less to look up at runtime.

Not particularly convinced with that. If Python handles it with ease
and apparently no runtime performance penalty, I don't see why Ruby
can't do the same.

I'm more convinced with the fragility argument I gave in my previous
post, which is why I proposed the "nonlocal" thing.

If however you want the closure (as I often do) you can still do it

x = 42

class << self; self end.module_eval do
  define_method :a do x end
  define_method :a= do |new_x| x = new_x end
end

puts a
a = 43
puts a
#! The next one is important !
puts x

Note the class << self; self end.module_eval
but I guess you know why I am doing this...

.. class << self
 end

does not create a closure.

That's pretty clever, albeit somewhat unreadable. I'd prefer to keep
Ruby as readable as possible

 

[Yuh-Ruey Chen]

What is the difference between your "local" variable x, which should
be accessible from everywhere, and a global variable?

Regards,
Pit

The difference that the local variable doesn't have to be defined in
global scope. Only child scopes should have access to that local
variable.

def foo
a = 10
def bar
# should somehow be able to access a
end
bar
end
foo
# here, we should not be able to access foo's a, but if there is
another a in scope, we can access that

Or to put it in Python:

a = 20
def foo():
a = 10
def bar():
print a # prints 10
bar()
foo()
print a # prints 20

 

[Stefan Lang]

The difference that the local variable doesn't have to be defined in
global scope. Only child scopes should have access to that local
variable.

def foo
a = 10
def bar
# should somehow be able to access a
end
bar
end
foo
# here, we should not be able to access foo's a, but if there is
another a in scope, we can access that

Or to put it in Python:

a = 20
def foo():
a = 10
def bar():
print a # prints 10
bar()
foo()
print a # prints 20

The first assignment to a creates a _module_ scoped variable in Python.
It is _not_ local, it can very well be accessed from anywhere via

import some_module
some_module.a

Since it's a bad idea anyway to reassign such variables
(metaprogramming might be an exception), you can use
Ruby's constants for this purpose:

module MyModule

A = 20

# can access A from anywhere within MyModule,
# including method definitions, nested classes, etc.

end

puts MyModule::A # access from outside of MyModule

At least anything that looks like local variable assignment
in Ruby is actually local variable assignment. The thing to
learn is that "module", "class" and "def" create new, empty
local scopes, they have no enclosing scope with regards to
local variables.

It is generally a better idea to genuinely try to understand
a new language and its idioms instead of bending it to ones
expectations from the beginning.

Stefan

 

[Mike Gold]

Yes I'm familiar with them, and I actually would like to use them, but
blocks can't have default arguments. Ideally, I would like to write
everything with blocks and avoid all this magical "new scope"
nonsense. It makes metaprogramming tricky in my experience.

I use often use Class.new and define_method for just the reasons you
describe: metaprogramming most naturally goes with locals bound to
closures. This style also avoids a handful of potential problems with
the "regular" way of using different scopes, in particular name
conflicts and lack of true read-only protection for variables.

I was told that I wasn't doing things in a "rubyish" way. Disregard
that BS. Use closures and define_method for metaprogramming. Your code
will be better.

Note 1.9 allows default arguments for lambdas as well as a &block
parameter. 1.8.7 allows a &block parameter where 1.8.6 does not. For
1.8.x you can get the same effect of default arguments like this

class << self
define_methodfoo) { |*args|
a, b =
case args.size
when 0
["fred", "barney"]
when 1
[args.first, "barney"]
when 2
args
else
raise ArgumentError,
"wrong number of arguments (#{args.size} for 2)"
end

puts "foo: #{a} #{b}"
}
end

foo #=> foo: fred barney
foo :a #=> foo: a barney
foo :a, :b #=> foo: a b

 

[Yuh-Ruey Chen]

2008/10/31 Yuh-Ruey Chen <[email protected]>:







The first assignment to a creates a _module_ scoped variable in Python.
It is _not_ local, it can very well be accessed from anywhere via

    import some_module
    some_module.a

Well then just wrap that example in another function:

def scope()
a = 20
def foo():
a = 10
def bar():
print a # prints 10
bar()
foo()
print a # prints 20
scope()

My point was that scopes nest as expected.

Since it's a bad idea anyway to reassign such variables
(metaprogramming might be an exception), you can use
Ruby's constants for this purpose:

    module MyModule

        A = 20

        # can access A from anywhere within MyModule,
        # including method definitions, nested classes, etc.

    end

    puts MyModule::A  # access from outside of MyModule

At least anything that looks like local variable assignment
in Ruby is actually local variable assignment. The thing to
learn is that "module", "class" and "def" create new, empty
local scopes, they have no enclosing scope with regards to
local variables.

It is generally a better idea to genuinely try to understand
a new language and its idioms instead of bending it to ones
expectations from the beginning.

Stefan

I am trying to understand Ruby with some metaprogramming practice. But
I still feel that this is a really backwards step in language design.

It's like telling me I should use GOTO in original BASIC instead of
trying to bend the language to simulate while loops. Would any sane
modern programmer do that?

 

[Yuh-Ruey Chen]

Since it's a bad idea anyway to reassign such variables
(metaprogramming might be an exception), you can use
Ruby's constants for this purpose:

    module MyModule

        A = 20

        # can access A from anywhere within MyModule,
        # including method definitions, nested classes, etc.

    end

    puts MyModule::A  # access from outside of MyModule

Oops, forgot to reply to this bit.

This is not what I'm really looking for. I don't care about accessing
variables within another unrelated scope - in fact, unless explicitly
allowed, it should not be allowed. I only care about accessing
variables in enclosing scopes.

 

[Stefan Lang]

Well then just wrap that example in another function:

def scope()
a = 20
def foo():
a = 10
def bar():
print a # prints 10
bar()
foo()
print a # prints 20
scope()

FWIW, Ruby doesn't have nested procedures like that.

I am trying to understand Ruby with some metaprogramming practice. But
I still feel that this is a really backwards step in language design.

It's like telling me I should use GOTO in original BASIC instead of
trying to bend the language to simulate while loops. Would any sane
modern programmer do that?

I don't think it's a good analogy. Ruby is more oriented towards
little objects and message passing than Pascal-like nested
procedures.

I'm sure we could work out an elegant solution to some higher
level goal than "accessing local variables", especially when it
comes to metaprogramming.

Stefan

 

[Mike Gold]

This is not what I'm really looking for. I don't care about accessing
variables within another unrelated scope - in fact, unless explicitly
allowed, it should not be allowed. I only care about accessing
variables in enclosing scopes.

Just to verify we are on the same page: define_method solves all your
problems, right? If you don't wish to use Ruby 1.9 (which will be
officially released in two months) then the corner case is default
arguments, which can be done via the example I gave.

Keep in mind that you are looking at just one side of the coin. There's
a flip side to it. Preventing local variables from willy-nilly
infecting your methods is a Good Thing, and is usually what you want.

x = 10

def foo
# how to access x from here?
end

class Klass
# how to access x from here?
def bar
# how to access x from here?
end
end

If you really want x to be globally accessible then you should do

require 'ostruct'
GLOBALS = OpenStruct.new
GLOBALS.x = 10

def foo
GLOBALS.x
end

class Klass
GLOBALS.x
def bar
GLOBALS.x
end
end

This makes your intention explicit. If you know python, you've probably
heard the phrase, "Explicit is better than implicit."

 

[Yuh-Ruey Chen]

Just to verify we are on the same page: define_method solves all your
problems, right?  If you don't wish to use Ruby 1.9 (which will be
officially released in two months) then the corner case is default
arguments, which can be done via the example I gave.

I was not aware that this was being fixed. Thanks for the tip.

I've also noticed that there is a break in symmetry between |def| and |
define_method| besides this block issue. Compare:

x = 0
C.define_methodfoo) do
puts x # refers to outside x
puts self.x # refers and calls this object's method x
end

with:

x = 0
C.module_eval do
def foo
# no way to refer to outside x, but something like |scope.x| would
be nice for symmetry
puts x # refers and calls this object's method x
end
end

These would be functionally equivalent if there were some way,
analagous to the former's |self.x|, to access vars in enclosing
scopes. I'd argue that such a symmetry would be elegant.

Keep in mind that you are looking at just one side of the coin.  There's
a flip side to it.  Preventing local variables from willy-nilly
infecting your methods is a Good Thing, and is usually what you want.

I agree, it's a good thing.

x = 10
def foo
x # accessed outside x
end

wouldn't be a good thing, because adding a new method called "x" would
ruin it.

But something like:

x = 10
def foo
scope.x
end

would not be as fragile, assuming |scope| is a new builtin method that
does what you'd expect it do.

If you really want x to be globally accessible then you should do

require 'ostruct'
GLOBALS = OpenStruct.new
GLOBALS.x = 10

def foo
  GLOBALS.x
end

class Klass
  GLOBALS.x
  def bar
    GLOBALS.x
  end
end

This makes your intention explicit.  If you know python, you've probably
heard the phrase, "Explicit is better than implicit."  

Except I do NOT want x to be globally accessible. I just want to to be
accessible be child scopes. If I wanted it to be globally accessible,
then I would just use global variables.

 

[Yuh-Ruey Chen]

FWIW, Ruby doesn't have nested procedures like that.

According to my copy of Ruby, it does.

I don't think it's a good analogy. Ruby is more oriented towards
little objects and message passing than Pascal-like nested
procedures.

Then why the first-class code blocks? Certainly makes Ruby a
functional language, if only for blocks.

I'm sure we could work out an elegant solution to some higher
level goal than "accessing local variables", especially when it
comes to metaprogramming.

The ability to access local vars in enclosing scopes is strictly
unnecessary. But the same could also be said for instance_variable_get/
defined?/set.

 

[Stefan Lang]

According to my copy of Ruby, it does.

Looks can be deceiving:

$ cat nest_def.rb
class Foo
def a
puts "in a"
def b
puts "in b"
end
b
end
end

f = Foo.new
p Foo.instance_methods(false)
f.a
p Foo.instance_methods(false)
f.b
$ ruby nest_def.rb
["a"]
in a
in b
["b", "a"]
in b

A def inside a method definition defines a method in
self's class. It is not local to the enclosing method definition.

Then why the first-class code blocks? Certainly makes Ruby a
functional language, if only for blocks.

Yes, Ruby has a good deal of functional features and they
fit in very nicely with the current scoping rules.

Although I'm not sure what you mean with "first class" code
blocks. The block syntax is actually tied to message passing.

The ability to access local vars in enclosing scopes is strictly
unnecessary. But the same could also be said for instance_variable_get/
defined?/set.

Ruby does let you access local vars in enclosing scopes. It's
only that your definition of enclosing scope differs from Ruby's.

Try to write some programs and you'll see it all fits together nicely.

Many design decisions may seem arbitrary. We could also debate
the whole day about explicit self, assigning to local variables in
outer scopes, etc. in Python. It's useless.

In the end, you'll find out what looks like a big problem now
actually isn't. The most important usecase for your desired
scoping rules would be nested functions, and Ruby doesn't
have them. And for the other use cases, constants, class-
and module instance variables suffice. OTOH, you can nest
blocks as much as you like and scoping is what you expect.

Stefan

 

[Mike Gold]

I agree, it's a good thing.

Here you agree, but earlier in the same post you disagree:

I've also noticed that there is a break in symmetry between |def| and |
define_method| besides this block issue. Compare:

The break in symmetry is intentional because it prevents local variables
from willy-nilly infecting your methods, which you agreed is a good
thing.

These would be functionally equivalent if there were some way,
analagous to the former's |self.x|, to access vars in enclosing
scopes. I'd argue that such a symmetry would be elegant.


x = 10
def foo
x # accessed outside x
end

wouldn't be a good thing, because adding a new method called "x" would
ruin it.

But something like:

x = 10
def foo
scope.x
end

would not be as fragile, assuming |scope| is a new builtin method that
does what you'd expect it do.

But then you've missed out on the advantage of lexical local variables.
What you propose takes the "lexical" out of "lexical scope". It is just
nested variables which are visible from other scopes.

If that is what you wish, then you do not mean local (lexical)
variables. You mean shared data, and Ruby wants you to be explicit
about when you want shared data that will "infect" your method and class
definitions. This is a good thing.

require 'ostruct'

module Top
SHARED = OpenStruct.new
SHARED.x = 10

class Foo
def bar
p SHARED.x
end
end
end

Top::Foo.new.bar #=> 10

Lexical variables are used when you most definitely DO NOT want that.
It is useful to have this distinction, because with lexicals you are
guaranteed that no such infection can occur. (Unless you explicitly
want it via define_method.)

Except I do NOT want x to be globally accessible. I just want to to be
accessible be child scopes. If I wanted it to be globally accessible,
then I would just use global variables.

So don't make it global, as in the Top example above.

 

[ara.t.howard]

But something like:

x = 10
def foo
scope.x
end

would not be as fragile, assuming |scope| is a new builtin method that
does what you'd expect it do.

you can write it yourself now


cfp:~ > cat a.rb
class C
include(
Module.new do
y = 42
def x() y end

define_method :method_missing do |_|
if((_ = local_variables.index(_.to_s)))
eval local_variables[_]
else
super
end
end
end
)
end

p C.new.x



cfp:~ > ruby a.rb
42


i might add, however, that a function that has access to non-local
state it generally called 'a method on an object with instance
variables' and ruby has several methods to accomplish this . one
very simple method is this


cfp:~ > cat a.rb
object = Module.new do
@x = 42

def y() @x end

extend self
end


p object.y

cfp:~ > ruby a.rb
42


and of course there are many more.

regards.

a @ http://codeforpeople.com/

 

[Yuh-Ruey Chen]

Replying to multiple posts:

According to my copy of Ruby, it does.

Looks can be deceiving:

$ cat nest_def.rb
class Foo
def a
puts "in a"
def b
puts "in b"
end
b
end
end

f = Foo.new
p Foo.instance_methods(false)
f.a
p Foo.instance_methods(false)
f.b
$ ruby nest_def.rb
["a"]
in a
in b
["b", "a"]
in b

A def inside a method definition defines a method in
self's class. It is not local to the enclosing method definition.

Wow, I did not realize that. That changes my thinking a lot.

Yes, Ruby has a good deal of functional features and they
fit in very nicely with the current scoping rules.

Although I'm not sure what you mean with "first class" code
blocks. The block syntax is actually tied to message passing.

Well, blocks aren't first class per-se, but they can easily be turned
into first-class procs.

Ruby does let you access local vars in enclosing scopes. It's
only that your definition of enclosing scope differs from Ruby's.

Try to write some programs and you'll see it all fits together nicely.

Many design decisions may seem arbitrary. We could also debate
the whole day about explicit self, assigning to local variables in
outer scopes, etc. in Python. It's useless.

In the end, you'll find out what looks like a big problem now
actually isn't. The most important usecase for your desired
scoping rules would be nested functions, and Ruby doesn't
have them. And for the other use cases, constants, class-
and module instance variables suffice. OTOH, you can nest
blocks as much as you like and scoping is what you expect.

I recognize that access to nonlocal vars can be simulated with
instance variables. It does seem weird to allow two different
paradigms that interact with each other in a confusing - or at the
least non-trivial - matter (just look at my confusion).

I guess my big beef with Ruby is that it is too complex and bloated
with features that often are inconsistent or lack symmetry with each
other. It's too "magical", making it hard to learn exactly how the
language works. From a language design perspective, Ruby really seems
like a mess to me, although with moments of brilliance. Most of the
time, when I'm metaprogramming, I'm guessing and looking at the
output, hoping I got so-and-so quirk correct.

Actually, I think this all could be mitigated to a certain extent if
the documentation for Ruby were improved. I could not find an official
language reference besides the API documentation, which is not
sufficient by itself. Obviously it did not contain much information on
scoping or non-API language features.

Here you agree, but earlier in the same post you disagree:



The break in symmetry is intentional because it prevents local variables
from willy-nilly infecting your methods, which you agreed is a good
thing.

I agreed in the sense that without extra syntax, it would be fragile,
but as my example pointed out, it doesn't necessarily have to be
fragile.

But then you've missed out on the advantage of lexical local variables.
What you propose takes the "lexical" out of "lexical scope".  It is just
nested variables which are visible from other scopes.

No, what I am proposing is a mechanism for lexical scope. This is what
I mean with that |scope.x| - treat x as if it were from an enclosing
scope.

So don't make it global, as in the Top example above.

But they are not local variables. Local vars for a function are
created each time the function is called. Having access to those local
vars outside of the function makes no sense at all.

But something like:

x = 10
def foo
scope.x
end

would not be as fragile, assuming |scope| is a new builtin method that
does what you'd expect it do.

you can write it yourself now

cfp:~ > cat a.rb
class C
include(
Module.new do
y = 42
def x() y end

define_method :method_missing do |_|
if((_ = local_variables.index(_.to_s)))
eval local_variables[_]
else
super
end
end
end
)
end

p C.new.x

cfp:~ > ruby a.rb
42

That's, um, interesting. Less efficient than I hoped, but then again
Ruby is not known for its efficiency. Though that method_missing does
give me an idea...

 

[Mike Gold]

No, what I am proposing is a mechanism for lexical scope. This is what
I mean with that |scope.x| - treat x as if it were from an enclosing
scope.

I understand what you are proposing. It cannot be called a mechanism
for lexical scope, because it's fundamentally inconsistent with the
term. A 'def' method definition has a _different_ scope, not a nested
scope. _By definition_, a scope cannot refer to lexical variables in a
different scope. That's the whole point of lexical local variables.

There's a reason why Lisp moved to lexical scoping and never looked
back. Ruby uses lexical scoping for the exact same reason. It is
better.

Again, just make your shared data explicit, like the SHARED struct in
the previous example. That's all. Others will be glad you were upfront
about what is shared. When debugging, it's valuable to know at a glance
what variables can possibly affect state.

Think of the alternative. When tracking down a bug, under your proposal
you would have to check -- recursively! -- for scope.x references to
each and every variable which exists.

But they are not local variables. Local vars for a function are
created each time the function is called. Having access to those local
vars outside of the function makes no sense at all.

This paragraph does not make sense to me. In my example, the SHARED
OpenStruct instance is not a local variable. It starts with a capital
letter.