Nested Scopes unintended behaviour ?

[Michael Sparks]

Hi,


Is the following behaviour expected ?

Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information..... print F("Hello")
.... print F("Hello")
.... print F("Hello")
.... print F("Hello")
.... print F("Hello")
........ on=True
.... def swtchfun(msg):
.... on_ = on
.... if on:
.... on = False
.... print "Switched to A"
.... return A
.... else:
.... print "Switched to B"
.... return B
.... #
.... return Toggler(swtchfun,True)
....Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 13, in Switcher
File "<stdin>", line 2, in Toggler
File "<stdin>", line 4, in swtchfun
UnboundLocalError: local variable 'on' referenced before assignment

The reason I ask is because logically it makes sense. The on_ = on
statement should resolve "on" to be the value on in the enclosing
scope, however it appears that the "on = False" statement is taking
priority. The reason I say this is because if you remove the "on =
False" line you get the expected name resolution:
.... print F("Hello")
.... print F("Hello")
.... print F("Hello")
.... print F("Hello")
.... print F("Hello")
........ on=True
.... def swtchfun(msg):
.... on_ = on
.... if on:
.... print "Switched to A"
.... return A
.... else:
.... print "Switched to B"
.... return B
.... #
.... return Toggler(swtchfun,True)
....Switched to A
1

ie it looks like python is not looking at the expected scope in the
first instance.

To me it looks like a bug, but I can also see a rationale where it's
considered a feature (because the "on" is on the left hand side
resolving the value to a local, rather than a value in an enclosed
scope)

I know that you can work around this as follows:
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information..... print F("Hello")
.... print F("Hello")
.... print F("Hello")
.... print F("Hello")
........ def switchgen():
.... while True:
.... yield A
.... yield B
.... G = switchgen()
.... def swtchfun(msg):
.... return G.next()
.... #
.... return Toggler(swtchfun,True)
....1
2
1
2

But I'm curious as to whether the nested scope issue above is
considered a bug or a feature, so I can deal with it appropriately.

Any comments welcome

Regards,


Michael.

 

[Emile van Sebille]

On 3/17/2010 8:16 AM Michael Sparks said...

Hi,


Is the following behaviour expected ?

In short, yes. Assignment within a function forces the variable to
locals. You can get around it like:

Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.... print F("Hello")
... print F("Hello")
... print F("Hello")
... print F("Hello")
... print F("Hello")
...... on=True
... def swtchfun(msg):

---> def swtchfun(msg, on=on):

... on_ = on
... if on:
... on = False
... print "Switched to A"
... return A
... else:
... print "Switched to B"
... return B
... #
... return Toggler(swtchfun,True)
...Traceback (most recent call last):
File "<stdin>", line 1, in<module>
File "<stdin>", line 13, in Switcher
File "<stdin>", line 2, in Toggler
File "<stdin>", line 4, in swtchfun
UnboundLocalError: local variable 'on' referenced before assignment

The reason I ask is because logically it makes sense. The on_ = on
statement should resolve "on" to be the value on in the enclosing
scope, however it appears that the "on = False" statement is taking
priority. The reason I say this is because if you remove the "on =
False" line you get the expected name resolution:
... print F("Hello")
... print F("Hello")
... print F("Hello")
... print F("Hello")
... print F("Hello")
...... on=True
... def swtchfun(msg):
... on_ = on
... if on:
... print "Switched to A"
... return A
... else:
... print "Switched to B"
... return B
... #
... return Toggler(swtchfun,True)
...Switched to A
1

ie it looks like python is not looking at the expected scope in the
first instance.

To me it looks like a bug, but I can also see a rationale where it's
considered a feature (because the "on" is on the left hand side
resolving the value to a local, rather than a value in an enclosed
scope)

I know that you can work around this as follows:
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.... print F("Hello")
... print F("Hello")
... print F("Hello")
... print F("Hello")
...... def switchgen():
... while True:
... yield A
... yield B
... G = switchgen()
... def swtchfun(msg):
... return G.next()
... #
... return Toggler(swtchfun,True)
...1
2
1
2

But I'm curious as to whether the nested scope issue above is
considered a bug or a feature, so I can deal with it appropriately.

Any comments welcome

Regards,


Michael.

 

[Terry Reedy]

On 3/17/2010 8:16 AM Michael Sparks said...

In short, yes. Assignment within a function forces the variable to
locals.

In 3.x, one can declare names to be nonlocal (ie, local to some outer
function, as opposed to local to the current function or module global).
In your case,
nonlocal on
in your inner swtchfun function would give the behavior you wanted.

Terry Jan Reedy

 

[Michael Sparks]

In 3.x, one can declare names to be nonlocal (ie, local to some outer
function, as opposed to local to the current function or module global).
In your case,
   nonlocal on
in your inner swtchfun function would give the behavior you wanted.

Ah, excellent. That makes python closures work more like I'd expect
them to. (A colleague had written the swtchfun I posted, whereas the
generator form was the version I wrote, and he was puzzled as to why
it didn't work as he expected. When I saw it I also couldn't see why.

After hearing it's expected behaviour in 2.6 it's clear that assigning
a name to a value declares the variable to be local, and that unlike
much of python (but like yield) this appears based on static analysis
of the function declaration, rather than dynamic. This does also make
sense since it prevents a name "switching scope" in a function, and a
"nonlocal" keyword also makes sense as a result.

Thanks to Emile for pointing out you can also do this in 2.6:
def Toggler(F, B):
print F("Hello")
print F("Hello")
print F("Hello")
print F("Hello")
print F("Hello")

def Switcher(A,B):
enclose={"on" : True}
def swtchfun(msg, enclose=enclose):
if enclose["on"]:
enclose["on"] = False
print "Switched to A"
return A
else:
enclose["on"] = True
print "Switched to B"
return B
#
return Toggler(swtchfun,True)

Switcher(1,2)


I think personally I'd use the generator form myself, since I think
it's clearer (more clearly loops between the two), but this may be a
useful thing to know occasionally.

Cheers


Michael.

 

[Terry Reedy]

After hearing it's expected behaviour in 2.6 it's clear that assigning
a name to a value declares the variable to be local,

unless there is a global/nonlocal declaration

and that unlike

much of python (but like yield) this appears based on static analysis
of the function declaration, rather than dynamic.

The language definition requires two passes after parsing.
One collects names and determines their scope (and looks for yield). The
second generates code.
This allows the local namespace to be implemented as an array rather
than a dict, so that local name lookup is an array index operation
rather than a dict lookup operation. This is somewhat made visible by
the dis module
b = 2
return a,b
2 0 LOAD_CONST 1 (2)
3 STORE_FAST 0 (b)

3 6 LOAD_GLOBAL 0 (a)
9 LOAD_FAST 0 (b)
12 BUILD_TUPLE 2
15 RETURN_VALUE

STORE/LOAD_FAST means store/load_local. Constants (2 in this case) are
stored in the same array. There is apparently a separate array of global
names, as opposed to local values.

Terry Jan Reedy