Nested function scope problem

[Gerhard Fiedler]

Would I do?

It seems so

If there's a binding to a name *anywhere* in the function's body then
that name is treated as local to the function.

This is a matter of static analysis, and is irrespective of where in the
body the assignment is found.

One difficulty I'm having is knowing the proper terminology when looking
for an effect like this. Just someone rephrasing the question using the
proper terms already helps a lot. "Binding" seems to be the keyword here.

It is surprising in the sense that binding seems not to be necessary for
read access.


Going back to the original question... What would be the most common/useful
way to access variables from the outer function for writing from within the
inner function?

Thanks,
Gerhard

 

[Dennis Lee Bieber]

It is surprising in the sense that binding seems not to be necessary for
read access.

Binding is the process of attaching the LHS NAME to the OBJECT that
is RHS result.

Python does not use the "mailbox model" of variables. In the mailbox
model (think of a post office sorting table, or the customer boxes in
the lobby), each "box" has one, and only one, name.

A = B

means "take the contents of box B, make a copy of those contents, leave
the original contents in B, and put the copy into box A"

Python has what I refer to as the "Post-It Note Model".

A = B

means "find the box that has a post-it note with 'A' (if there isn't
one, get a blank note and put 'A' on it), take that post-it note and
move it to the box that already has a post-it note with 'B' ". Notice
that this means that THAT box (the object) NOW has TWO Post-It notes,
each with a different "name".

Lists/dictionaries/tuples, etc. complicate this a small bit. What
you now have is the "object" (the box with the name of the list on it)
having a lot of pieces of "string" -- each string is stuck to a post-it
note with the final object. This is why

A[1] = B

is not a rebinding of A. It looks for the box with the post-it of A,
then gets the second (0 is first) "string" and follows that string to
the post-it it is stuck on... That end of the string is then moved to
the post-it with the name B
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Steve Holden]

It does, I would agree, seem a little counter-intuitive that assignment
(or binding) forces the name to be considered local. That's just one of
Python's features.

Binding is the process of attaching the LHS NAME to the OBJECT that
is RHS result.

Python does not use the "mailbox model" of variables. In the mailbox
model (think of a post office sorting table, or the customer boxes in
the lobby), each "box" has one, and only one, name.

A = B

means "take the contents of box B, make a copy of those contents, leave
the original contents in B, and put the copy into box A"

Python has what I refer to as the "Post-It Note Model".

A = B

means "find the box that has a post-it note with 'A' (if there isn't
one, get a blank note and put 'A' on it), take that post-it note and
move it to the box that already has a post-it note with 'B' ". Notice
that this means that THAT box (the object) NOW has TWO Post-It notes,
each with a different "name".

I've never really understood the value of the "Post-It" analogy. Despite
the fact that it allows you to explain that objects can have several
names, it doesn't really help in explaining that the names themselves
live in particular namespaces and have lifetimes largely independent of
the objects to which they refer.

It really seems easier (to me) to explain that names are references or
pointers to values, and that the names live in scopes that can disappear
(for example when a function or method returns). This also allows one to
explain that a function can return references to (one or more) objects,
and if those references are saved by the caller then the lifetime of the
referenced object(s) can be longer than the lifetime of the namespace in
which they were originally created.

Lists/dictionaries/tuples, etc. complicate this a small bit. What
you now have is the "object" (the box with the name of the list on it)
having a lot of pieces of "string" -- each string is stuck to a post-it
note with the final object. This is why

A[1] = B

is not a rebinding of A. It looks for the box with the post-it of A,
then gets the second (0 is first) "string" and follows that string to
the post-it it is stuck on... That end of the string is then moved to
the post-it with the name B

Well here I'd rather simply say that references can also be saved in
container objects, with all the same implications as when they are saved
in names.

This is explained extremely well in Ascher and Lutz's "Learning Python"
(at least in my first edition copy) using diagrams that make it obvious
how objects are created and what their lifetimes are likely to be.

All this business about Post-It notes and pieces of string seems
artificial in the extreme, not to say a little unhelpful. Maybe I've
just been programming too long ...

Perhaps you'd like to try explaining argument passing in terms of
Post-Its? That might be amusing ;-)

regards
Steve

 

[Dennis Lee Bieber]

Perhaps you'd like to try explaining argument passing in terms of
Post-Its? That might be amusing ;-)

No real difference, in my mind...

The names in the parameter list of the "def" statement are bound to
the objects associated with the actual call. After that, they behave
very much as locals... Now -- with defaults it gets a touch trickier...
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Gerhard Fiedler]

It does, I would agree, seem a little counter-intuitive that assignment
(or binding) forces the name to be considered local. That's just one of
Python's features.

Ok... I can live with that, and thanks for the confirmation that it's not
only me to feel that this is somewhat surprising

Surprising for me are actually two things: 1- the fact itself, and 2- that
term "binding", and that whatever it means (I'll have to read more on that,
now that I know the term) is different for read-only and read/write access.

Neither the Post-It note metaphor nor the pointer explanation address that.
Using the Post-It note metaphor, I'm asking myself why the label doesn't
get attached to a different box when reading, but only when writing. ("Just
one of Python's features", I know Same thing with the pointer
explanation: AFAIK, no language that uses pointers explicitly does
something similar (that is, the storage the pointer points to is different
depending on whether the pointer gets used for writing and reading, or only
for reading).

Thanks,
Gerhard

 

[Bruno Desthuilliers]

Ok... I can live with that, and thanks for the confirmation that it's not
only me to feel that this is somewhat surprising

Since Python has no "local variable declaration", there must be a rule
to distinguish local names from names living in the enclosing
namespaces. The rule is: unless previously declared 'global' (ie
module-level) with the appropriate statement, any name bound in the
local namespace is local. If declared 'global', it has to exist in the
global namespace.

This was much more simple to understand when we didn't have nested
functions - we mostly had global and local scope. Fact is that we now
have nested functions, but still no statement equivalent to 'global' for
parent namespaces - with the result that we cannot actually rebind
parent's namespace names from within a nested function. But we are still
free to modify any mutable object we can access, so the usual workaround
for immutable objects is to wrap them in a mutable container (list,
dict, etc):

def outer():
def _inner():
outer_var[0] = 'dead'

outer_var = ['parrot']
print "before, outer_var = %s" % outer_var
_inner()
print "after, outer_var = %s" % outer_var

and before you say it: yes, it's a dirty hack.

Surprising for me are actually two things: 1- the fact itself, and 2- that
term "binding", and that whatever it means > (I'll have to read more on that,
now that I know the term)

a "binding" is the association of a name and a reference to an object in
a given namespace. It's different from the common notion of "variable",
which is usually a symbolic name for a memory address storing a value
(like a pointer to an object's address).

is different for read-only and read/write access.

What makes the difference is that binding a name to an object in a
namespace creates the name in this namespace (unless the name as been
declared global, cf above). With the result that, the name existing in
the local namespace, it won't be looked up in enclosing namespaces.

Neither the Post-It note metaphor nor the pointer explanation address that.
Using the Post-It note metaphor, I'm asking myself why the label doesn't
get attached to a different box when reading,

The normal lookup rule for names is local namespace then enclosing
namespaces until top-level (module, aka 'global'), then builtins. Else,
you would have to declare as global any "module / function / whatever"
name in each and every function.

but only when writing.

cf above and below.

("Just
one of Python's features", I know Same thing with the pointer
explanation: AFAIK, no language that uses pointers explicitly does
something similar (that is, the storage the pointer points to is different
depending on whether the pointer gets used for writing and reading, or only
for reading).

In most languages, you have to explicitly declare local names one way or
another. Python takes the opposite route : you have to explicitly
declare global names. Since you don't declare local names, binding
creates the name if it doesn't already exists in the local namespace.


HTH

 

[Dennis Lee Bieber]

Surprising for me are actually two things: 1- the fact itself, and 2- that
term "binding", and that whatever it means (I'll have to read more on that,
now that I know the term) is different for read-only and read/write access.

Binding only happens when the bare name is on the LHS of an
assignment (or, to complicate matters, passed as an argument). Binding
has nothing to do with "read-only" vs "read/write" access.

Neither the Post-It note metaphor nor the pointer explanation address that.
Using the Post-It note metaphor, I'm asking myself why the label doesn't
get attached to a different box when reading, but only when writing. ("Just

Reading is a look-up operation -- you are looking for the object
that "currently" has the "post-it" (which is the Name) attached to it.
All read operations are the same -- find the object with label X
attached. All write operations are the same -- move/create a label X to
the object of the RHS.


The original situation of this thread, if I recall it, is not really
a matter of the binding of the name, but of the static behavior of
Python during the byte-code compilation phase. During the compilation
pass, Python finds a bare name on the LHS and, in the absence of an
immediately prior "global" statement, generates code to access that name
within the "current" stack frame (using my post-it explanation, during
the compile, a post-it "<func>.<name>" is created if <name> is on the
LHS, and reserved). Then, during execution, when <name> is seen, Python
looks first in the stack frame and sees the reserved "post-it" with
"<func>.<name>" -- and complains that this "post-it" has not yet been
bound to an object.

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Terry Reedy]

The names in the parameter list of the "def" statement are bound to
the objects associated with the actual call. After that, they behave
very much as locals... Now -- with defaults it gets a touch trickier...

A function's parameters are not just 'very much as locals', they *are*
locals.{'x': 3}

In particular, parameters are just those locals that are initialized in the
call process; it is an error for a parameter name to not become bound to
some object. The default objects fill in the slack when there are not
enough argument objects.

From the calling side, the arguments are objects to be used in that initial

binding process, either directly or as part of a new collective object. It
is an error for an argument to not be used. The calling code does not care
about the parameter names, but just their number and nature.

So one can think of calling as cross-namespace name binding followed by
control transfer. Returning is similar except that return objects may
ignored or bound to slots rather than names.

Terry Jan Reedy

 

[Gerhard Fiedler]

Binding only happens when the bare name is on the LHS of an assignment
(or, to complicate matters, passed as an argument).

This makes a lot more sense now.

Binding has nothing to do with "read-only" vs "read/write" access.

Isn't being on the LHS the only way to write to a non-mutable object? Are
there situations where binding happens without writing to a variable? Are
there write accesses to non-mutable objects where the name is not on the
LHS?

Reading is a look-up operation -- you are looking for the object that
"currently" has the "post-it" (which is the Name) attached to it.

Thanks, to you and Bruno. I think I got this now, and also why.

(And see, here you are talking about reading... see my questions above.)

Gerhard

 

[danielx]

Ok... I can live with that, and thanks for the confirmation that it's not
only me to feel that this is somewhat surprising

Since Python has no "local variable declaration", there must be a rule
to distinguish local names from names living in the enclosing
namespaces. The rule is: unless previously declared 'global' (ie
module-level) with the appropriate statement, any name bound in the
local namespace is local. If declared 'global', it has to exist in the
global namespace.

This was much more simple to understand when we didn't have nested
functions - we mostly had global and local scope. Fact is that we now
have nested functions, but still no statement equivalent to 'global' for
parent namespaces - with the result that we cannot actually rebind
parent's namespace names from within a nested function. But we are still
free to modify any mutable object we can access, so the usual workaround
for immutable objects is to wrap them in a mutable container (list,
dict, etc):

def outer():
def _inner():
outer_var[0] = 'dead'

outer_var = ['parrot']
print "before, outer_var = %s" % outer_var
_inner()
print "after, outer_var = %s" % outer_var

and before you say it: yes, it's a dirty hack.

Surprising for me are actually two things: 1- the fact itself, and 2- that
term "binding", and that whatever it means > (I'll have to read more on that,
now that I know the term)

a "binding" is the association of a name and a reference to an object in
a given namespace. It's different from the common notion of "variable",
which is usually a symbolic name for a memory address storing a value
(like a pointer to an object's address).

Wait, I'm not sure I see the difference. Isn't "reference" ~ "C
pointer". Are you saying Python variables don't hold references to
"actual" Python objects? That idea has been working well for me so far.

 

[Antoon Pardon]

Since Python has no "local variable declaration", there must be a rule
to distinguish local names from names living in the enclosing
namespaces. The rule is: unless previously declared 'global' (ie
module-level) with the appropriate statement, any name bound in the
local namespace is local. If declared 'global', it has to exist in the
global namespace.

This was much more simple to understand when we didn't have nested
functions - we mostly had global and local scope.

But having only a global an local scope, didn't prevent nested
functions. The nested functions just didn't have nested scopes
and that had it's own problems.

 

[Jeremy Sanders]

Going back to the original question... What would be the most
common/useful way to access variables from the outer function for writing
from within the inner function?

I've done something like this (which doesn't look very nice)

def myfunc():

tok = ['']

def inner():
tok[0] += 'hi'
...
tok[0] = 'foo'

inner()
print tok[0]

 

[Bruno Desthuilliers]

Bruno Desthuilliers wrote:
(snip)


Wait, I'm not sure I see the difference. Isn't "reference" ~ "C
pointer".

For a very large definition of '~' !-)

Are you saying Python variables don't hold references to
"actual" Python objects?
Exactly.

That idea has been working well for me so far.

It can only take you so far. Now it's time you know the truth: there are
*no* 'variables' in Python (hence the term 'binding').

What you really have is (somewhat simplified, of course) a dict with
names as keys and objects references (think of 'smart' pointers) as
values. So the name doesn't 'hold' anything - it's really nothing more
than a name. And the object doesn't know nothing about which names it's
bound to.

 

[Antoon Pardon]

For a very large definition of '~' !-)


It can only take you so far. Now it's time you know the truth: there are
*no* 'variables' in Python (hence the term 'binding').

That depends on what you want to mean with the term. IMO 'variables'
in python behave sufficiently similar as in other languages to use
the term.

What you really have is (somewhat simplified, of course) a dict with
names as keys and objects references (think of 'smart' pointers) as
values.

That is just an implementation issue. Nothing prevents variables to
be associated with an index in a list.

So the name doesn't 'hold' anything - it's really nothing more
than a name.

In a language like C the name doesn't hold anything either.
The name is just a way for refering to a memory space which
will hold something.

And the object doesn't know nothing about which names it's
bound to.

In a language like C, the value '3' doesn't know which variables
will hold it either.

 

[Gerhard Fiedler]

It can only take you so far. Now it's time you know the truth: there are
*no* 'variables' in Python (hence the term 'binding').

What you really have is (somewhat simplified, of course) a dict with
names as keys and objects references (think of 'smart' pointers) as
values. So the name doesn't 'hold' anything - it's really nothing more
than a name. And the object doesn't know nothing about which names it's
bound to.

I see that, and this thread and your and others explanations have been
helpful in seeing deeper.

But can't this be expressed -- maybe -- as the variable name "holding" the
reference that is associated to it through the dict? Aside the explicit
mentioning of the dict (which I agree is in itself a value, especially
since that dict exists and can be accessed as dict), the image seems to
provide the same analogy.

Gerhard

 

[Bruno Desthuilliers]

That depends on what you want to mean with the term. IMO 'variables'
in python behave sufficiently similar as in other languages

Which ones ? C ? or Lisp ? Or Haskell ?

to use
the term.
IYO.



That is just an implementation issue. Nothing prevents variables to
be associated with an index in a list.

Do you understand what "somewhat simplified" means ?

In a language like C the name doesn't hold anything either.

Yes : it holds type and storage class informations too.

The name is just a way for refering to a memory space which
will hold something.

The name is a symbolic name for a memory address in which bits will be
stored (and the type information is used to know how to interpret the
bits at this address - I leave storage class problems aside). There's a
direct translation from symbolic name to memory address to bits. Which
is not the case in Python...

In a language like C, the value '3' doesn't know which variables
will hold it either.

Indeed. Now could you tell us what was your point, exactly ?

 

[Antoon Pardon]

Which ones ? C ? or Lisp ? Or Haskell ?

Whatever, if both C and Lisp can both talk about variables, I don't
see why python can't

IYO.

So? The statement that python doesn't has variables is just your opinion.

Do you understand what "somewhat simplified" means ?

Not the difference between a list and a dictionary.

Yes : it holds type and storage class informations too.

If you want so. But that is totally irrelevant for considering
something as a variable or not. If someone would wish to extend
python with a type system, so that a variable could be associated
with type information, it would not essentially change what
one could do with an identiefier in python.

Like wise one could make a C-like language without type and
storage class information associated with identifiers. Variables
would essentially still behave the same.

The name is a symbolic name for a memory address in which bits will be
stored (and the type information is used to know how to interpret the
bits at this address - I leave storage class problems aside). There's a
direct translation from symbolic name to memory address to bits.

That doesn't need to be. A C-implementation could use directories.
Most implementations don't but that is just because it would slow
performance, but perfectly legal C-behaviour would be possible.

Besides what you say isn't true in general. There is no direct
translation from a local name to memory address. The same name
can refer to a different memory address on different invocations
of the function.

Which is not the case in Python...

An implementation detail.

Indeed. Now could you tell us what was your point, exactly ?

Just showing that the object not knowing which identifiers
it is bound to is irrelevant.

 

[Bruno Desthuilliers]

(snip)


Whatever, if both C and Lisp can both talk about variables, I don't
see why python can't

Guess where the term "binding" comes from.

So? The statement that python doesn't has variables is just your opinion.

The statement that "Python doesn't has variables" is meant to emphasis
the very difference between C-like variables and Python bindings.

yes, your favourite argument ever.

Not the difference between a list and a dictionary.

You don't understand the difference between a list and a dict ?

FWIW, the module and the classes namespaces are really implemented with
dicts. Functions namespaces are not IIRC, but I can't find the link no more.

If you want so. But that is totally irrelevant for considering
something as a variable or not.

This is relevant when talking about the difference between C variables
and Python bindings.

If someone would wish to extend
python with a type system,

In case you don't know, it already has one.

That doesn't need to be. A C-implementation could use directories.

"could", yes. But I'm not talking about language grammar.

Just showing that the object not knowing which identifiers
it is bound to is irrelevant.

Someone in this thread used a 'post-it' metaphor that more or less
implied that the name was 'post-it'ed' on the object. Which is not the
case. Hence my precision on this.

And I still fail to get the whole point of your post.

 

[Bruno Desthuilliers]

I see that, and this thread and your and others explanations have been
helpful in seeing deeper.

But can't this be expressed -- maybe -- as the variable name "holding" the
reference
s/the/a/

that is associated to it through the dict? Aside the explicit
mentioning of the dict (which I agree is in itself a value, especially
since that dict exists and can be accessed as dict), the image seems to
provide the same analogy.

What bother me with the "hold" term is that I understand it as meaning
that the name is some kind of container by itself - which it is not.
Consider the following:

d = dict()
d['name'] = 'parrot'

Would you say that the string "name" 'holds a reference' to the string
"parrot" ? Obviously not - it's the dict that holds this reference.

My 2 cents...

 

[Dennis Lee Bieber]

In a language like C the name doesn't hold anything either.
The name is just a way for refering to a memory space which
will hold something.

Except for one difference... In C (and most other languages) that
memory space is FIXED -- name "X" ALWAYS refers to memory space "Y".
Even in the case of a C pointer -- "P" always refers to one memory space
"p" wherein the contents of that memory space is a user-accessible
reference to some other memory space; to actually get to the object one
must dereference "P" using "*P" or "P->" type notation

In Python, even the location of the "name" may vary -- if you "del
name" then do some operations and rebind "name", the new "name" itself
could be somewhere else in memory.

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/