Newcomer question wrt variable scope/namespaces

  • Thread starter Florian Daniel Otel
  • Start date
F

Florian Daniel Otel

Hello all,

As the subject says, I am a newcomer to Python and I have a newcomer
question wrt namespaces and variable scope. Obviously, I might be
missing smth obvious, so TIA for the patience and/or pointers to
relevant resources

My problem: I just discovered (by mistake) that attempting to assign a
value to a non-local dictionary/list member does NOT generate an "
UnboundLocalError" exception and the assignment is preserved upon
exiting that scope (i.e. function). This would "violate" the python
scoping rules where a variable in a global scope can only be
referenced to (and not assigned to) -- unless declared as "global".

Attached are 3 small python scripts that illustrate my "problem". The
first one uses string tuples and behaves as expected. The other two
use dictionaries and (resp.) lists and illustrate my "problem"

TIA for any pointers,

Florian

P.S. I am not reading the newsgroup / subscribed to the mailing list,
so please Cc: me on the replys.
 
G

gene tani

Florian said:
Hello all,


Attached are 3 small python scripts that illustrate my "problem". The
first one uses string tuples and behaves as expected. The other two
use dictionaries and (resp.) lists and illustrate my "problem"

TIA for any pointers,

Florian

P.S. I am not reading the newsgroup / subscribed to the mailing list,
so please Cc: me on the replys.

ok pointers: put the code in your post (inline, not as attachments),
they're small, like you said. And maybe you ahve access to google
groups or gmane, hmmm?
 
G

Gary Duzan

Florian said:
My problem: I just discovered (by mistake) that attempting to assign a
value to a non-local dictionary/list member does NOT generate an "
UnboundLocalError" exception and the assignment is preserved upon
exiting that scope (i.e. function). This would "violate" the python
scoping rules where a variable in a global scope can only be
referenced to (and not assigned to) -- unless declared as "global".

Right. However, assigning to a['foo'] modifies the object to which
'a' refers, not the 'a' variable itself. The rule is limited to the
rebinding of variables, not the modification of the objects to which
they refer.

Gary Duzan
Motorola CHS
 
G

Gary Duzan

Florian said:
My problem: I just discovered (by mistake) that attempting to assign a
value to a non-local dictionary/list member does NOT generate an "
UnboundLocalError" exception and the assignment is preserved upon
exiting that scope (i.e. function). This would "violate" the python
scoping rules where a variable in a global scope can only be
referenced to (and not assigned to) -- unless declared as "global".

Right. However, assigning to a['foo'] modifies the object to which
'a' refers, not the 'a' variable itself. The rule is limited to the
rebinding of variables, not the modification of the objects to which
they refer.

Gary Duzan
Motorola CHS
 
F

Florian Daniel Otel

Gary,

First of all, many thanks for the reply. Do I understand it correctly
that actually the rule has to be refined as pertaining to the (so
called) "immutable" types (like e.g. integers, tuples/strings)
whereas lists and dictionaries are "mutable" types and the said
scoping rule does not apply ?

Thanks again,

Florian
 
D

Diez B. Roggisch

Florian said:
Gary,

First of all, many thanks for the reply. Do I understand it correctly
that actually the rule has to be refined as pertaining to the (so
called) "immutable" types (like e.g. integers, tuples/strings)
whereas lists and dictionaries are "mutable" types and the said
scoping rule does not apply ?

The scoping rules _do_ apply in all circumstances - they forbid changing
the binding of a name to an _object_.

So (without any scoping whatsoever):
>>> a = 1
>>> b = []
>>> id(a) 2000000
>>> id(b)
3000000

there is no way you can change the value of the object with id 2000000,
which the name a points to. Because it is immutable, which tuples are, too.

But on the list with id 3000000 you can invoke some methods that mutate it.

The scoping has _nothing_ to do with these facts. But in case of a
closure, the operation

b = 1

is forbidden, as it tries to rebind the name b to another object.
Similar, the scoping rules in case of global variables without
global-declaration allow access to the global through its name, but
prevent the global name being rebound to another value - instead,
silently a local name is created.

Regards,

Diez
 
G

Gary Duzan

Florian said:
Gary,

First of all, many thanks for the reply. Do I understand it correctly
that actually the rule has to be refined as pertaining to the (so
called) "immutable" types (like e.g. integers, tuples/strings)
whereas lists and dictionaries are "mutable" types and the said
scoping rule does not apply ?

The rule has to do with modifying namespaces. Setting a variable to
a value modifies the variable's binding in its namespace; it doesn't
matter whether the variable references a mutable or immutable type:

=========================================================================
$ python
Python 2.4.1 (#1, May 27 2005, 18:02:40)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = 1
>>> b = [1,2,3]
>>> def foo():
.... a = 9
........ b = [7,8,9]
....
>>> foo()
>>> print a 1
>>> bar()
>>> b [1, 2, 3]
>>>
=========================================================================

Note that there is no error, but the binding affect each function's
local namespace instead of the global one. However, modifying a mutable
type, in this case a list, is not a namespace binding operation, i.e.,
it doesn't make the variable point to a different object. Since you only
need to access a variable's contents to modify the object to which it
refers, you can do this:

=========================================================================.... b[2] = 6
....
=========================================================================

It is easy to think that the scoping rules give you read-only behavior,
but with mutable objects it just isn't correct.

Gary Duzan
Motorola CHS
 
M

Magnus Lycka

Florian said:
Do I understand it correctly
that actually the rule has to be refined as pertaining to the (so
called) "immutable" types (like e.g. integers, tuples/strings)
whereas lists and dictionaries are "mutable" types and the said
scoping rule does not apply ?

No! The only difference between mutable and immutable
objects are that the value of mutable objects can be
changed after object creation.

The thing you need to understand is how variables, objects
and assignments work in Python! We are always dealing with
references to objects in Python.

In e.g. C, the code "int c; c=1; c=2;" roughly means:
Make a place for an integer in memory, and let's refer
to that place as "c". Then place the value 1 in c.
Finally, replace the value 1 with the value 2 in c.

This isn't at all how Python works. I think of C variables
as differently shaped (typed) boxes with a label glued onto
the side, but I think of Python variables as a tag or label
tied to a string (string as in thin rope, not text). The
other end of that string is tied to an object in a big,
shared storage room.

C assignments mean that you make a copy of some data and
put in a box, discarding whatever was previously in that
box. Python assignment means that you untie a string from
an object and tie it to another (possibly newly created)
object. When an object no longer has any strings attached,
it's (usually) considered to be garbage.

So, in Python, "c=1; c=2" means that you have a variable
(or name) called c in the current namespace. That's not a
place for some kind of object as in C, it's just a name
that can be bound to some object in that shared storage
room that we nerds call the heap. First, c will be bound
to, or refer to, an integer object on the heap containing
the value 1. Then c will be rebound to another object, 2.

In C, you have your different boxes neatly stacked in
various namespaces. In Python the boxes are all in one place,
but your labels are neatly organized in various namespaces.

You can make "strings" like this in C too, they are called
pointers, but it's not automatic as in Python, and C
pointers are much more difficult and error prone.

So, returning to your question, scoping rules are just
the same. The issue with mutable object are that they can
be mutated. Different variables in different namespaces
can refer to the same object. E.g.
>>> l = [1,2]
>>> def addToList1(aList, value):
.... aList.append(value)
.... return aList
........ newList = aList + [value]
.... return newList
....
>>> print l [1, 2]
>>> print id(l) 182894638800
>>> l1=addToList1(l,3)
>>> print l, l1 [1, 2, 3] [1, 2, 3]
>>> print id(l), id(l1) 182894638800 182894638800
>>> l2=addToList2(l,4)
>>> print l, l2 [1, 2, 3] [1, 2, 3, 4]
>>> print id(l), id(l2)
182894638800 182894638736

You see? Calling addToList1 caused the list object that was
passed in to be modified. This modification is obviously
seen by all variables that are bound to that object, whatever
namespace they exist in.

Calling addToList2 doesn't cause any modification of the
objects that the parameters are bound to. Instead a new object
is created.

With immutable objects, you can't possibly do something along
the lines of addToList1. This means that if x is bound to an
immutable object, x will always have the same value before and
after a call to f(x), whatever code function f contains. If
x is mutable, you have no such guarantees. It depends on f...

It's not only variables that can be bound to objects. E.g:
>>> l1 = []
>>> l2 = [l1]
>>> t1 = [l1]
>>> print l1, l2, t1 [] [[]] [[]]
>>> t1[0].append(1)
>>> print l1, l2, t1
[1] [[1]] [[1]]

l1, position 0 in l2 and position 0 in t1 are all bound to the
same object, an initially empty list.
 
F

Florian Daniel Otel

Mike, Gary, Magnus

First of all, thanks to you all for the clarifications.

Python variables are just names. They refer to (are "bound" to)
objects. An assignment statement doesn't change a value. It rebinds
the variable (or lvalue) to the value on the right side of the
statement.

The Python rule is that you can only bind a local variable or a
variable declared global. It doesn't say anything at all about what
you can do to the objects so bound.

Based on Mike explanation and Gary and Magnus illustrative examples
(and a perusal of the Chapter 4 from my current bible -- "Python in a
Nutshell") I now understood the way Python handles varialbles, objects
(mutable or not) and the bindings between the two.

As Mike figured it out, coming from a "low-level" language (e.g. C,
where "call by value" and "call by reference" are like night&day and
memory mngmnt is to be handled explicitely) was a bit of a challenge.
Even if I am not exactly new to programming it wasn't quite obvious in
the first place how Python handles the mapping btw lvalues and rvalues
(the s.c. "variable binding") and how object and (respectively)
variable scoping works. One can only wish that various python texts
(like e.g. the official Python tutorial) would be more forthcoming on
this fundamental paradigm shift.

Anyway, thanks Mike, Gary and Magnus for the very useful hints and
illustrations.

Florian
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top