Nested scopes, and augmented assignment

Discussion in 'Python' started by Tim N. van der Leeuw, Jul 4, 2006.

  1. Hi,

    The following might be documented somewhere, but it hit me unexpectedly
    and I couldn't exactly find this in the manual either.

    Problem is, that I cannot use augmented assignment operators in a
    nested scope, on variables from the outer scope:

    PythonWin 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit
    (Intel)] on win32.
    Portions Copyright 1994-2004 Mark Hammond () -
    see 'Help/About PythonWin' for further copyright information.
    >>> def foo():

    .... def nestedfunc(bar):
    .... print bar, ';', localvar
    .... localvar += 1
    .... localvar=0
    .... nestedfunc('bar')
    ....
    >>> foo()

    bar =Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "<interactive input>", line 6, in foo
    File "<interactive input>", line 3, in nestedfunc
    UnboundLocalError: local variable 'localvar' referenced before
    assignment
    >>>


    This is entirely counter-intuitive to me, and searching the manual for
    nested scoping rules and for augmented assignment rules, I still feel
    that I couldn't have predicted this from the docs.

    Is this an implementation artifact, bug, or should it really just
    follow logically from the language definition?

    Regards,

    --Tim
     
    Tim N. van der Leeuw, Jul 4, 2006
    #1
    1. Advertising

  2. Tim N. van der Leeuw wrote:

    > Hi,
    >
    > The following might be documented somewhere, but it hit me unexpectedly
    > and I couldn't exactly find this in the manual either.
    >
    > Problem is, that I cannot use augmented assignment operators in a
    > nested scope, on variables from the outer scope:


    <snip/>

    > Is this an implementation artifact, bug, or should it really just
    > follow logically from the language definition?


    From the docs:

    """
    An augmented assignment expression like x += 1 can be rewritten as x = x + 1
    to achieve a similar, but not exactly equal effect. In the augmented
    version, x is only evaluated once. Also, when possible, the actual
    operation is performed in-place, meaning that rather than creating a new
    object and assigning that to the target, the old object is modified
    instead.
    """

    The first part is the important one. If you expand

    x += 1

    to

    x = x + 1

    it becomes clear under the python scoping rules that x is being treated as a
    local to the inner scope.

    There has been a discussion about this recently on python-dev[1] and this
    NG - google for it.



    Regards,

    Diez


    [1] http://mail.python.org/pipermail/python-dev/2006-June/065902.html
     
    Diez B. Roggisch, Jul 4, 2006
    #2
    1. Advertising

  3. On 2006-07-04, Diez B. Roggisch <> wrote:
    > Tim N. van der Leeuw wrote:
    >
    >> Hi,
    >>
    >> The following might be documented somewhere, but it hit me unexpectedly
    >> and I couldn't exactly find this in the manual either.
    >>
    >> Problem is, that I cannot use augmented assignment operators in a
    >> nested scope, on variables from the outer scope:

    >
    ><snip/>
    >
    >> Is this an implementation artifact, bug, or should it really just
    >> follow logically from the language definition?

    >
    > From the docs:
    >
    > """
    > An augmented assignment expression like x += 1 can be rewritten as x = x + 1
    > to achieve a similar, but not exactly equal effect. In the augmented
    > version, x is only evaluated once. Also, when possible, the actual
    > operation is performed in-place, meaning that rather than creating a new
    > object and assigning that to the target, the old object is modified
    > instead.
    > """
    >
    > The first part is the important one. If you expand
    >
    > x += 1
    >
    > to
    >
    > x = x + 1
    >
    > it becomes clear under the python scoping rules that x is being treated as a
    > local to the inner scope.
    >
    > There has been a discussion about this recently on python-dev[1] and this
    > NG - google for it.


    Well no matter what explanation you give to it, and I understand how it
    works, I keep finding it strange that something like

    k = [0]
    def f(i):
    k[0] += i
    f(2)

    works but the following doesn't

    k = 0
    def f(i):
    k += i
    f(2)


    Personnaly I see no reason why finding a name/identifier on the
    leftside of an assignment should depend on whether the name
    is the target or prefix for the target.


    But maybe that is just me.

    --
    Antoon Pardon
     
    Antoon Pardon, Jul 5, 2006
    #3
  4. Antoon Pardon wrote:
    (snip)
    > Well no matter what explanation you give to it, and I understand how it
    > works,


    I'm not sure of this.

    > I keep finding it strange that something like
    >
    > k = [0]
    > def f(i):
    > k[0] += i
    > f(2)
    >
    > works but the following doesn't
    >
    > k = 0
    > def f(i):
    > k += i
    > f(2)
    >
    >
    > Personnaly I see no reason why finding a name/identifier on the
    > leftside of an assignment should depend on whether the name
    > is the target or prefix for the target.
    >


    It's not about "finding a name/identifier", it's about the difference
    between (re)binding a name and mutating an object.


    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    Bruno Desthuilliers, Jul 5, 2006
    #4
  5. Bruno Desthuilliers wrote:

    > It's not about "finding a name/identifier", it's about the difference
    > between (re)binding a name and mutating an object.


    the difference between binding and performing an operation on an object
    (mutating or not), in fact.

    this is Python 101.

    </F>
     
    Fredrik Lundh, Jul 5, 2006
    #5
  6. On 2006-07-05, Bruno Desthuilliers <> wrote:
    > Antoon Pardon wrote:
    > (snip)
    >> Well no matter what explanation you give to it, and I understand how it
    >> works,

    >
    > I'm not sure of this.


    Should I care about that?

    >> I keep finding it strange that something like
    >>
    >> k = [0]
    >> def f(i):
    >> k[0] += i
    >> f(2)
    >>
    >> works but the following doesn't
    >>
    >> k = 0
    >> def f(i):
    >> k += i
    >> f(2)
    >>
    >>
    >> Personnaly I see no reason why finding a name/identifier on the
    >> leftside of an assignment should depend on whether the name
    >> is the target or prefix for the target.

    >
    > It's not about "finding a name/identifier", it's about the difference
    > between (re)binding a name and mutating an object.


    The two don't contradict each other. Python has chosen that it won't
    rebind variables that are out of the local scope. So if the lefthand
    side of an assignment is a simple name it will only search in the
    local scope for that name. But if the lefthand side is more complicated
    if will also search the outerscopes for the name.

    Python could have chosen an approach with a "nested" keyword, to allow
    rebinding a name in an intermediate scope. It is not that big a deal
    that it hasn't, but I keep finding the result strange and somewhat
    counterintuitive.

    Let me explain why:

    Suppose someone is rather new of python and writes the following
    code, manipulating vectors:

    A = 10 * [0]
    def f(B):
    ...
    for i in xrange(10):
    A += B
    ...

    Then he hears about the vector and matrix modules that are around.
    So he rewrites his code naively as follows:

    A = NullVector(10):
    def f(B):
    ...
    A += B
    ...

    And it won't work. IMO the principle of least surprise here would
    be that it should work.

    --
    Antoon Pardon
     
    Antoon Pardon, Jul 5, 2006
    #6
  7. Antoon Pardon wrote:

    > Python could have chosen an approach with a "nested" keyword


    sure, and Python could also have been invented by aliens, powered by
    space potatoes, and been illegal to inhale in Belgium.

    have any of your "my mental model of how Python works is more important
    than how it actually works" ever had a point ?

    </F>
     
    Fredrik Lundh, Jul 5, 2006
    #7
  8. >>>>> Antoon Pardon <> (AP) wrote:

    >AP> On 2006-07-05, Bruno Desthuilliers <> wrote:
    >>> Antoon Pardon wrote:
    >>> (snip)
    >>>> Well no matter what explanation you give to it, and I understand how it
    >>>> works,
    >>>
    >>> I'm not sure of this.


    >AP> Should I care about that?


    Yes, because as long as you don't understand it, you are in for unpleasant
    surprises.

    >>> It's not about "finding a name/identifier", it's about the difference
    >>> between (re)binding a name and mutating an object.


    >AP> The two don't contradict each other. Python has chosen that it won't
    >AP> rebind variables that are out of the local scope. So if the lefthand
    >AP> side of an assignment is a simple name it will only search in the
    >AP> local scope for that name. But if the lefthand side is more complicated
    >AP> if will also search the outerscopes for the name.


    No. It will always use the same search order. But a variable that is bound
    inside the function (with an asignment) and is not declared global, is in
    the local namespace. A variable that is not assigned to inside the function
    is not in the local namespace. An assignment to A is not rebinding A, so
    it doesn't count to make the variable A local.

    >AP> Python could have chosen an approach with a "nested" keyword, to allow
    >AP> rebinding a name in an intermediate scope. It is not that big a deal
    >AP> that it hasn't, but I keep finding the result strange and somewhat
    >AP> counterintuitive.


    Maybe it would have been nice if variables could have been declared as
    nested, but I think it shows that nested variables have to be used with
    care, similar to globals. Especially not allowing rebinding in intermediate
    scopes is a sound principle (`Nested variables considered harmful').
    If you need to modify the objects which are bound to names in intermediate
    scopes, use methods and give these objects as parameters.

    >AP> Let me explain why:


    >AP> Suppose someone is rather new of python and writes the following
    >AP> code, manipulating vectors:


    >AP> A = 10 * [0]
    >AP> def f(B):
    >AP> ...
    >AP> for i in xrange(10):
    >AP> A += B
    >AP> ...


    >AP> Then he hears about the vector and matrix modules that are around.
    >AP> So he rewrites his code naively as follows:


    >AP> A = NullVector(10):
    >AP> def f(B):
    >AP> ...
    >AP> A += B
    >AP> ...


    >AP> And it won't work. IMO the principle of least surprise here would
    >AP> be that it should work.


    Well, A = f(B) introduces a local variable A. Therefore also A = A+B.
    And therefore also A += B.

    The only way to have no surprises at all would be if local variables would
    have to be declared explicitly, like 'local A'. But that would break
    existing code. Or maybe the composite assignment operators should have been
    exempted. Too late now!

    You have the same problem with:

    A = [10]
    def inner():
    A.append(2)

    works but

    A = [10]
    def inner():
    A += [2]

    doesn't. The nasty thing in this example is that although A += [2] looks
    like a rebinding syntactically, semantically there is no rebinding done.

    I think the cleaner solution is (use parameters and don't rebind):
    def inner(A):
    A.append(2)

    --
    Piet van Oostrum <>
    URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
    Private email:
     
    Piet van Oostrum, Jul 5, 2006
    #8
  9. On 2006-07-05, Fredrik Lundh <> wrote:
    > Antoon Pardon wrote:
    >
    >> Python could have chosen an approach with a "nested" keyword

    >
    > sure, and Python could also have been invented by aliens, powered by
    > space potatoes, and been illegal to inhale in Belgium.


    At one time one could have reacted the same when people suggested
    python could use a ternary operator. In the mean time a ternary
    operator is in the pipeline. If you don't want to discuss how
    python could be different with me, that is fine, but I do no
    harm discussing such things with others.

    > have any of your "my mental model of how Python works is more important
    > than how it actually works" ever had a point ?


    Be free to correct me. But just suggesting that I'm wrong doesn't help
    me in changing my mental model.

    --
    Antoon Pardon
     
    Antoon Pardon, Jul 6, 2006
    #9
  10. On 2006-07-05, Piet van Oostrum <> wrote:
    >>>>>> Antoon Pardon <> (AP) wrote:

    >
    >>AP> On 2006-07-05, Bruno Desthuilliers <> wrote:
    >>>> Antoon Pardon wrote:
    >>>> (snip)
    >>>>> Well no matter what explanation you give to it, and I understand how it
    >>>>> works,
    >>>>
    >>>> I'm not sure of this.

    >
    >>AP> Should I care about that?

    >
    > Yes, because as long as you don't understand it, you are in for unpleasant
    > surprises.


    Well if someone explains what is wrong about my understanding, I
    certainly care about that (although I confess to sometimes being
    impatient) but someone just stating he is not sure I understand?

    >>>> It's not about "finding a name/identifier", it's about the difference
    >>>> between (re)binding a name and mutating an object.

    >
    >>AP> The two don't contradict each other. Python has chosen that it won't
    >>AP> rebind variables that are out of the local scope. So if the lefthand
    >>AP> side of an assignment is a simple name it will only search in the
    >>AP> local scope for that name. But if the lefthand side is more complicated
    >>AP> if will also search the outerscopes for the name.

    >
    > No. It will always use the same search order.


    So if I understand you correctly in code like:

    c.d = a
    b = a

    All three names are searched for in all scopes between the local en global
    one. That is what I understand with your statement that [python] always
    uses the same search order.

    My impression was that python will search for c and a in the total current
    namespace but will not for b.

    > But a variable that is bound
    > inside the function (with an asignment) and is not declared global, is in
    > the local namespace.


    Aren't we now talking about implementation details? Sure the compilor
    can set things up so that local names are bound to the local scope and
    so the same code can be used. But it seems somewhere was made the
    decision that b was in the local scope without looking for that b in
    the scopes higher up.

    Let me explain a bit more. Suppose I'm writing a python interpreter
    in python. One implemantation detail is that I have a list of active
    scopes which are directories which map names to objects. At the
    start of a new function the scope list is adapted and all local
    variables are inserted to the new activated scope and mapped to
    some "Illegal Value" object. Now I also have a SearchName function
    that will start at the begin of a scope list and return the
    first scope in which that name exists. The [0] element is the
    local scope. Now we come to the line "b = a"

    This could be then executed internally as follows:

    LeftScope = SearchName("b", ScopeList)
    RightScope = SearchName("a", ScopeList)
    LeftScope["b"] = RightScope["a"]

    But I don't have to do it this way. I already know in which scope
    "b" is, the local one, which has index 0. So I could just as well
    have that line exucuted as follows:

    LeftScope = ScopeList[0]
    RightScope = SearchName("a", ScopeList)
    LeftScope["b"] = RightScope["a"]

    As far as I understand both "implementations" would make for
    a correct execution of the line "b = a" and because of the
    second possibility, b is IMO not conceptually searched for in
    the same way as a is searched for, although one could organise
    things that the same code is used for both.

    Of course it is possible I completely misunderstood how python
    is supposed to work and the above is nonesense in which case
    I would appreciate it if you correct me.

    >>AP> Python could have chosen an approach with a "nested" keyword, to allow
    >>AP> rebinding a name in an intermediate scope. It is not that big a deal
    >>AP> that it hasn't, but I keep finding the result strange and somewhat
    >>AP> counterintuitive.

    >
    > Maybe it would have been nice if variables could have been declared as
    > nested, but I think it shows that nested variables have to be used with
    > care, similar to globals. Especially not allowing rebinding in intermediate
    > scopes is a sound principle (`Nested variables considered harmful').
    > If you need to modify the objects which are bound to names in intermediate
    > scopes, use methods and give these objects as parameters.


    But shouldn't we just do programming in general with care? And if
    Nested variables are harmfull, what is then the big difference
    between rebinding them and mutating them that we should forbid
    the first and allow the second?

    I understand that python evolved and that this sometimes results
    in things that in hindsight could have been done better. But
    I sometimes have the impression that the defenders try to defend
    those results as a design decision. With your remark above I have
    to wonder if someone really thought this through at design time
    and came to the conclusion that nested variables are harmfull
    and thus may not be rebound but not that harmfull so mutation
    is allowed and if so how he came to that conclusion.


    >>AP> Let me explain why:

    >
    >>AP> Suppose someone is rather new of python and writes the following
    >>AP> code, manipulating vectors:

    >
    >>AP> A = 10 * [0]
    >>AP> def f(B):
    >>AP> ...
    >>AP> for i in xrange(10):
    >>AP> A += B
    >>AP> ...

    >
    >>AP> Then he hears about the vector and matrix modules that are around.
    >>AP> So he rewrites his code naively as follows:

    >
    >>AP> A = NullVector(10):
    >>AP> def f(B):
    >>AP> ...
    >>AP> A += B
    >>AP> ...

    >
    >>AP> And it won't work. IMO the principle of least surprise here would
    >>AP> be that it should work.

    >
    > Well, A = f(B) introduces a local variable A. Therefore also A = A+B.
    > And therefore also A += B.
    >
    > The only way to have no surprises at all would be if local variables would
    > have to be declared explicitly, like 'local A'. But that would break
    > existing code. Or maybe the composite assignment operators should have been
    > exempted. Too late now!


    Let me make one thing clear. I'm not trying to get the python people to
    change anything. Most I hope for is that they would think about this
    behaviour for python 3000.

    > You have the same problem with:
    >
    > A = [10]
    > def inner():
    > A.append(2)
    >
    > works but
    >
    > A = [10]
    > def inner():
    > A += [2]
    >
    > doesn't. The nasty thing in this example is that although A += [2] looks
    > like a rebinding syntactically, semantically there is no rebinding done.
    >
    > I think the cleaner solution is (use parameters and don't rebind):
    > def inner(A):
    > A.append(2)
    >


    For what it is worth. My proposal would be to introduce a rebinding
    operator, ( just for the sake of this exchange written as := ).

    A line like "b := a", wouldn't make b a local variable but would
    search for the name b in all active scopes and then rebind b
    there. In terms of my hypothetical interpreter something like
    the above.

    LeftScope = SearchName("b", ScopeList)
    RightScope = SearchName("a", ScopeList)
    LeftScope["b"] = RightScope["a"]

    With the understanding that b wouldn't be inserted in the local
    scope unless there was an assignment to be somewhere else in
    the function.

    The augmented assignments could then be redefined in terms of the
    rebinding operator instead of the assignment.

    I'm not sure this proposal would eliminate all surprises but as
    far as I can see it wouldn't break existing code. But I don't think
    this proposal would have a serious chance.

    In any case thanks for your contribution.

    --
    Antoon Pardon
     
    Antoon Pardon, Jul 6, 2006
    #10
  11. Antoon Pardon wrote:
    > On 2006-07-05, Piet van Oostrum <> wrote:
    >
    >>>>>>>Antoon Pardon <> (AP) wrote:

    >>
    >>>AP> On 2006-07-05, Bruno Desthuilliers <> wrote:
    >>>
    >>>>>Antoon Pardon wrote:
    >>>>>(snip)
    >>>>>
    >>>>>>Well no matter what explanation you give to it, and I understand how it
    >>>>>>works,
    >>>>>
    >>>>>I'm not sure of this.

    >>
    >>>AP> Should I care about that?

    >>
    >>Yes, because as long as you don't understand it, you are in for unpleasant
    >>surprises.

    >
    >
    > Well if someone explains what is wrong about my understanding, I
    > certainly care about that (although I confess to sometimes being
    > impatient) but someone just stating he is not sure I understand?


    From what you wrote, I cannot decide if you really understand Python's
    lookup rules and poorly express some disagreement on it, or if you just
    don't understand Python lookup rules at all.

    >
    >>>>>It's not about "finding a name/identifier", it's about the difference
    >>>>>between (re)binding a name and mutating an object.

    >>
    >>>AP> The two don't contradict each other. Python has chosen that it won't
    >>>AP> rebind variables that are out of the local scope. So if the lefthand
    >>>AP> side of an assignment is a simple name it will only search in the
    >>>AP> local scope for that name. But if the lefthand side is more complicated
    >>>AP> if will also search the outerscopes for the name.


    Now it's pretty clear you *don't* understand.

    In the second case, ie:

    k = [0]
    def f(i):
    k[0] += i

    'k[0]' is *not* a name. The name is 'k'. If we rewrite this snippet
    without all the syntactic sugar, we get something like:

    k = [0]
    def f(i):
    k.__setitem_(0, k.__getitem__(0) + i)

    Now where do you see any rebinding here ?
    >
    >>No. It will always use the same search order.

    >
    >
    > So if I understand you correctly in code like:
    >
    > c.d = a
    > b = a
    >
    > All three names


    which ones ?

    > are searched for in all scopes between the local en global
    > one.


    In this example, we're at the top level, so the local scope is the
    global scope. I assert what you meant was:

    c = something
    a = something_else

    def somefunc():
    c.d = a
    b = a

    (NB : following observations will refer to this code)

    > That is what I understand with your statement that [python] always
    > uses the same search order.


    yes.


    > My impression was that python will search for c and a in the total current
    > namespace


    what is "the total current namespace" ?

    > but will not for b.


    b is bound in the local namespace, so there's no need to look for it in
    enclosing namespaces.

    >
    >>But a variable that is bound
    >>inside the function (with an asignment) and is not declared global, is in
    >>the local namespace.

    >
    > Aren't we now talking about implementation details?


    Certainly not. Namespaces and names lookup rules are fundamental parts
    of the Python language.

    > Sure the compilor
    > can set things up so that local names are bound to the local scope and
    > so the same code can be used. But it seems somewhere was made the
    > decision that b was in the local scope without looking for that b in
    > the scopes higher up.


    binding creates a name in the current namespace. b is bound in the local
    namespace, so b is local. period.

    (snip)



    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    Bruno Desthuilliers, Jul 6, 2006
    #11
  12. >>>>> Antoon Pardon <> (AP) wrote:

    >AP> Well if someone explains what is wrong about my understanding, I
    >AP> certainly care about that (although I confess to sometimes being
    >AP> impatient) but someone just stating he is not sure I understand?


    That is just a euphemistic way of stating `I think you do not understand
    it'.

    >AP> So if I understand you correctly in code like:


    >AP> c.d = a
    >AP> b = a


    >AP> All three names are searched for in all scopes between the local en global
    >AP> one. That is what I understand with your statement that [python] always
    >AP> uses the same search order.


    The d is different because it is an attribute. So it is looked up in the
    context of the object that is bound to c. For a, b, and c it is correct.

    >AP> My impression was that python will search for c and a in the total current
    >AP> namespace but will not for b.


    The assignment to b inside the function (supposing the code above is part
    of the function body) tells the compiler that b is a local variable. So the
    search stops in the local scope. The search order is always from local to
    global. First the current function, then nested function, then the module
    namespace, and finally the builtins. The first match will stop the search.
    Now for local variables inside a function, including the parameters, the
    compiler will usually optimize the search because it knows already that it
    is a local variable. But that is an implementation detail.

    >>> But a variable that is bound
    >>> inside the function (with an asignment) and is not declared global, is in
    >>> the local namespace.


    >AP> Aren't we now talking about implementation details? Sure the compilor
    >AP> can set things up so that local names are bound to the local scope and
    >AP> so the same code can be used. But it seems somewhere was made the
    >AP> decision that b was in the local scope without looking for that b in
    >AP> the scopes higher up.


    Yes, as I (and others) have already said several times: an assignment to a
    variable inside a function body (but not an assignment to an attribute or
    part of an object) without a global declaration makes that variable a local
    variable. That is not an implementation detail; it is part of the language definition.

    >AP> Let me explain a bit more. Suppose I'm writing a python interpreter
    >AP> in python. One implemantation detail is that I have a list of active
    >AP> scopes which are directories which map names to objects. At the
    >AP> start of a new function the scope list is adapted and all local
    >AP> variables are inserted to the new activated scope and mapped to
    >AP> some "Illegal Value" object. Now I also have a SearchName function
    >AP> that will start at the begin of a scope list and return the
    >AP> first scope in which that name exists. The [0] element is the
    >AP> local scope. Now we come to the line "b = a"


    >AP> This could be then executed internally as follows:


    >AP> LeftScope = SearchName("b", ScopeList)
    >AP> RightScope = SearchName("a", ScopeList)
    >AP> LeftScope["b"] = RightScope["a"]


    >AP> But I don't have to do it this way. I already know in which scope
    >AP> "b" is, the local one, which has index 0. So I could just as well
    >AP> have that line exucuted as follows:


    >AP> LeftScope = ScopeList[0]
    >AP> RightScope = SearchName("a", ScopeList)
    >AP> LeftScope["b"] = RightScope["a"]


    >AP> As far as I understand both "implementations" would make for
    >AP> a correct execution of the line "b = a" and because of the
    >AP> second possibility, b is IMO not conceptually searched for in
    >AP> the same way as a is searched for, although one could organise
    >AP> things that the same code is used for both.


    That is the optimization I spoke of above. But it is not the problem we
    were discussing. Conceptually it is the same. It is similar to constant
    folding (replacing x = 2+3 by x = 5).

    >AP> Of course it is possible I completely misunderstood how python
    >AP> is supposed to work and the above is nonesense in which case
    >AP> I would appreciate it if you correct me.


    >AP> Python could have chosen an approach with a "nested" keyword, to allow
    >AP> rebinding a name in an intermediate scope. It is not that big a deal
    >AP> that it hasn't, but I keep finding the result strange and somewhat
    >AP> counterintuitive.
    >>>
    >>> Maybe it would have been nice if variables could have been declared as
    >>> nested, but I think it shows that nested variables have to be used with
    >>> care, similar to globals. Especially not allowing rebinding in intermediate
    >>> scopes is a sound principle (`Nested variables considered harmful').
    >>> If you need to modify the objects which are bound to names in intermediate
    >>> scopes, use methods and give these objects as parameters.


    >AP> But shouldn't we just do programming in general with care? And if
    >AP> Nested variables are harmfull, what is then the big difference
    >AP> between rebinding them and mutating them that we should forbid
    >AP> the first and allow the second?


    There is no big difference I think. Only Python doesn't have syntax for the
    former. Older versions of Python didn't even have nested scopes. maybe it
    was a mistake to add them. I think an important reason was the use in
    lambda expressions, which Guido also regrets IIRC.

    >AP> I understand that python evolved and that this sometimes results
    >AP> in things that in hindsight could have been done better. But
    >AP> I sometimes have the impression that the defenders try to defend
    >AP> those results as a design decision. With your remark above I have
    >AP> to wonder if someone really thought this through at design time
    >AP> and came to the conclusion that nested variables are harmfull
    >AP> and thus may not be rebound but not that harmfull so mutation
    >AP> is allowed and if so how he came to that conclusion.


    I don't think that was the reasoning. On the other hand I think nested
    scopes were mainly added for read-only access (to the namespace that is,
    not to the values) . But then Python doesn't forbid you to change mutable
    objects once you have access to them.
    --
    Piet van Oostrum <>
    URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
    Private email:
     
    Piet van Oostrum, Jul 6, 2006
    #12
  13. On 2006-07-06, Piet van Oostrum <> wrote:

    >>AP> Aren't we now talking about implementation details? Sure the compilor
    >>AP> can set things up so that local names are bound to the local scope and
    >>AP> so the same code can be used. But it seems somewhere was made the
    >>AP> decision that b was in the local scope without looking for that b in
    >>AP> the scopes higher up.

    >
    > Yes, as I (and others) have already said several times: an assignment to a
    > variable inside a function body (but not an assignment to an attribute or
    > part of an object) without a global declaration makes that variable a local
    > variable. That is not an implementation detail; it is part of the language definition.


    You seem to think I didn't understand this. Maybe I'm not very good
    at explaining what I mean, but you really haven't told me anything
    here I didn't already know.

    >>AP> [ ... ]
    >>AP> Now we come to the line "b = a"

    >
    >>AP> This could be then executed internally as follows:

    >
    >>AP> LeftScope = SearchName("b", ScopeList)
    >>AP> RightScope = SearchName("a", ScopeList)
    >>AP> LeftScope["b"] = RightScope["a"]

    >
    >>AP> But I don't have to do it this way. I already know in which scope
    >>AP> "b" is, the local one, which has index 0. So I could just as well
    >>AP> have that line exucuted as follows:

    >
    >>AP> LeftScope = ScopeList[0]
    >>AP> RightScope = SearchName("a", ScopeList)
    >>AP> LeftScope["b"] = RightScope["a"]

    >
    >>AP> As far as I understand both "implementations" would make for
    >>AP> a correct execution of the line "b = a" and because of the
    >>AP> second possibility, b is IMO not conceptually searched for in
    >>AP> the same way as a is searched for, although one could organise
    >>AP> things that the same code is used for both.

    >
    > That is the optimization I spoke of above. But it is not the problem we
    > were discussing.


    Could you maybe clarify what problem we are discussing? All I wrote
    was that with an assignment the search for the lefthand variable
    depends on whether the lefthand side is a simple variable or
    more complicated. Sure people may prefer to speak about (re)binding
    vs mutating variables, but just because I didn't use the prefered terms,
    starting to doubt my understanding of the language, seems a bit
    premature IMO. I'm sure there are areas where my understanding of
    the language is shaky, metaclasses being one of them, but understanding
    how names are searched doesn't seem to be one of them.

    --
    Antoon Pardon
     
    Antoon Pardon, Jul 6, 2006
    #13
  14. On 2006-07-06, Bruno Desthuilliers <> wrote:
    > Antoon Pardon wrote:
    >> On 2006-07-05, Piet van Oostrum <> wrote:
    >>
    >>>>>>It's not about "finding a name/identifier", it's about the difference
    >>>>>>between (re)binding a name and mutating an object.
    >>>
    >>>>AP> The two don't contradict each other. Python has chosen that it won't
    >>>>AP> rebind variables that are out of the local scope. So if the lefthand
    >>>>AP> side of an assignment is a simple name it will only search in the
    >>>>AP> local scope for that name. But if the lefthand side is more complicated
    >>>>AP> if will also search the outerscopes for the name.

    >
    > Now it's pretty clear you *don't* understand.
    >
    > In the second case, ie:
    >
    > k = [0]
    > def f(i):
    > k[0] += i
    >
    > 'k[0]' is *not* a name. The name is 'k'. If we rewrite this snippet
    > without all the syntactic sugar, we get something like:
    >
    > k = [0]
    > def f(i):
    > k.__setitem_(0, k.__getitem__(0) + i)
    >
    > Now where do you see any rebinding here ?


    What point do you want to make? As far as I can see, I
    didn't write anything that implied I expected k to
    be rebound in code like

    k[0] += i

    So why are you trying so hard to show me this?

    >>
    >>>No. It will always use the same search order.

    >>
    >>
    >> So if I understand you correctly in code like:
    >>
    >> c.d = a
    >> b = a
    >>
    >> All three names

    >
    > which ones ?
    >
    >> are searched for in all scopes between the local en global
    >> one.

    >
    > In this example, we're at the top level, so the local scope is the
    > global scope. I assert what you meant was:


    I'm sorry I should have been more clear. I meant it to be
    a piece of function code.

    > c = something
    > a = something_else
    >
    > def somefunc():
    > c.d = a
    > b = a
    >
    > (NB : following observations will refer to this code)
    >
    >> That is what I understand with your statement that [python] always
    >> uses the same search order.

    >
    > yes.
    >
    >> My impression was that python will search for c and a in the total current
    >> namespace

    >
    > what is "the total current namespace" ?
    >
    >> but will not for b.

    >
    > b is bound in the local namespace, so there's no need to look for it in
    > enclosing namespaces.


    Now could you clarify please. First you agree with the statement that python
    always uses the same search order, then you state here there is no need
    to look for b because it is bound to local namespace. That seems to
    imply that the search order for b is different.

    AFAIR my original statement was that the search for b was different than
    the search for a; meaning that the search for b was limited to the local
    scope and this could be determined from just viewing a line like "b = a"
    within a function. The search for a (or c in a line like: "c.d = a")
    is not limited to the local scope.

    I may see some interpretation where you may say that the search order
    for b is the same as for a and c but I still am not comfortable with
    it.

    >>>But a variable that is bound
    >>>inside the function (with an asignment) and is not declared global, is in
    >>>the local namespace.

    >>
    >> Aren't we now talking about implementation details?

    >
    > Certainly not. Namespaces and names lookup rules are fundamental parts
    > of the Python language.


    I don't see the contradiction. That Namespaces and names lookup are
    fundamentel parts of the Python language, doesn't mean that
    the right behaviour can't be implemented in multiple ways and
    doesn't contradict that a specific explanation depend on a specific
    implementation instead of just on language definition.

    >> Sure the compilor
    >> can set things up so that local names are bound to the local scope and
    >> so the same code can be used. But it seems somewhere was made the
    >> decision that b was in the local scope without looking for that b in
    >> the scopes higher up.

    >
    > binding creates a name in the current namespace. b is bound in the local
    > namespace, so b is local. period.


    I wrote nothing that contradicts that.

    --
    Antoon Pardon
     
    Antoon Pardon, Jul 6, 2006
    #14
  15. Tim N. van der Leeuw

    Terry Reedy Guest

    "Antoon Pardon" <> wrote in message
    news:...
    > And if Nested variables are harmfull,


    I don't know if anyone said that they were, but Guido obviously does not
    think so, or he would not have added them. So skip that.

    > what is then the big difference between rebinding them and mutating them


    A variable is a name. Name can be rebound (or maybe not) but they cannot
    be mutated. Only objects (with mutation methods) can be mutated. In other
    words, binding is a namespace action and mutation is an objectspace action.
    In Python, at least, the difference is fundamental.

    Or, in other other words, do not be fooled by the convenient but incorrect
    abbreviated phrase 'mutate a nested variable'.

    > that we should forbid the first and allow the second?


    Rebinding nested names is not forbidden; it has just not yet been added
    (see below).

    Being able to mutate a mutable object is automatic once you have a
    reference to it. In other words, it you can read the value, you can mutate
    it (if it is mutable).

    > I understand that python evolved and that this sometimes results
    > in things that in hindsight could have been done better.


    So does Guido. That is one explicit reason he gave for not choosing any of
    the nunerous proposals for the syntax and semantics of nested scope write
    access. In the face of anti-consensus among the developers and no
    particular personal preference, he decided, "Better to wait than roll the
    dice and make the wrong, hard to reverse, choice now". (Paraphrased quote)

    >I have to wonder if someone really thought this through at design time


    Giving the actual history of, if anything, too many people thinking too
    many different thoughts, this is almost funny.

    Recently however, Guido has rejected most proposals to focus attention on
    just a few variations and possibly gain a consensus. So I think there is
    at least half a chance that some sort of nested scope write access will
    appear in 2.6 or 3.0.

    Terry Jan Reedy
     
    Terry Reedy, Jul 7, 2006
    #15
  16. >>>>> Antoon Pardon <> (AP) wrote:

    >AP> Could you maybe clarify what problem we are discussing? All I wrote
    >AP> was that with an assignment the search for the lefthand variable
    >AP> depends on whether the lefthand side is a simple variable or
    >AP> more complicated.


    What do you mean with `the lefthand variable'? Especially when talking
    about `complicated lefthand sides'?

    >AP> Sure people may prefer to speak about (re)binding
    >AP> vs mutating variables, but just because I didn't use the prefered terms,
    >AP> starting to doubt my understanding of the language, seems a bit
    >AP> premature IMO. I'm sure there are areas where my understanding of
    >AP> the language is shaky, metaclasses being one of them, but understanding
    >AP> how names are searched doesn't seem to be one of them.


    You didn't understand it in your OP. Maybe your understanding has gained in
    the meantime?
    --
    Piet van Oostrum <>
    URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
    Private email:
     
    Piet van Oostrum, Jul 7, 2006
    #16
  17. >>>>> "Terry Reedy" <> (TR) wrote:

    >TR> "Antoon Pardon" <> wrote in message
    >TR> news:...
    >>> And if Nested variables are harmfull,


    >TR> I don't know if anyone said that they were, but Guido obviously does not
    >TR> think so, or he would not have added them. So skip that.


    I used that phrase (with correct spelling). I had supposed that it would
    be recognised as a variation of 'Global variables considered harmful',
    (William Wulf and Mary Shaw, ACM SIGPLAN Notices, 1973, 8 (2) pp. 28--34).
    I think nested variables have the same troubles as global variables, so
    they should be used with care.
    --
    Piet van Oostrum <>
    URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
    Private email:
     
    Piet van Oostrum, Jul 7, 2006
    #17
  18. Antoon Pardon wrote:

    >> have any of your "my mental model of how Python works is more important
    >> than how it actually works" ever had a point ?

    >
    > Be free to correct me. But just suggesting that I'm wrong doesn't help
    > me in changing my mental model.


    over the years, enough people have wasted enough time on trying to get
    you to understand how Python works, in various aspects. if you really
    were interested in learning, you would have learned something by now,
    and you wouldn't keep repeating the same old misunderstandings over and
    over again.

    </F>
     
    Fredrik Lundh, Jul 7, 2006
    #18
  19. Antoon "I'm no nincompoop, but I play one on the internet" Pardon wrote:

    > I don't see the contradiction. That Namespaces and names lookup are
    > fundamentel parts of the Python language, doesn't mean that
    > the right behaviour can't be implemented in multiple ways and
    > doesn't contradict that a specific explanation depend on a specific
    > implementation instead of just on language definition.


    the behaviour *is* defined in the language definition, and has nothing
    to do with a specific implementation. have you even read the language
    reference ? do you even know that it exists ?

    </F>
     
    Fredrik Lundh, Jul 7, 2006
    #19
  20. Piet van Oostrum wrote:

    > There is no big difference I think. Only Python doesn't have syntax for the
    > former. Older versions of Python didn't even have nested scopes.


    arbitrarily nested scopes, at least. the old local/global/builtin
    approach (the LGB rule) is of course a kind of nesting; the new thing is
    support for "enclosing scopes" in Python 2.1/2.2 (the LEGB rule). for
    some background, see:

    http://www.python.org/dev/peps/pep-0227/

    the section "Rebinding names in enclosing scopes" discusses the 2.X-
    specific thinking; this may be revised in 3.0 (see current python-dev
    discussions).

    </F>
     
    Fredrik Lundh, Jul 7, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Doug Tolton

    Augmented Assignment question

    Doug Tolton, Jul 16, 2003, in forum: Python
    Replies:
    6
    Views:
    480
    John Machin
    Jul 19, 2003
  2. Delaney, Timothy C (Timothy)

    RE: Augmented Assignment question

    Delaney, Timothy C (Timothy), Jul 17, 2003, in forum: Python
    Replies:
    0
    Views:
    383
    Delaney, Timothy C (Timothy)
    Jul 17, 2003
  3. Pierre Barbier de Reuille

    User-defined augmented assignment

    Pierre Barbier de Reuille, Sep 29, 2005, in forum: Python
    Replies:
    4
    Views:
    305
    Tom Anderson
    Oct 2, 2005
  4. Suresh Jeevanandam

    Augmented assignment

    Suresh Jeevanandam, Feb 21, 2006, in forum: Python
    Replies:
    8
    Views:
    333
    Terry Hancock
    Feb 21, 2006
  5. Josiah Carlson
    Replies:
    4
    Views:
    355
    Boris Borcic
    Jun 19, 2006
Loading...

Share This Page