Class Variable Access and Assignment

[Antoon Pardon]

No matter wat the OO model is, I don't think the following code
exhibits sane behaviour:

class A:
a = 1

b = A()
b.a += 2
print b.a
print A.a

Which results in

3
1

... a = [1]
...

b=C()
b.a += [2]
b.a [1, 2]
C.a

[1, 2]

I can understand that Guido was a bit reluctant to introduce
+= etc into Python, and it's important to understand that they
typically behave differently for immutable and mutable objects.

All fine by me. I won't be using python any less because of this,
because I use class variable very little and you can avoid this
problem by avoiding instance that shadow class variables and
always refer to class variables by class name.

But that doesn't mean we should consider this kind of behaviour
as it should be, just because it is in python.

 

[Magnus Lycka]

That is an implemantation detail. The only answer that you are given
means nothing more than: because it is implemented that way.

Something that is written in the language reference is not
an implementation detail. Every implementation that aims to
be Python must follow this. It's a design decision.

Whether you like it or not, you will find out that the behaviour
of Python is largely based on an idea of an underlying structure.
A lot of the syntax is basically just convenient ways to access
this structure, and there is a strong tradition to avoid magic.

The explicit use of self might be the most obvious example of that,
but you can find a lot of other things in Python that shows you
this, __dict__ for instance.

I agree that the behaviour you are questioning isn't completely
unsurprising for someone who stumbles over it the first time, but
considering how things work in Python classes, where the class
scope is searched if a name isn't found in the instance scope
(self.__dict__), any other solution would involve more magic, and
be more surprising to someone who actually knows what is going on.

It's possible that a oldie like me, who started coding Python in
1996 is just blind to the warts in Python by now, but no language
is perfect, and whatever design decisions you make, they will have
both positive and negative consequences.

I frankly don't understand what you are after Antoon. Just to
vent your frustrations? If you want to use Python in an effective
way, try to learn how to use the language that actually exists.

Asking questions in this forum is clearly a part of that, but
your confrontational style, and idea that everything that bothers
you is a language bug that needs to be fixed is not the most
constructive approach. I'm pretty sure that it doesn't really solve
your coding problems, instead it leads the discussion away from the
practical solutions.

If you really want to improve the Python language, your approach
is completely off target. First of all, this isn't really the right
forum for that, and secondly, improvements to Python requires a
lot of cooperation and substantial contributions of work, not just
complaints, even if you might have a point now and then.

 

[Antoon Pardon]

class RedList(list):
colour = "red"

L = RedList(())

What behaviour would you expect from len(L), given that L doesn't have a
__len__ attribute?

Since AFAICT there is no single reference to the __len__ attribute that
will be resolved to two different namespace I don't see the relevance.

It seems to me that you do.

You didn't appear to be objecting to a line like x = b.a assigning the
value of 1 to x (although perhaps you do). If that was the case, then it
is perfectly reasonable to expect b.a = x + 2 to store 3 into b.a, while
leaving b.__class__.a untouched.

Of course, if you object to inheritance, then you will object to x = b.a
as well.

What I object to is that the mentioning of one instance gets resolved
to two different namespaces.

That's an implementation detail only in the sense that "while condition"
is a loop is an implementation detail. It is a *design* detail.

b is a name, and any reference to b (in the same namespace) will refer
to the same object. At least until you rebind it to another object.

But some namespaces take great care not to allow a rebinding that would
result in the same name being resolved to a different namespace during
this namespace's lifetime.

But b.a is not a name, it is an attribute lookup,

An other implementation detail. b.a is a name search of 'a' in the
namespace b.

and by Python's rules of
inheritance that lookup will look up attributes in the instance, the
class, and finally any superclasses.

If you persist in thinking of b.a as a name referring to a single object,
of course you will be confused by the behaviour. But that's not what
attribute lookup does.

On the right hand side of an assignment, it will return the first existing
of b.__dict__['a'] or b.__class__.__dict__['a']. On the left hand of an
assignment, it will store into b.__dict__['a'].

That holly python does it this way, doesn't imply it is reasonable to
do it this way or that all consequences of doing it this way are
reasonable.

Then it seems to me you have some serious design problems. Which would you
prefer to happen?

# Scenario 1
# imaginary pseudo-Python code with no inheritance:
class Paragraph:
ls = '\n' # line separator

para = Paragraph()
para.ls

=> AttributeError - instance has no attribute 'ls'


# Scenario 2
# imaginary pseudo-Python code with special inheritance:
class Paragraph:
ls = '\n' # line separator

linux_para = Paragraph()
windows_para = Paragraph()
windows_para.ls = '\n\r' # magically assigns to the class attribute
linux_para.ls

=> prints '\n\r'

# Scenario 3
# Python code with standard inheritance:
class Paragraph:
ls = '\n' # line separator

linux_para = Paragraph()
windows_para = Paragraph()
windows_para.ls = '\n\r'
linux_para.ls

=> prints '\n'

I don't see the relevance of these pieces of code. In none of them is
there an occurence of an attribute lookup of the same attribute that resolves
to different namespaces.

 

[Antoon Pardon]

Well, then you must think this code is *completely* insane too:

py> x = 0
py> for i in range(1, 5):
... x += i
... print id(x)
...
140838200
140840184
140843160
140847128

Look at that: the object which is referred to depends on how many times
you've already been through the loop. How nuts is that?

It is each time the 'x' from the same name space. In the code above
the 'a' is not each time from the same namespace.

I also think you new very well what I meant.

 

[Antoon Pardon]

Except they *don't*. This happens in any language that resolves
references at run time.

Python doesn't resolve references at run time. If it did the following
should work.

a = 1
def f():
a = a + 1

f()

But letting that aside. There is still a difference between resolving
reference at run time and having the same reference resolved twice
with each resolution a different result.

Changing that would be changing a fundamental
- and *important* - feature of Python. Arbitrary restrictions to
prevent a single case of this from doing something people who aren't
used to suvh behavior are kludges, and would constitute a wart on the
language, pure and simple.

Python already has its warts. If you want to argue that fixing this
would make a bigger wart then the wart it is now. Fine I will accept
that.

If you think this is bad, you should consider Jensen's device. It uses
call-by-name, which Python doesn't have.

Actually, I would have thought it very interesting should python
have provided some choice in parameter semantics.

That's a wart in +=, nothing less. The fix to that is to remove +=
from the language, but it's a bit late for that.

<mike

Well we agree that there is a wart somewhere.

 

[Antoon Pardon]

Excuse me. The statement

a += 2

causes a to refer to a different object after the assignment than it did
before. So does the statement

But the 'a' is both times in the same namespace.

self.a += 2

In this case the 'a' is not necessarily both times in the same
name space.

So why are you so concerned that the pre-assignment reference comes from
a different scope to the post-assignment reference? The fact remains
that after both assignments the rebound name can no longer (ever) be
used to refer to its former referent without a further rebinding taking
place.

I concerned with the a refering to different variables. A variable being
a name in a specific namespace.

Wring, wring, wring. (Sorry, got that wrong

Yes. So does this mean you also have a problem with

def f(x):
x += 2

g = 3
print f(g)

When the function call executes, the name x is bound to an object in the
call's containing scope. Is it then your contention that the augmented
assignment in the function should add two to that object, changing the
value of g?

Whether I have a problem with this specific behaviour or not is
irrelevant. In this case we have only one namespace with an 'x'.
So searching for 'x' will not result in different variables being
found.

It doesn't, it simply proceeds along the lines of all Python assignments
and resolves the name as a reference to a specific object. It then
computes a new value from the referenced object and the augmented
assignment operator's right operand, and rebinds the name to the
newly-computed value.

Please stop talking about variables.

No I think variable is the right term here. It refers to a name
in a specific namespace.

Although augmented assignment operators have the *option* of updating
objects in place, surely not even you can require that they do so when
they are bound to an immutable object such as an integer.

No but I can require that the namespace to which a name is resolved,
doesn't change during the operation.

Why "should" it? Why, why, why? And gain, just for good measure, why?
Augmented assignment to a function argument doesn't modify the passed
object when immutable, and you have no problem with that (I know as I
write that this is just asking for trouble, and it will turn out that
you also find that behavior deeply controversial ...)

Repeat after me: "Python assignment binds values to names".

But not just to a name, it binds a name in a specific namespace.

When I write

class something:
a = 1
def __init__(self, val=None):
if val:
self.a += val

then in the last statement the augmented assignment rebinds "self.a"
from the class "variable" to a newly-created instance "variable". I am
of course using the word "variable" here in a Pythonic sense, rather
than in the sense that, say, a C programmer would use. In Python I
prefer to talk about binding names because talking of variables leads
people to expect that a name is bound to an area of memory whose value
is modified by assignment, but this is in fact not so.

The initial access to self.a uses the defined name resolution order to
locate a value that was bound to the name "a" in class scope. So what?
This is a long-documented fact of Python life. It's *supposed* to be
that way, dammit.

That it is documented, doesn't make it sane behaviour. Otherwise
all companies had to do was to document there bugs.

In a line like b.a += 2, you only have one reference to a name
to be resolved in a spefied namespace (hierarchy). Since there
is only one reference I don't think it is sane that two resolutions
are done with two different variables as a result.

I fail to understand why this is such a problem for you. But then it's
clear from long past experience that our perceptions of Python's
execution model differ quite radically, and that I seem to find it quite
satisfactory overall, whereas you are forever banging on about what
"should" be true of Python and what Python "should" do. Which, as is
probably obvious by now, I sometimes find just a teeny bit irritating.
Kindly pardon my tetchiness.

I suppose ultimately I'm just more pragmatic than you.

It has nothing to do with being more pragmatic. Being pragmatic
is about how you handle things with real life projects. It has
little to do with the degree in which you agree with the design
of the tool you have to work with. I would say I am more pragmatic
than most defenders of python, because when it comes done to
do my work, I just use python as best as I can, while a lot
of people here seem to think that every little criticism I have
is enough to go and look for a different language.

Plus I started
using Icon, whose assignment semantics are very similar, back in the
1970's, so Python's way of doing things fits my brain quite nicely,
thank you.

So, people can probably say that about any language they started with.
That a language suits a certain persons brain, may say more about
the person than about the language.

 

[Stefan Arentz]

....

Would it be too much to ask that in a line like.

x = x + 1.

both x's would resolve to the same namespace?

This is starting to look more like a nagging contest than a real
discussion imo.

Consider changing the semantics of what you are proposing and
think about all those Python projects that will break because they
depend on the above behaviour and even take advantage of it.

So in short: yes, it would be too much to ask

But, you can fix it in your own code. Simply make sure that your
class variables have different names or a prefix.

S.

 

[Antoon Pardon]

...


I have. I like the Python model.

Fine good for you.

Personally I don't see it as a shortcoming.

Which isn't the point.

No not at all. Just look at all the PEPs and the changes in the language
that have been made in the past. Python is very much community driven and
that shows in it's progress.

You on the other hand keep talking about emo things like 'sane' and
'madness' without giving any technical backing about these problems
that you are having with the language.

That you took those emotionally, is not my responsibility. Would you
prefered it, had I called it a wart? As far as I see it, any negative
comment on python is reacted to in pretty much the same way. So
don't blame it on me using emotional language.

 

[Stefan Arentz]

Python doesn't resolve references at run time. If it did the following
should work.

a = 1
def f():
a = a + 1

f()

No that has nothing to do with resolving things at runtime. Your example
does not work because the language is very specific about looking up
global variables. Your programming error, not Python's shortcoming.

S.

 

[Paul Rubin]

...
Consider changing the semantics of what you are proposing and
think about all those Python projects that will break because they
depend on the above behaviour and even take advantage of it.

Are you seriously saying there's lots of Python projects that would
break if this particular weirdness were fixed?

 

[Mike Meyer]

Python doesn't resolve references at run time. If it did the following
should work.

You left out a key word: "all".

a = 1
def f():
a = a + 1

f()

If Python didn't resolve references at run time, the following
wouldn't work:
.... global a
.... a = a + 1
....

But letting that aside. There is still a difference between resolving
reference at run time and having the same reference resolved twice
with each resolution a different result.

The second is a direct result of the first. The environment can change
between the references, so they resolve to different results.

Python already has its warts. If you want to argue that fixing this
would make a bigger wart then the wart it is now. Fine I will accept
that.

I've already argued that the kludges suggested to "solve" this problem
create worse problems than this. This is a simple case of something
being unexpected to those used to less dynamic languages. The other
solutions break useful functionality, and require adding special cases
to the language - which aren't special enough to break the rules.

<mike

 

[Stefan Arentz]

Are you seriously saying there's lots of Python projects that would
break if this particular weirdness were fixed?

I have no numbers of course. But, why is this a weirdness?

S.

 

[Mike Meyer]

Would it be too much to ask that in a line like.

x = x + 1.

both x's would resolve to the same namespace?

Yes. That's to much bondage for programmers who've become accustomed
to freedom. Explain why this should be illegal:
.... def __getattr__(self, name):
.... x = 1
.... return locals()[name]
.... def __setattr__(self, name, value):
.... globals()[name] = value
....

<mike

 

[Paul Rubin]

I have no numbers of course. But, why is this a weirdness?

Do you seriously think the number is larger than zero? Do you think
that's any good way to write code?

Examples of the weirdness have already been given. My favorite is the
one where b.a is a list instead of an integer, in which case the class
variable gets updated instead of an instance variable getting created.
If you don't find the inconsistency to be weird, then ducky for you.

 

[Paul Rubin]

I've already argued that the kludges suggested to "solve" this problem
create worse problems than this.

The most obvious solution is to permit (or even require) the
programmer to list the instance variables as part of the class
definition. Anything not in the list is not an instance variable,
i.e. they don't get created dynamically. That's what most other
languages I can think of do. Some Python users incorrectly think this
is what __slots__ does, and try to use __slots__ that way. That they
try to do that suggests that the approach makes some sense.

 

[Antoon Pardon]

Something that is written in the language reference is not
an implementation detail. Every implementation that aims to
be Python must follow this. It's a design decision.

I have looked and didn't find it in the language reference.

This is what I have found:

An augmented assignment expression like x += 1 can be rewritten
as x = x + 1 to achieve a similar, but not exactly equal effect.

I think one could argue that in the case of b.a += 1 and a
being a class variable that incrementing the class variable
was a similar effect in this case.

But I can be and maybe a more strict definition is available
that I looked over. Do happen to know one?

Whether you like it or not, you will find out that the behaviour
of Python is largely based on an idea of an underlying structure.
A lot of the syntax is basically just convenient ways to access
this structure, and there is a strong tradition to avoid magic.

Fine. I already wrote that if people think that changing this
behaviour would cause more problems than it solved or that
solving it would cause more problems than it is worth, I would
have no problem with that.

That doesn't change the fact that the current behaviour is
on occasions awkward or whatever you want to call it.

The explicit use of self might be the most obvious example of that,
but you can find a lot of other things in Python that shows you
this, __dict__ for instance.

I agree that the behaviour you are questioning isn't completely
unsurprising for someone who stumbles over it the first time, but
considering how things work in Python classes, where the class
scope is searched if a name isn't found in the instance scope
(self.__dict__), any other solution would involve more magic, and
be more surprising to someone who actually knows what is going on.

It would be more suprising to someone depending on what is now
going on. I also find that people underestimate the magic that
is going on in python. But just because you are familiar with
the magic, doesn't make it less magic. IMO python shows its
history a little.

It's possible that a oldie like me, who started coding Python in
1996 is just blind to the warts in Python by now, but no language
is perfect, and whatever design decisions you make, they will have
both positive and negative consequences.

I completely agree. Personnaly I find python has withstood
its changes remarkebly well and I find the design in general
still very consistent despite the changes it underwent.

I frankly don't understand what you are after Antoon. Just to
vent your frustrations? If you want to use Python in an effective
way, try to learn how to use the language that actually exists.

I'm after nothing particular. The only thing I'm frustrated about
is the way in which some people seem willing to defend python
just because it is python. If the only reaction I would have
gotten would have been something like: Yeah that seems a bit
awkward but fixing this would break more than it would cure,
I would have left it as it is.

Asking questions in this forum is clearly a part of that, but
your confrontational style, and idea that everything that bothers
you is a language bug that needs to be fixed is not the most
constructive approach.

I have rarely indicated I wanted things to be fixed. Sure I would
like it if some things were different, but I recognize that there
are more important things that needs to be resolved.

Does that mean I shouldn't mention things that IMO could have been
better or that I should only mention them in the softest of
language that certainly can't be interpreted as emotional language.

 

[Antoon Pardon]

Would it be too much to ask that in a line like.

x = x + 1.

both x's would resolve to the same namespace?

Yes. That's to much bondage for programmers who've become accustomed
to freedom. Explain why this should be illegal:
... def __getattr__(self, name):
... x = 1
... return locals()[name]
... def __setattr__(self, name, value):
... globals()[name] = value
... 2

I'll answer with a contra question.

Please explain why this is illegal.

x = 1
def f():
x += 1

f()

IMO your example and mine are essentially the same issue. A name in one
namespace shadowing a name in a different namespace.

So please explain how the same kind of bondage is no problem in the
function but is too much for those who've become accustomed to
freedom in the case of objects with class variables?

 

[Antoon Pardon]

No that has nothing to do with resolving things at runtime. Your example
does not work because the language is very specific about looking up
global variables. Your programming error, not Python's shortcoming.

It has nothing to do with global variables, the same thing happens
with nested scopes.

def f():
a = 1
def g():
a = a + 1

g()

f()

 

[Antoon Pardon]

You left out a key word: "all".


If Python didn't resolve references at run time, the following
wouldn't work:

... global a
... a = a + 1
...

Why do you think so? I see nothing here that couldn't work with
a reference resolved during compile time.

The second is a direct result of the first. The environment can change
between the references, so they resolve to different results.

No the second is not a direct result of the first. Since there is
only one reference, I see nothing wrong with the environment
remebering the reference and reusing it if it needs the reference
a second time.

Take the code:

lst[f()] += 1

Now let f be a function with a side effect, that in succession
produces the positive integers starting with one.

What do you think this should be equivallent to:

t = f()
lst[t] = lst[t] + 1

or

lst[f()] = lst[f()] + 1

If you think the environment can change between references then I
suppose you prefer the second approach.

 

[Antoon Pardon]

Once again, many thanks, your explainations are very detailed and i
think i'm in full understanding of the what/when/why of it all.

And with further introspection i can see why its done this way from a
language processing point of view rather than programming one. I also
now realize that <instance>.<classvarname> is there so that you dont
have to type <instance>.__class__.<varname> all the time.

You still have to if you want to change the class variable.