Getting rid of "self."

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

I think it would be cool if you could refer to instance variables
without prefixing with "self." I know noone else thinks like me so
Python will never be changed, but maybe you can already do it with
Python today?

..import sys
..
..def magic():
.. s = ""
.. for var in sys._getframe(1).f_locals["self"].__dict__:
.. s += var + " = self." + var + "\n"
.. return s
..
..class A:
.. def __init__(self):
.. self.hi = "yo"
..
.. def meth(self):
.. exec(magic())
.. print hi
..
..a = A()
..a.meth()

It works! exec(magic()) does the needed hi = self.hi. Not so
impressive in this case but much cooler when there is more instance
variables around. But the solution is very ugly because you have to
write exec(magic()) in every method. So I'm asking here if someone
knows a better way, maybe using decorators or metaclasses or other
black magic?

 

[Bruno Desthuilliers]

BJörn Lindqvist a écrit :

I think it would be cool if you could refer to instance variables
without prefixing with "self." I know noone else thinks like me so
Python will never be changed, but maybe you can already do it with
Python today?
(snip code)

It works! exec(magic()) does the needed hi = self.hi. Not so
impressive in this case but much cooler when there is more instance
variables around. But the solution is very ugly because you have to
write exec(magic()) in every method. So I'm asking here if someone
knows a better way, maybe using decorators or metaclasses or other
black magic?

The better way is definitively to forget about black magic and
understand why mandatory 'self' is Good Thing (tm).

(Tip : even when [this|self|@|whatsoever] is not mandatory, using it
makes for much more readable code.)

Bruno

 

[John Roth]

I think it would be cool if you could refer to instance variables
without prefixing with "self." I know noone else thinks like me so
Python will never be changed, but maybe you can already do it with
Python today?

....

It works! exec(magic()) does the needed hi = self.hi. Not so
impressive in this case but much cooler when there is more instance
variables around. But the solution is very ugly because you have to
write exec(magic()) in every method. So I'm asking here if someone
knows a better way, maybe using decorators or metaclasses or other
black magic?

[response]

Having to specify the instance explicitly is something that
Python needs because it isn't a statically typed language.
In a statically typed language, all variables are pre-declared,
so the compiler knows where they all are.

Python's compiler knows about local variables. It
doesn't know where any other variables are, so it
has to search. Including the instance as one of the
method parameters means that the search splits
right at the front: either it starts looking up the
instance, or it goes up the definition chain (usually
empty) to the module namespace and then the
builtins.

Eliminating "self" would mean it would have to
either search the instance before the module and
builtins, or search the module and builtins before
the instance. This is both a performance issue
and a maintenance issue because of the increased
possibility of one shadowing the other.

This is distinct from the issue of how to spell
"self". As another responder has already said,
you can spell it any way you want; it's simply
whatever you choose to call the first paramteter
to the method.

Going the other way, the word "self" could become
a keyword, removing the necessity of specifying it
among the method parameters. While I like the idea,
there's enough dislike of the notion that it's not going
to happen.

John Roth

 

[Jeremy Bowers]

It works! exec(magic()) does the needed hi = self.hi.

No it doesn't. Try "hi = 'newValue'" and see what happens.

So the next step is to write an "unmagic" function. So now how do you add
instance variables?

There is no way to avoid "self" *and* not pre-declare variables in some
fashion as belonging to the instance (as declarations, as sigils, what
have you). Given that Python is not, will not, and should not do the
latter, I submit that "self" is, at least for you, the lesser of two
evils. (I don't consider it evil at all, so it isn't such for me; were I
programming in C++ routinely now I'd prefix "this" and dispense with that
ugly "m_" garbage. (One of the things I ***hate*** about C++ culture is
its acceptance of hideously ugly variable names, but now I'm two
parentheticals deep so I probably ought to stop.))

 

[Roy Smith]

were I programming in C++ routinely now I'd prefix "this" and
dispense with that ugly "m_" garbage. (One of the things I ***hate***
about C++ culture is its acceptance of hideously ugly variable names,
but now I'm two parentheticals deep so I probably ought to stop.))

I'm currently working in a C++ system where they have a wrapper class
that provides some transaction locking functionality for member access.
The colloquial name for the wrapped "this" pointer is self, i.e. they do
"self = wrapper (this)" at the beginning of functions that need it. You
can then do "member" to get the bare access or "self.member" to get the
locking functionality.

It's actually kind of neat, but boy does it play headgames with me when
I switch back and forth between that and Python.

 

[Nick Coghlan]

It's actually kind of neat, but boy does it play headgames with me when
I switch back and forth between that and Python.

Switching back and forth betwen C++ and Python plays headgames *anyway* }:>

Cheers,
Nick.
Hardware control with Python is nice. . .

 

[Tim Roberts]

I think it would be cool if you could refer to instance variables
without prefixing with "self." I know noone else thinks like me so
Python will never be changed, but maybe you can already do it with
Python today?

.import sys
.
.def magic():
. s = ""
. for var in sys._getframe(1).f_locals["self"].__dict__:
. s += var + " = self." + var + "\n"
. return s
.
.class A:
. def __init__(self):
. self.hi = "yo"
.
. def meth(self):
. exec(magic())
. print hi
.
.a = A()
.a.meth()

It works! exec(magic()) does the needed hi = self.hi.

Does it?

class A:
def __init__(self):
self.hi = "yo"

def meth(self):
exec(magic())
print hi
hi = "baby"
print hi

def other(self):
exec(magic())
print hi

a = A()
a.meth()
a.other()

That's way too fragile to be useful.

 

[Alex Martelli]

I think it would be cool if you could refer to instance variables
without prefixing with "self." I know noone else thinks like me so

Some do -- Kent Beck's excellent book on TDD-by-example has a specific
grouse against that in the chapter where he develops the unittest module
(in Python). But that's how comes Kent is a _Smalltalk_ programmer
rather than a _Python_ programmer, see?-)

Python will never be changed, but maybe you can already do it with
Python today?
Sure.

impressive in this case but much cooler when there is more instance
variables around. But the solution is very ugly because you have to
write exec(magic()) in every method. So I'm asking here if someone
knows a better way, maybe using decorators or metaclasses or other
black magic?

A decorator can entirely rewrite the bytecode (and more) of the method
it's munging, so it can do essentially anything that is doable on the
basis of information available at the time the decorator executes. You
do, however, need to nail down the specs. What your 'magic' does is
roughly the equivalent of a "from ... import *" (except it "imports", so
to speak, from a namespace that's not a module): it makes a local copy
of all names defined in the given namespace, and that's all. The names
stay local, any rebinding of a name has no non-local effect whatsoever,
etc. Is this indeed what you want -- just give a method this kind of
copies? And still have to write self.x=23 for re-binding (without
effect on the local copy, note...)? Or what else?

Then, you must decide whether this applies to all names the method
accesses (which aren't already local). For example, if the method has a
statement such as:
x = len(y)
and does not otherwise as locals len nor y, does this mean
x = self.len(self.y)
or
x = len(self.y)
or
x = self.len(y)
or
x = len(y)
....? I.e., which of the names len and y is meant to be a global or
builtin, which is meant to be an isntance variable? The decorator must
know, because it needs to generate different bytecode. Your approach,
injecting an exec statement in the method, makes the compiler punt: the
compiler knows, seeing 'exec', that it has no idea about which names are
locals or globals any more, so it generates horribly-slow code for
completely-general accesses instead of normal local-access-is-optimized
code. Is that what you want to do -- slow all of your Python code down
by an order of magnitude in order to be able to avoid writing 'self.' in
a few cases?

If you can give totally complete specifications, I can tell you whether
your specs are doable (by a decorator, or other means), how, and at what
cost. Without knowing your specs, I can't tell; I can _guess_ that the
answer is "probably doable" (as long as you're not demanding the code in
the decorator to be an oracle for the future, but are content to limit
it to information known when it runs; and as long as you don't care how
much you slow everything down) and most definitely not WORTH doing for
anything except mental gym.


Alex

 

[John Roth]

Some do -- Kent Beck's excellent book on TDD-by-example has a specific
grouse against that in the chapter where he develops the unittest module
(in Python). But that's how comes Kent is a _Smalltalk_ programmer
rather than a _Python_ programmer, see?-)

And of course, the reason it's possible in Smalltalk but not in Python
is that Smalltalk requires the declaration of instance variables. Also
Smalltalk does not have things like module variables and builtins.
The interpreter knows exactly what every name references, which
isn't true in Python.

John Roth

 

[Jeremy Bowers]

This is promising, I'm content with whatever slowdowns necessary as long
as I can prove those who say "you can't do it" wrong.

Since I think I'm the only person in this discussion that said anything
about what you can't do, be clear on what I said. You can't have both of
undeclared attributes on self and no use of "self", in particular to add
new attributes.

This is, if you take the time to understand what I mean, trivially true;
*somewhere* you need to declare whether a var is local to the function or
an instance member. For me, I prefer the explicit "self" and getting rid
of "self" now leaves you with the need to declare member variables
*somehow*, which I don't consider progress. But no matter what other magic
Alex works, you're only going to get one or the other; it's impossible for
the compiler to divine what you mean otherwise.

My point here isn't that you "can't" hack together code to do something
like what you want, and it is certainly a valid exercise in plumbing the
depths of Python and learning. My point is that you'll have to pay a price
in other ways. You can't make self go away "for free". And that "can't" I
do mean.

(You weren't necessarily claiming you could. But I thought it still worth
saying; even if you weren't trying to remove "self" "for free", others
certainly would mean it.)

 

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

Thank you for your replies. It is very nice to see that a thread you
started is generating so much discussion, but please, I have read the
previous debates so I know all about what people think about self.
Spare the "you shouldn't do that" and "self is here to stay" replies
to the threads in which people are actually suggesting changing the
syntax.

I know the solution I presented is not working ideally, because you
have to use self to assign to instance attributes like "self.hi =
'baby'".

Sean Ross:
http://starship.python.net/crew/mwh/hacks/selfless.py

That's excellent! There is one small problem with the code though:

..class Hi(Selfless):
.. __attrs__ = ["x"]
.. def __init__(x):
.. self.x = x

In this case, I think the Python interpreter should realise that the
parameter x shadows the attribute x. But the selfless code has
problems with that. I want it to work exactly like how the situation
is handled in Java and C++.

Alex Martelli:
Some do -- Kent Beck's excellent book on TDD-by-example has a specific
grouse against that in the chapter where he develops the unittest module

I have to get myself that book.

Alex Martelli:
A decorator can entirely rewrite the bytecode (and more) of the method
it's munging, so it can do essentially anything that is doable on the
basis of information available at the time the decorator executes.

Which I believe means that the instance variables have to be declared
in the class? I am content with declaring them like the selfless
approach does:

__attrs__ = ["hi", "foo"]

It's not optimal because it would lead to some "name duplication" when
a class is __init__:ed.

..__attrs__ = ["hi", "foo"]
..def __init__(_hi, _foo):
.. hi = _hi
.. foo = _foo

I guess you can solve that adequately by using one of the recipes from
the Cookbook that automagically initialises an objects variable
depending on which variables was passed in the parameter list. Another
alternative would be not to declare the variables in an __attr__ list,
and instead let them be "declared" by having them initialised in the
__init__. I.e:

..def __init__(hi, foo):
.. self.hi = hi
.. self.foo = foo

When the metaclass then does it magic, it would go through the code of
the __init__ method, see the assignments to "self.hi" and "self.foo",
decide that "hi" and "foo" are attributes of the object and replace
"hi" and "foo" in all other methods with "self.hi" and "self.foo". The
downside is that it probably could never be foolproof against code
like this:

..def __init__(hi, foo):
.. if hi:
.. self.hi = hi
.. else:
.. self.foo = foo

But AFAIK, that example is a corner case and you shouldn't write such
code anyway.

Alex Martelli:
You do, however, need to nail down the specs. What your 'magic' does
is roughly the equivalent of a "from ... import *" (except it
...
Then, you must decide whether this applies to all names the method
accesses (which aren't already local). For example, if the method
has a statement such as:
x = len(y)

All names should be checked like this:
1. Is the name in the parameter list? If so, do not rebind it.
2. Is the name in the objects attribute list? If so, prepend "self."
3. Do stuff like normal.

Example:

..class Foo(Selfless):
.. def __init__(x):
.. print x
.. self.x = x*2
.. def meth():
.. x = x + 10
.. print x
.. def meth2(x):
.. print x
.. print self.x
.. self.x = x
..
..o = Foo(50)
..print o.x
..o.meth()
..o.meth2(12)
..print o.x

Outputs:
50
100
110
12
110
12

Alex Martelli:
If you can give totally complete specifications, I can tell you
whether your specs are doable (by a decorator, or other means), how,
and at what cost. Without knowing your specs, I can't tell; I can
_guess_ that the answer is "probably doable" (as long as you're not
demanding the code in the decorator to be an oracle for the future,

This is promising, I'm content with whatever slowdowns necessary as
long as I can prove those who say "you can't do it" wrong. It seems
to me that it should be doable by having the metaclass that modifies
the class go through the class and bytecode-rewrite all its methods.
So there has to be a big slowdown when the class is created, but after
that, it should execute at pure Python speed? That doesn't seem to
hard, and pretty robust too since bytecode doesn't change so often.
And THEN I'll rewrite python-mode so that it syntax highlights member
attributes! It will be cool.

 

[Alex Martelli]

That's excellent! There is one small problem with the code though:

It shows the fundamentals of how to rewrite the bytecode, yes.

.class Hi(Selfless):
. __attrs__ = ["x"]
. def __init__(x):
. self.x = x

In this case, I think the Python interpreter should realise that the
parameter x shadows the attribute x. But the selfless code has
problems with that. I want it to work exactly like how the situation
is handled in Java and C++.

I believe you're referring to the test in rewrite_method:

if op.arg < code.co_argcount:
raise ValueError, "parameter also instance member!"

If you think that parameters that are also instance members should
"shadow" instance members, just skip the op.arg cases which are less
than code.co_argcount -- those are the parameters.

Which I believe means that the instance variables have to be declared
in the class? I am content with declaring them like the selfless
approach does:

It means the information about which names are names of instance
attributes must be available somewhere, be that "declared", "inferred",
or whatever. For example, many C++ shops have an ironclad rule that
instance attributes, and ONLY instance attributes, are always and
invariably named m_<something>. If that's the rule you want to enforce,
then you don't necessarily need other declarations or inferences, but
rather can choose to infer the status of a name from looking at the name
itself, if you wish. "Declarations" or other complications yet such as:

alternative would be not to declare the variables in an __attr__ list,
and instead let them be "declared" by having them initialised in the
__init__. I.e:

.def __init__(hi, foo):
. self.hi = hi
. self.foo = foo

When the metaclass then does it magic, it would go through the code of
the __init__ method, see the assignments to "self.hi" and "self.foo",
decide that "hi" and "foo" are attributes of the object and replace
"hi" and "foo" in all other methods with "self.hi" and "self.foo". The

OK, but this approach is not compatible with your stated desire, which I
re-quote...:

I want it to work exactly like how the situation
is handled in Java and C++.

....because for example it does not deal with any attributes which may be
initialized in a *superclass*'s __init__. However, I guess it could be
extended at the cost of some further lack of transparency, to obtain
just as horrid a mess as you require, where it's impossible for any
human reader to guess whether, say,
hi = 33
is setting a local variable, or an instance variable, without chasing
down and studying the sources of an unbounded number of superclasses.

I do not think there is any _good_ solution (which is why many C++ shops
have that rule about spelling this m_hi if it's an instance variable,
keeping the spelling 'hi' for non-instance variables -- an attempt to
get SOME human readability back; a smaller but non-null number of such
shops even achieve the same purpose by mandating the use of 'this->hi'
-- just the Python rule you want to work around, essentially). The
least bad might be to rely on __attrs__, enriching whatever is in the
current class's __attr__ with any __attrs__ that may be found in base
classes PLUS any member variables specifically set in __init__ -- if you
focus on convenience in writing the code, to the detriment of ability to
read and understand it; or else, for more readability, demand that
__attrs__ list everything (including explicitly attributes coming from
subclasses and ones set in any method by explicit "self.whatever = ...")
and diagnose the problem, with at least a warning, if it doesn't.

Yes, there's redundancy in the second choice, but that's what
declarations are all about: if you want to introduce the equivalent of
declarations, don't be surprised if redundancy comes with them.

downside is that it probably could never be foolproof against code
like this:

.def __init__(hi, foo):
. if hi:
. self.hi = hi
. else:
. self.foo = foo

But AFAIK, that example is a corner case and you shouldn't write such
code anyway.

I don't see any problem with this code. A static analysis will show
that both hi and foo are local variables. Either may be not
initialized, of course, but having to deal with variables which are not
initialized IS a common problem of C++: you said you want to do things
like in C++, so you should be happy to have this problem, too.

All names should be checked like this:
1. Is the name in the parameter list? If so, do not rebind it.
2. Is the name in the objects attribute list? If so, prepend "self."
3. Do stuff like normal.

That's basically what the already-mentioned "selfless" does, then, with
the small change to consider name conflicts (parameter vs instance
attribute) to be OK rather than errors, as above mentioned; and possibly
larger changes to determine the attribute names, depending on what
strategy you want to pursue for that.

This is promising, I'm content with whatever slowdowns necessary as
long as I can prove those who say "you can't do it" wrong. It seems

Nobody (that I saw) said you can't do it, as long as you're willing to
pay the price in terms of some mixture of extra stuff to write
(__attrs__ or whatever), more difficulty in human reading (not being
able to tell locally what's a local variable and what isn't), new and
interesting kinds of errors such as "uninitialized member variables",
and time to execute the class statement or its methods, depending on the
exact mix and strategies you choose.

to me that it should be doable by having the metaclass that modifies
the class go through the class and bytecode-rewrite all its methods.
So there has to be a big slowdown when the class is created, but after
that, it should execute at pure Python speed? That doesn't seem to
hard, and pretty robust too since bytecode doesn't change so often.

Starting with "selfless" and the few extra pointers I've given, I agree
it should not be too hard. Not sure what you mean by bytecode not
changing often: the bytecode rewrite will happen every time you execute
the 'class' statement, and only then.

And THEN I'll rewrite python-mode so that it syntax highlights member
attributes! It will be cool.

This part might well be pretty hard, given the possibility of
inheritance from classes that might be anywhere at all -- sys.path can
change dynamically, so to know exactly what classes from other modules
your class is inheriting from (and that is crucial to determine which
names are those of instance attributes) basically requires executing all
the program up to the 'class' statement. If you're keen on this part I
suggest you use one of the approaches that also facilitate human
reading: for exactly the same reasons they'll facilitate the
highlighting. Either use something like m_<blah> as the name for
instance attributes, or have all instance attributes listed in
__attrs__, considering it an error (worth at least a warning... and a
lack of highlighting!) to use other instance attributes (from
superclasses or whatever) that _aren't_ listed in __attrs__.


Alex