newbie: self.member syntax seems /really/ annoying

[Carl Banks]

Making, say, 'a' hiddenly mean 'x.a', within a function, requires a
decorator that suitably rewrites the function's bytecode... (after
which, it WOULD still be terrible and horrible and not to be used, just
as you say, but it might at least _work_;-). Main problem is, the
decorator needs to know the set of names to be "faked out" in this
terrible and horrible way at the time the 'def' statement executes: it
can't wait until runtime (to dynamically determine what's in var(self))
before it rewrites the bytecode (well, I guess you _could_ arrange a
complicated system to do that, but it _would_ be ridiculously slow).

How about this? The decorator could generate a bytecode wrapper that
would have the following behavior, where __setlocal__ and
__execute_function__ are special forms that are not possible in
Python. (The loops would necessarily be unwrapped in the actual
bytecode.)


for sym in sys._getframe(0).f_code.co_varnames:
if sym == "self":
continue
try:
__setlocal__(sym,getattr(self,sym))
except AttributeError:
pass
try:
__execute_function__()
finally:
for sym,val in locals.iteritems():
if sym == "self":
continue
if sym in self:
setattr(self,sym,val)


This wouldn't be that much slower than just assigning local variables
to locals by hand, and it would allow assignments in the
straightforward way as well.

There'd be some gotchas, so extra care is required, but it seems like
for the OP's particular use case of a complex math calculation script,
it would be a decent solution.

I understand where the OP is coming from. I've done flight
simulations in Java where there are lot of complex calculations using
symbols. This is a typical formula (drag force calculation) that I
would NOT want to have to use self.xxx for:

FX_wind = -0.5 * rho * Vsq * Sref * (C_D_0 + C_D_alphasq*alpha*alpha +
C_D_esq*e*e)

Even if I'd habitually used "this." in Java, I'd probably skip it in
functions like htis.


Carl Banks

 

[Carl Banks]

(The loops would necessarily be unwrapped in the actual
bytecode.)

And by unwrapped, I mean unrolled.

:E3


Carl Banks

 

[madzientist]

As a newbie to Python (and OOP), I would love to hear what people
think of Steven's suggestion below. Is there a reason why classes
would be useful for the OP's question ? If you can point me to a brief
online tutorial addressing this, I would happily go there to read it
too

Thanks, Suresh

 

[Ryan Ginstrom]

On Behalf Of madzientist

As a newbie to Python (and OOP), I would love to hear what
people think of Steven's suggestion below. Is there a reason
why classes would be useful for the OP's question ? If you
can point me to a brief online tutorial addressing this, I
would happily go there to read it too

In general, you should use classes when you need to maintain state. The
classic example is a BankAccount class, each instance of which maintains a
"balance" state.

When you don't need to maintain state, module-level functions are fine. In
fact, for testability/reliability, they're preferred, because when a method
twiddles some internal state, it's much harder to test. It's valuable to
have a function that always gives output x for input y, with no side
effects. That (to me) is the appeal of the functional programming style.

Regards,
Ryan Ginstrom

 

[Alex Martelli]

How about this? The decorator could generate a bytecode wrapper that
would have the following behavior, where __setlocal__ and
__execute_function__ are special forms that are not possible in
Python. (The loops would necessarily be unwrapped in the actual
bytecode.)

I'm not entirely sure how you think those "special forms" would work.

Right now, say, if the compiler sees somewhere in your function
z = 23
print z
it thereby knows that z is a local name, so it adds a slot to the
function's locals-array, suppose it's the 11th slot, and generates
bytecode for "LOAD_FAST 11" and "STORE_FAST 11" to access and bind that
'z'. (The string 'z' is stored in f.func_code.co_varnames but is not
used for the access or storing, just for debug/reporting purposes; the
access and storing are very fast because they need no lookup).

If instead it sees a "print z" with no assignment to name z anywhere in
the function's body, it generates instead bytecode "LOAD_GLOBAL `z`"
(where the string `z` is actually stored in f.func_code.co_names). The
string (variable name) gets looked up in dict f.func_globals each and
every time that variable is accessed or bound/rebound.

If the compiler turns this key optimization off (because it sees an exec
statement anywhere in the function, currently), then the bytecode it
generates (for variables it can't be sure are local, but can't be sure
otherwise either as they MIGHT be assigned in that exec...) is different
again -- it's LOAD_NAME (which is like LOAD_GLOBAL in that it does need
to look up the variable name string, but often even slower because it
needs to look it up in the locals and then also in the globals if not
currently found among the locals -- so it may often have to pay for two
lookups, not just one).

So it would appear that to make __setlocal__ work, among other minor
revolutions to Python's code objects (many things that are currently
tuples, built once and for all by the compiler at def time, would have
to become lists so that __setlocal__ can change them on the fly), all
the LOAD_GLOBAL occurrences would have to become LOAD_NAME instead (so,
all references to globals would slow down, just as they're slowed down
today when the compiler sees an exec statement in the function body).
Incidentally, Python 3.0 is moving the OTHER way, giving up the chore of
dropping optimization to support 'exec' -- the latter will become a
function instead of a statement and the compiler will NOT get out of its
way to make it work "right" any more; if LOAD_NAME remains among Python
bytecodes (e.g. it may remain in use for class-statement bodies) it
won't be easy to ask the compiler to emit it instead of LOAD_GLOBAL (the
trick of just adding "exec 'pass'" will not work any more;-).

So, "rewriting" the bytecode on the fly (to use LOAD_NAME instead of
LOAD_GLOBAL, despite the performance hit) seems to be necessary; if
you're willing to take those two performance hits (at decoration time,
and again each time the function is called) I think you could develop
the necessary bytecode hacks even today.

This wouldn't be that much slower than just assigning local variables
to locals by hand, and it would allow assignments in the
straightforward way as well.

The big performance hit comes from the compiler having no clue about
what you're doing (exactly the crucial hint that "assigning local
variables by hand" DOES give the compiler;-)

There'd be some gotchas, so extra care is required, but it seems like
for the OP's particular use case of a complex math calculation script,
it would be a decent solution.

Making such complex calculations even slower doesn't look great to me.

I understand where the OP is coming from. I've done flight
simulations in Java where there are lot of complex calculations using
symbols. This is a typical formula (drag force calculation) that I
would NOT want to have to use self.xxx for:

FX_wind = -0.5 * rho * Vsq * Sref * (C_D_0 + C_D_alphasq*alpha*alpha +
C_D_esq*e*e)

If ALL the names in every formula always refer to nothing but instance
variables (no references to globals or builtins such as sin, pi, len,
abs, and so on, by barenames) then there might be better tricks, ones
that rely on that knowledge to actually make things *faster*, not
slower. But they'd admittedly require a lot more work (basically a
separate specialized compiler to generate bytecode for these cases).


Alex

 

[Carl Banks]

...


I'm not entirely sure how you think those "special forms" would work.

Right now, say, if the compiler sees somewhere in your function
z = 23
print z
it thereby knows that z is a local name, so it adds a slot to the
function's locals-array, suppose it's the 11th slot, and generates
bytecode for "LOAD_FAST 11" and "STORE_FAST 11" to access and bind that
'z'. (The string 'z' is stored in f.func_code.co_varnames but is not
used for the access or storing, just for debug/reporting purposes; the
access and storing are very fast because they need no lookup).

If instead it sees a "print z" with no assignment to name z anywhere in
the function's body, it generates instead bytecode "LOAD_GLOBAL `z`"
(where the string `z` is actually stored in f.func_code.co_names). The
string (variable name) gets looked up in dict f.func_globals each and
every time that variable is accessed or bound/rebound.

Withdrawn. For some reason it was in my mind that variables not bound
within in the function would be local.

:|



[snip]

The big performance hit comes from the compiler having no clue about
what you're doing (exactly the crucial hint that "assigning local
variables by hand" DOES give the compiler;-)

Right.



Carl Banks

 

[Steve Holden]

For what it's worth, if you stick with python for a while, you will stop noticing the self, and be able to see what you're looking for quite clearly.

Moreso if you use s.

Not sure that's true, since it still appears to bug Bruce Eckel ...

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Michele Simionato]

Making, say, 'a' hiddenly mean 'x.a', within a function, requires a
decorator that suitably rewrites the function's bytecode... (after
which, it WOULD still be terrible and horrible and not to be used, just
as you say, but it might at least _work_;-). Main problem is, the
decorator needs to know the set of names to be "faked out" in this
terrible and horrible way at the time the 'def' statement executes: it
can't wait until runtime (to dynamically determine what's in var(self))
before it rewrites the bytecode

All right Alex, and since I had 20 minutes of spare
time I implemented the decorator you are talking about by
using the wonderful byteplay module Here it is:


import dis
from byteplay import Code, LOAD_GLOBAL, LOAD_ATTR, LOAD_FAST,
STORE_FAST

def instance2local(varname):
yield LOAD_FAST, 'self'
yield LOAD_ATTR, varname
yield STORE_FAST, varname

def declare_instance_vars(*varnames):
def dec(f):
c = Code.from_code(f.func_code)
# change LOAD_GLOBAL -> LOAD_FAST
for i, (opcode, value) in enumerate(c.code):
if opcode == LOAD_GLOBAL and value in varnames:
c.code = (LOAD_FAST, value)
# insert instance2local assigments at the beginning
assignments = []
for varname in varnames:
for pair in instance2local(varname):
assignments.append(pair)
c.code[0:0] = assignments
# redefine the code object
f.func_code = c.to_code()
return f
return dec

class Test(object):
def __init__(self):
self.a = 1
self.b = 2

def test1(self):
a = self.a
b = self.b
return a * b

@declare_instance_vars('a', 'b')
def test2(self):
return a * b


co1 = Test.__dict__['test1'].func_code
co2 = Test.__dict__['test2'].func_code

print 'bytecode for test1'
dis.dis(co1)
print 'bytecode for test2'
dis.dis(co2)

t = Test()
assert t.test1() == t.test2()

It is still a hack, since one is not supposed to mess around
with bytecodes, but at least it seems to work [warning:
I have not tested it more than you see]

Michele Simionato

 

[genro]

I've just started playing around with Python, as a possible
replacement for a mix of C++, Matlab and Lisp. The language looks
lovely and clean with one huge exception: I do a lot of numerical
modeling, so I deal with objects (like neurons) described
mathematically in papers, by equations like
a_dot = -k(a-u)
In other languages, this translates nicely into code, but as far as I
can tell, Python needs the ugly:
self.a_dot = -self.k(self.a-self.u)
For large equations this is going to make my code seriously unreadable
to the point of needing to switch back to Matlab -- and it seems to go
against everything else about python's simplicity and elegance. Am I
missing something? Is there something like a 'with' command that lets
me set the scope, like

with self:
.a_dot = -.k(.a-.u)

It's premature to make language suggestions as I am new to the
language, but I would have though that making a 'with self' explicit
in all methods would have been neat, so I could just write
.a_dot = -.k(.a-.u)
which would still avoid confusion with local function variables, since
'.a' is different from 'a'.

Please help if I am missing something -- this looks like a great
language but I am going to mad trying to read numerical code full of
'self.'s breaking up the equations.

Just an idea: I think that if you have to get a more readable code
you
could abstract the function from the object using a significative
name and then use this named function passing **self.__dict__

For example in your module you could write:

#------ functional section -----------


def dothis(k,a,u):
return k*(a-u)

def dothat(m,r,k):
return m*r+(k.m)^m


#----- Class definition ------

class Bar(object):

def foo(self):
self.a_dot = dothis(**self.__dict__)
self.boo = dothat(**self.__dict__)

In this way the functional part is pure expression and very readable.


HTH

G.

 

[timothy.soehnlin]

Why not use '_' as the self variable. It is minimal and achieves
close to '.var', as '_.var' isn't that different. I know its a little
perl-esque, but its not a bad convention if you are aiming to up
readability of your code.

class test:
def __init__(self):
_ = self;
_.a = 0;
_.u = 0;
_.a_dot = 0;
def k(self, val):
return val**2; #not sure what k does
def getA_Dot(self):
_ = self;
_.a_dot = _.k(_.a-_.u);

 

[Steven D'Aprano]

Why not use '_' as the self variable. It is minimal and achieves close
to '.var', as '_.var' isn't that different. I know its a little
perl-esque, but its not a bad convention if you are aiming to up
readability of your code.

I think the definitions of "up" or "readability" you are using are very
different from mine. To me, to up something means to increase it, and
readability means the ease of comprehension when reading something. You
seem to be using the opposite definition for one or the other.

 

[Wildemar Wildenburger]

I think the definitions of "up" or "readability" you are using are very
different from mine. To me, to up something means to increase it, and
readability means the ease of comprehension when reading something. You
seem to be using the opposite definition for one or the other.

OK, making a pointless reply to pointless reply, but anyway:

I see merit in using

(_.foo + _.bar) * _.baz

instead of

(s.foo + s.bar) * s.baz

because I'm trained to interpret the underscore as a synonym for one
space. It's not particularly beautiful, but that is probably a matter of
habituation. And that exact word is probably the reason why I'd still
use self or s (explained by a comment, because I can get very dumb if I
have to).

It's a matter of taste, so there is no point in bashing a valid suggestion.

/W (I am aware that I'm banging a philosophy on people's heads just as
Steven did, so I'm no better, I know.)

 

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

The name "self" is just a convention. You can give it any name you
wish. Using "s" is common.

Not it's not common. And the name "self" is a convention codified in
PEP8 which you shouldn't violate.

And I agree with the OP that the convention is really annoying.

self.rect.width = self.foo(self.rect.x + self.rect.y) * self.boo()

is much less concise than

s.rect.width = s.foo(s.rect.x + s.rect.y) * s.boo()

 

[Eduardo O. Padoan]

because I'm trained to interpret the underscore as a synonym for one
space. It's not particularly beautiful, but that is probably a matter of
habituation. And that exact word is probably the reason why I'd still
use self or s (explained by a comment, because I can get very dumb if I
have to).

Have you worked with code using gettext functions imported as "_"?
When I see a single underscore as a name, i18n comes to mind.

 

[attn.steven.kuo]

On Sep 12, 10:05 am, (e-mail address removed) (Alex Martelli) wrote:

....

into a function and a call to it:

def f():
with implicit_self(t):
print a
print b
a = 40
b = a * 2
f()

...even with different values for the argument to _getframe. You just
can't "dynamically" add local variables to a function, beyond the set
the compiler has determined are local (and those are exactly the ones
that are *assigned to* in the function's body -- no less, no more --
where "assigned to" includes name-binding statements such as 'def' and
'class' in addition to plain assignment statements, of course).

Making, say, 'a' hiddenly mean 'x.a', within a function, requires a
decorator that suitably rewrites the function's bytecode... (after
which, it WOULD still be terrible and horrible and not to be used, just
as you say, but it might at least _work_;-). Main problem is, the
decorator needs to know the set of names to be "faked out" in this
terrible and horrible way at the time the 'def' statement executes: it
can't wait until runtime (to dynamically determine what's in var(self))
before it rewrites the bytecode (well, I guess you _could_ arrange a
complicated system to do that, but it _would_ be ridiculously slow).

You could try defeating the fundamental optimization that stands in your
way by adding, say,
exec 'pass'
inside the function-needing-fakeouts -- but we're getting deeper and
deeper into the mire...;-)

What about abusing 'import' like this:

def f():
with implicit_self(t):
sys.modules['__the_mire__'] = t
from __the_mire__ import a, b
print a
print b
a = 40
b = a * 2


If the list of attributes is long then
'import' can save some typing compared to

a = self.a
b = self.b
....
z = self.z

 

[Steven D'Aprano]

And I agree with the OP that the convention is really annoying.

self.rect.width = self.foo(self.rect.x + self.rect.y) * self.boo()

is much less concise than

s.rect.width = s.foo(s.rect.x + s.rect.y) * s.boo()

Y do u thnk bng cncis is lwys a gd thng?

Being concise is not always a good thing. I notice that you separate
terms with spaces, even when Python itself doesn't need them. You also
are happy to use infix notation, which requires brackets, instead of a
postfix or RPN language that doesn't. You also have attributes named
"rect" and "width" instead of "r" and "w":

s.r.w=s.f(s.r.x+s.r.y)*s.b()

There. See how much more readable that isn't?

 

[Charles Fox]

hmm, I guess this is the difference between numerical programming and
the rest -- sure, if I was writing a database server or something it
would be great to have thisObject.veryLongName to know what everything
is -- however when you are implementing a model from a published
paper, the variables tend to be single greek or roman letter names,
possibly with subscripts and superscripts, and it helps if the name
you see on the screen is the same as the name on the paper, as is the
case in matlab. You want the equation on screen to look as similar to
the one on the paper as possible, especially if its going to be read
by another programmer who is familiar with the paper.

Maybe for now I will just fix up my emacs to display the world 'self'
in 10% gray...

 

[Steven D'Aprano]

I see merit in using

(_.foo + _.bar) * _.baz

instead of

(s.foo + s.bar) * s.baz

because I'm trained to interpret the underscore as a synonym for one
space. It's not particularly beautiful, but that is probably a matter of
habituation. And that exact word is probably the reason why I'd still
use self or s (explained by a comment, because I can get very dumb if I
have to).

It's a matter of taste, so there is no point in bashing a valid
suggestion.

It's not just a matter of taste.

Reading comprehensibility is an objective, measurable quantity, and I
would bet that "(self.foo + self.bar) * self.baz" would be measurably
more readable on average than either of your two alternatives _even for
those people who claim to hate it_.

But what do I care? If people are writing code that only they will see,
they can use any conventions they feel like. It's no skin off my nose.

But if you're writing code that is going to be seen by others in the
wider community, perhaps because it's open source, or because you're
working in a team of coders, or even because you're posting snippets to
comp.lang.python asking for assistance, then it is wise to write using
conventions that others use. The harder you make it for people to read
your code, the fewer people will be willing or able or bothered to read
it.

 

[Wildemar Wildenburger]

Maybe for now I will just fix up my emacs to display the world 'self'
in 10% gray...

Now *that* is a clever idea! (And that's no irony.)

/W

 

[Wildemar Wildenburger]

Have you worked with code using gettext functions imported as "_"?
When I see a single underscore as a name, i18n comes to mind.

See? Habituation. Just what I'm saying.

/W