Why this doesn't work?

[mk]

Sorry to bother everyone again, but I have this problem bugging me:

#!/usr/bin/python -i

class Foo(object):

def nostat(self,val):
print val

nostat.__orig_get__ = nostat.__get__

@staticmethod
def nostatget(*args, **kwargs):
print 'args:', args, 'kwargs:', kwargs
nostat.__orig_get__(*args, **kwargs)

nostat.__get__ = nostatget
setattr(nostat,'__get__',nostatget)


f = Foo()

f.nostat('a')

print f.nostat.__get__ is f.nostat.__orig_get__


This produces:

a
False

I expected to see 'nostatget' output: nostat.__get__ = nostatget
obviously failed to replace this function's __get__ method.

The question is why? Isn't __get__ a normal attribute of a function nostat?

This is made so much weirder that nostat.__get__ is no longer original
__get__, so it looks like it should have been replaced, but if so, why
nostatget doesn't get called?

Regards,
mk

 

[Steven D'Aprano]

nostat.__orig_get__ = nostat.__get__

I should point out that leading-and-trailing-double-underscore names are
reserved for use by the language.

It's unlikely that Python will introduce a special method named
__orig_get__, and in truth the prohibition against using such names is
honoured more in the breach than in the observance (e.g. people often use
metadata names like __version__, __author__, etc.). But just be aware
that you are in technical breach of Python best practices, and should
consider renaming it as _orig_get or __orig_get.

 

[John Posner]

Sorry to bother everyone again, but I have this problem bugging me:

#!/usr/bin/python -i

class Foo(object):

def nostat(self,val):
print val

nostat.__orig_get__ = nostat.__get__

@staticmethod
def nostatget(*args, **kwargs):
print 'args:', args, 'kwargs:', kwargs
nostat.__orig_get__(*args, **kwargs)

nostat.__get__ = nostatget
setattr(nostat,'__get__',nostatget)


f = Foo()

f.nostat('a')

print f.nostat.__get__ is f.nostat.__orig_get__


This produces:

a
False

I expected to see 'nostatget' output: nostat.__get__ = nostatget
obviously failed to replace this function's __get__ method.

I don't quite understand the above sentence, so I'm assuming that you
wanted the final "is" test to be "True" instead of "False".

It looks like you've replaced ...

(A) the original __get__() method

.... with ...

(B) a new method that *calls* the original __get__() method

So why should you expect (A) and (B) to be the same object?

The question is why? Isn't __get__ a normal attribute of a function nostat?

This is made so much weirder that nostat.__get__ is no longer original
__get__, so it looks like it should have been replaced, but if so, why
nostatget doesn't get called?

Regards,
mk

-John

 

[Bruno Desthuilliers]

mk a écrit snip)

Sorry, no time to get into details now - but I can at least provide a
couple hints.

The first point is that, to override a method on an _instance_, you have
to provide a method object, not a plain function - remember that the
descriptor protocol is only invoked on _class_ attributes, not on
instance attributes.

class Foo(object):
def bar(self):
print "the original bar"

def mybar(self):
print "mybar"

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/tmp/python-1287O_i.py", line 32, in <module>
f.bar()
TypeError: mybar() takes exactly 1 argument (0 given)
mybar


As you see, "bar" is here resolved as an ordinary instance attribute -
so here it evals to the "mybar" function object. If you want it to be a
method object, you do have to invoke the descriptor protocol manually:
mybar

Or alternatively, you can use the types module:mybar


Second point is that the descriptor protocol is invoked on each and
every lookup. So when it comes to function class attributes, you get a
new method object each time:

I think this should help you understand why your code doesn't work as
you assumed it would !-)


PS : as a side note, a common optimization trick is to "short-circuit"
the whole lookup / descriptor mechanism when you have to call the same
method of the same object in a loop:

l = []
append = l.append
for i in xrange(100):
append(i)
print l

HTH

 

[mk]

I should point out that leading-and-trailing-double-underscore names are
reserved for use by the language.

Right... I completely missed that. I will try to change the habit.

I am under impression that a function with no underscore in name is
meant to be called "publicly" on instance, like Foo().nostat, a function
with one underscore (and no trailing underscores) is meant to be like
"it's mostly intended for internal use, but you can still call it",
somewhat like "protected" in C++, and a function with two leading
underscores (and no trailing underscores) is meant as completely
internal to the class, not meant to be called by outsiders, somewhat
like "private" in C++ (I abstract from obvious point in C++ that
semantics of those keywords is enforced by a compiler in C++).

Is that correct?

Regards,
mk

 

[mk]

I don't quite understand the above sentence, so I'm assuming that you
wanted the final "is" test to be "True" instead of "False".

No. I should have described my goal in plain English in first place.

I wanted to replace nostat function's __get__ descriptor with a function
that would allow me to peek inside it (i.e. into __get__ descriptor
arguments) and that would then call original nostat.__get__.

So no, I expected it to be false, that is, I expected to replace
original descriptor with a "custom" descriptor. I just wondered why this
custom "nostatget" descriptor didn't get called even though the new
nostat.__get__ didn't seem to be its old self (so quite likely it was
the new, nostatget, descriptor).

But Bruno pointed out that I need instancemethod for that, not plain
function.

Regards,
mk

 

[Bruno Desthuilliers]

mk a écrit :

Right... I completely missed that. I will try to change the habit.

I am under impression that a function with no underscore in name is
meant to be called "publicly" on instance, like Foo().nostat, a function
with one underscore (and no trailing underscores) is meant to be like
"it's mostly intended for internal use, but you can still call it",
somewhat like "protected" in C++, and a function with two leading
underscores (and no trailing underscores) is meant as completely
internal to the class, not meant to be called by outsiders, somewhat
like "private" in C++ (I abstract from obvious point in C++ that
semantics of those keywords is enforced by a compiler in C++).

Is that correct?

Mostly, yes.

- s/function/whatever/ : these rules apply to all names

- the single leading underscore denote implementation, IOW : you should
not use it, unless you have a pretty good reason to do so AND you take
full responsability for what may happens

- the double leading underscore is in fact mostly to protect your
attribute from accidental overloading. It triggers a (simple)
name-mangling operation that turns "__yourname" into
"_Yourclassname__yourname". FWIW, this is something I must have a use
for at most once a year, and even then most of the times because I'm a
bit of a control freak sometimes.

 

[Bruno Desthuilliers]

mk a écrit :

No. I should have described my goal in plain English in first place.

I wanted to replace nostat function's __get__ descriptor

A "descriptor" (shortcut for "object implementing the descriptor
protocol) is an object that have a __get__ method - not the __get__
method itself. A function is a descriptor. The __get__ method of the
function type is a method, and methods are not descriptors.

with a function
that would allow me to peek inside it (i.e. into __get__ descriptor
arguments)

=> __get__(self, instance, cls=None)

When called as a result of a look up on an instance f of class Foo,
nostat.__get__ will get (nostat, f, Foo) as args.

So no, I expected it to be false, that is, I expected to replace
original descriptor with a "custom" descriptor.

s/descriptor/__get__/

Anyway: these types (function, method etc) are implemented in (higly
optimized) C, and with a lot of restriction for both performance and
sanity reasons. It's a case of practicality beats purity.

Another - perhaps more rewarding - exercise might be to implement a
custom callable type that implements the descriptor protocol the same
way the function do.