Function mistaken for a method

[Eric Brunel]

Hi all,

I just stepped on a thing that I can't explain. Here is some code showing
the problem:

-----------------------------
class C:
f = None
def __init__(self):
if self.f is not None:
self.x = self.f(0)
else:
self.x = 0

class C1(C):
f = int

class C2(C):
f = lambda x: x != 0

o1 = C1()
print o1.x

o2 = C2()
print o2.x
-----------------------------

Basically, I want an optional variant function across sub-classes of the
same class. I did it like in C1 for a start, then I needed something like
C2. The result is... surprising:

0
Traceback (most recent call last):
File "func-vs-meth.py", line 18, in ?
o2 = C2()
File "func-vs-meth.py", line 5, in __init__
self.x = self.f(0)
TypeError: <lambda>() takes exactly 1 argument (2 given)

So the first works and o1.x is actually 0. But the second fails because
self is also being passed as the first argument to the lambda. Defining a
"real" function doesn't help: the error is the same.

My actual question is: why does it work in one case and not in the other?
As I see it, int is just a function with one parameter, and the lambda is
just another one. So why does the first work, and not the second? What
'black magic' takes place so that int is not mistaken for a method in the
first case?

 

[Maric Michaud]

Le Jeudi 01 Juin 2006 13:12, Eric Brunel a écrit :

class C1(C):
f = int

int is not a function but a type, but it's callable so int(0) return 0.

class C2(C):
f = lambda x: x != 0

lambda is a function, applied as a class attribute it becomes a method so it's
called with a first parameter representing the instance, self.f(0) in the
__init__ becomes C2.f(self, 0), so the lambda should be :

f = lambda s, x: x != 0 # s for self, some poeple use _

this exactly the same as :

def f(self, val) :
return x != 0

(that lambda will return True or False i expect this is not what you want)

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Maric Michaud]

Le Jeudi 01 Juin 2006 13:29, Maric Michaud a écrit :

this exactly the same as :

   def f(self, val) :
       return x != 0

oops,
def f(self, val) :
return val != 0


--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Peter Otten]

My actual question is: why does it work in one case and not in the other?
As I see it, int is just a function with one parameter, and the lambda is
just another one. So why does the first work, and not the second? What
'black magic' takes place so that int is not mistaken for a method in the
first case?

A python-coded function has a __get__ attribute, a C-function doesn't.
Therefore C1.f performs just the normal attribute lookup while C2.f also
triggers the f.__get__(C2(), C2) call via the descriptor protocol which
happens to return a bound method.

Peter

 

[Maric Michaud]

Le Jeudi 01 Juin 2006 13:34, Peter Otten a écrit :

A python-coded function has a __get__ attribute, a C-function doesn't.
Therefore C1.f performs just the normal attribute lookup while C2.f also
triggers the f.__get__(C2(), C2) call via the descriptor protocol which
happens to return a bound method.

I don't think it's about c-coded versus python-coded stuff, C1.f is a type,
C2.f is a method.

In [14]: class t : pass
....:

In [15]: class u :
....: f = t
....:
....:

In [16]: u().f()
Out[16]: <__main__.t instance at 0xa795a9ec>


--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Peter Otten]

Le Jeudi 01 Juin 2006 13:34, Peter Otten a écrit :

I don't think it's about c-coded versus python-coded stuff, C1.f is a
type, C2.f is a method.

You are right, int is a type not a function, but presence (and
implementation, of course) of __get__ is still the distinguishing factor:
.... class __metaclass__(type):
.... def __get__(*args): print "XXX", args
........ int = Int
....XXX (<class '__main__.Int'>, <__main__.C instance at 0x402948cc>, <class
__main__.C at 0x40281f2c>)

Also:
.... sin = sin
.... son = son
....Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: son() takes exactly 1 argument (2 given)

Peter

 

[John Machin]

Le Jeudi 01 Juin 2006 13:34, Peter Otten a écrit :
I don't think it's about c-coded versus python-coded stuff, C1.f is a type,
C2.f is a method.

Try putting f = chr (a C function); it behaves like int, not like a
1-arg Python function. See below.

Cheers,
John

C:\junk>type func_meth.py
class C:
f = None
def __init__(self):
if self.f is not None:
self.x = self.f(0)
else:
self.x = 99 # differs from int(0)
class C1(C):
f = int
class C2(C):
def f(self, arg):
return arg != 0
class C3(C):
pass
class C4(C):
f = chr
for cls in (C1, C2, C3, C4):
o = cls()
print "callable: %r; result: %r" % (o.f, o.x)

C:\junk>func_meth.py
callable: <type 'int'>; result: 0
callable: <bound method C2.f of <__main__.C2 instance at 0x00AE6F58>>;
result: False
callable: None; result: 99
callable: <built-in function chr>; result: '\x00'

C:\junk>

 

[bruno at modulix]

Eric Brunel wrote:




A python-coded function has a __get__ attribute, a C-function doesn't.
Therefore C1.f performs just the normal attribute lookup while C2.f also
triggers the f.__get__(C2(), C2) call via the descriptor protocol which
happens to return a bound method.

FWIW:

class Obj(object):
def __new__(cls, val, *args, **kw):
print "in Obj.__new__"
print "- called with :"
print " cls :", cls
print " val :", val
print " args:", str(args)
print " kw :", kw
obj = object.__new__(cls, *args, **kw)
print "got : %s - %s" % (obj, dir(obj))
return obj

class CPlus(C):
f = Obj

 

[Eric Brunel]

A python-coded function has a __get__ attribute, a C-function doesn't.
Therefore C1.f performs just the normal attribute lookup while C2.f also
triggers the f.__get__(C2(), C2) call via the descriptor protocol which
happens to return a bound method.

Thanks for your explanations, Peter. I'll have to find another way to do
what I want...

 

[John Machin]

Hello:


I have a Tkinter GUI Dialog with many buttons and labels and text
widgets.

So start a *new* thread.

 

[Maric Michaud]

Le Jeudi 01 Juin 2006 13:12, Eric Brunel a écrit :

Thanks for your explanations, Peter. I'll have to find another way to do  
what I want...

maybe :

class C:
   f = None
   def __init__(self):
     if self.f is not None:
       self.x = self.f(0)
     else:
       self.x = 0

class C2(C):
   def __init__(self) :
self.f = lambda x: x != 0

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Peter Otten]

Thanks for your explanations, Peter. I'll have to find another way to do
what I want...

Maybe just

class C2(C):
f = staticmethod(lambda x: x != 0)

Peter

 

[bruno at modulix]

Hi all,

I just stepped on a thing that I can't explain. Here is some code
showing the problem:

Do yourself a favour : use new-style classes.
class C(object)

f = None
def __init__(self):
if self.f is not None:
self.x = self.f(0)
else:
self.x = 0

class C1(C):
f = int

class C2(C):
f = lambda x: x != 0

o1 = C1()
print o1.x

o2 = C2()
print o2.x
-----------------------------

Basically, I want an optional variant function across sub-classes of
the same class.

I did it like in C1 for a start, then I needed
something like C2. The result is... surprising:

0
Traceback (most recent call last):
File "func-vs-meth.py", line 18, in ?
o2 = C2()
File "func-vs-meth.py", line 5, in __init__
self.x = self.f(0)
TypeError: <lambda>() takes exactly 1 argument (2 given)

Not surprising at all.

Functions implement the descriptor protocol[1]. When bound to a class
and looked up via an instance, it's the __get__ method of the function
object that get called - with the instance as param, as defined by the
descriptor protocol. This method then return the function wrapped - with
the instance - in an Method object - which itself, when called, returns
the result of calling the function *with the instance as first
parameter*. Which is how methods can work on the instance, and why one
has to explicitly declare the instance parameter in "functions to be
used as methods", but not explicitly pass it at call time.

(please some guru correct me if I missed something here, but AFAIK it
must be a correct enough description of method invocation mechanism in
Python).

[1] about descriptors, see:
http://docs.python.org/ref/descriptors.html
http://www.geocities.com/foetsch/python/new_style_classes.htm#descriptors

So the first works and o1.x is actually 0.

int is not a function.<type 'type'>

int is a type. A Python type is a callable object, and act as a factory
for instances of it. If the type doesn't implement the descriptor
protocol, when bound to a class and looked up via an instance, normal
lookup rules apply. So the type object is returned as is.


In your case, since int does'nt implement the descriptor protocol, once
looked up (and returned as is), it's called with a correct argument - so
everything runs fine.

Try this:

class Obj(object):
def __new__(cls, val, *args, **kw):
print "in Obj.__new__"
print "- called with :"
print " cls :", cls
print " val :", val
print " args: %s" % str(args)
print " kw : %s" % kw
obj = object.__new__(cls, *args, **kw)
print "got : %s - %s" % (obj, dir(obj))
return obj

def __init__(self, *args, **kw):
print "in Obj.__init__"
print "- called with :"
print " args: %s" % str(args)
print " kw : %s" % kw


class C4(C):
f = Obj

But the second fails because
self is also being passed as the first argument to the lambda.

Of course. It's a function, and it's bound to a class, and looked up via
an instance of the class.

Try this:

def truc(*args, **kw):
print "in truc()__"
print "- called with :"
print " args: %s" % str(args)
print " kw : %s" % kw
if len(args) > 1:
return args[1]

class C6(C):
f = truc

Defining
a "real" function doesn't help: the error is the same.

What' a "real" function ?-) lambdas *are* real functions.

My actual question is: why does it work in one case and not in the
other?

cf above.

As I see it, int is just a function with one parameter,

Nope, it's a type. Functions are just one kind of callable. Types are
callables too, as are any object overloading the call operator - which
is '()' - by implementing the __call__(self, ...) method.

class NotAFunc(object):
def __call__(self):
print "I'm not a function"
return 42

func = NotAFunc()
func()

and the
lambda is just another one.

True. And functions implement the descriptor protocol.

So why does the first work, and not the
second? What 'black magic' takes place so that int is not mistaken for
a method in the first case?

cf above.

If you understood all my explanations, you now know how to solve the
problem.

Else, here the solution:

class C3(C):
f = lambda self, x: return x

 

[Christophe]

Eric Brunel a écrit :

Thanks for your explanations, Peter. I'll have to find another way to
do what I want...

You have 2 ways to do it already, here's a third :

class C:
f = None
def __init__(self):
if self.__class__.f is not None:
self.x = self.__class__.f(0)
else:
self.x = 0

 

[Maric Michaud]

Le Jeudi 01 Juin 2006 15:36, Christophe a écrit :

       self.x = self.__class__.f(0)

nope, this will result in a TypeError "unbound method must be called with
instance as first argument"
--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Eric Brunel]

Do yourself a favour : use new-style classes.
class C(object)

I would if I could: I'm stuck with Python 2.1 for the moment (I should
have mentionned it; sorry for that).

[snip]

Basically, I want an optional variant function across sub-classes of
the same class.

I did it like in C1 for a start, then I needed
something like C2. The result is... surprising:

0
Traceback (most recent call last):
File "func-vs-meth.py", line 18, in ?
o2 = C2()
File "func-vs-meth.py", line 5, in __init__
self.x = self.f(0)
TypeError: <lambda>() takes exactly 1 argument (2 given)

Not surprising at all.

Functions implement the descriptor protocol[1]. When bound to a class
and looked up via an instance, it's the __get__ method of the function
object that get called - with the instance as param, as defined by the
descriptor protocol. This method then return the function wrapped - with
the instance - in an Method object - which itself, when called, returns
the result of calling the function *with the instance as first
parameter*. Which is how methods can work on the instance, and why one
has to explicitly declare the instance parameter in "functions to be
used as methods", but not explicitly pass it at call time.

(please some guru correct me if I missed something here, but AFAIK it
must be a correct enough description of method invocation mechanism in
Python).

[1] about descriptors, see:
http://docs.python.org/ref/descriptors.html
http://www.geocities.com/foetsch/python/new_style_classes.htm#descriptors

So the first works and o1.x is actually 0.

int is not a function.<type 'type'>

int is a type.

Python 2.1 again:<type 'builtin_function_or_method'>

But as someone mentionned, the problem is the same with other built-in
functions, such as chr, even in the latest Python version.

I still find that a little counter-intuitive to have different behaviours
for a built-in function and a Python function. I really would expect them
to work (or not to work) the same in all situations, even if I now
understand better how all works behind the scenes. But I'll live with it...

[snip]

If you understood all my explanations, you now know how to solve the
problem.

Else, here the solution:

class C3(C):
f = lambda self, x: return x

This is actually the first thing I did, but it seemed a bit weird to me to
have a - let's say - callable with one parameter in one case and another
with two parameters in the other one. So I finally turned my callable
attribute into a real instance method, just for consistency.

Anyway, thanks a lot for your explanations.

 

[Bruno Desthuilliers]

Eric Brunel a écrit :

I would if I could: I'm stuck with Python 2.1 for the moment (I should
have mentionned it; sorry for that).

Err, yes - it actually makes most of my explanations inaccurate.

[snip]

(snip too)

 

[Christophe]

Maric Michaud a écrit :

Le Jeudi 01 Juin 2006 15:36, Christophe a écrit :


nope, this will result in a TypeError "unbound method must be called with
instance as first argument"

Your right

staticmethod it is then.