keeping a ref to a non-member function in a class

[Gregory Bond]

I'm building a class hierarchy that needs to keep as a class variable a
reference to a (non-member) function, so that different subclasses can
use different generator functions. But it seems Python is treating the
function as a member function because the reference to it is in class
scope....

Here's a simple case of what I'm trying (in the real code, fn is a
function that returns a database connection relevant to that subclass):

def foo():
print "foo called"

class S(object):
fn = foo

def bar(cls):
cls.fn()
bar = classmethod(bar)

def foo2():
print "foo2 called"

class D(S):
fn = foo2

D.bar()

[I'm on python 2.3.4 so no @classmethod decorator! ]

When I run this:

Traceback (most recent call last):
File "t_o.py", line 19, in ?
D.bar()
File "t_o.py", line 10, in bar
cls.fn()
TypeError: unbound method foo2() must be called with D instance as first argument (got nothing instead)

This shows it is trying to do what I expect (call foo2()) but somehow
the type is all wrong.

I've tried playing with staticmethod() but I can't quite get it all
worked out...

 

[Peter Otten]

I'm building a class hierarchy that needs to keep as a class variable a
reference to a (non-member) function, so that different subclasses can
use different generator functions. But it seems Python is treating the
function as a member function because the reference to it is in class
scope....

Here's a simple case of what I'm trying (in the real code, fn is a
function that returns a database connection relevant to that subclass):

I've tried playing with staticmethod() but I can't quite get it all
worked out...

You are on the right track with staticmethod, but you have to apply it to
fn:
.... fn = staticmethod(foo)
.... def bar(cls):
.... cls.fn()
.... bar = classmethod(bar)
........ fn = staticmethod(foo2)
....second foo

In its current form, the bar() method is not necessary:
second foo

Peter

 

[Gregory Bond]

You are on the right track with staticmethod, but you have to apply it to
fn:

... fn = staticmethod(foo)

Thanks Peter, that's a big help.

I can solve my problem now, but I'm chasing this further in the name of
education, because it seems there is some deep magic happening here that
I don't understand.

It seems that applying staticfunction() (or perhaps assigning to the
class object) is treated differently if it happens in the class
defininition, to when it happens at "run time". (My previous attempts
to get staticmember() to do the right thing were all "run time" like the
examples below.)

Say I wanted to keep the need for "staticmember" hidden from subclasses,
and do the staticmember() conversion in the base class methods. I've
tried 2 ways of doing this (no subclassing here, just to keep it simple):

class B(object):
fn = foo
def try1(self):
print "B.fn is", type(B.fn)
B.fn = staticmethod(B.fn)
print "B try1"
print "B.fn is now", type(B.fn)
B.fn()

def try2(self):
fn2 = staticmethod(B.fn)
print "B try2"
print "fn2 is now", type(fn2)
fn2()

If I try method 1 (assigning to the class object - ignore for a moment
the problem of only doing this once!) I get a set of very surprising
results:

B.fn is <type 'instancemethod'>
B try1
B.fn is now <type 'instancemethod'>
Traceback (most recent call last):
File "t_o1.py", line 28, in ?
B().try1()
File "t_o1.py", line 17, in try1
B.fn()
TypeError: unbound method foo() must be called with B instance as first argument (got nothing instead)

note that assigning the staticmember() result to B.fn does NOT change
the type of B.fn!! And foo is treated as a member function.

So if I try method 2 (using staticmethod() at runtime):

B try2
fn2 is now <type 'staticmethod'>
Traceback (most recent call last):
File "t_o1.py", line 27, in ?
B().try2()
File "t_o1.py", line 22, in try2
fn2()
TypeError: 'staticmethod' object is not callable

fn2 is a static method, as I'd expect, but it is somehow not callable?

Can someone explain what is going on here? Pointers to language spec,
code, PEPs etc gladly accepted.

Greg,
caught in a twisty little maze of descriptors, all different!

 

[Peter Otten]

Thanks Peter, that's a big help.

You're welcome.

I can solve my problem now, but I'm chasing this further in the name of
education, because it seems there is some deep magic happening here that
I don't understand.

Python resorts to deep magic only when it's inevitable, i. e. hardly
ever

It seems that applying staticfunction() (or perhaps assigning to the
class object) is treated differently if it happens in the class
defininition, to when it happens at "run time". (My previous attempts
to get staticmember() to do the right thing were all "run time" like the
examples below.)

There is no distinction between "run time" and "class definition time". The
class "body" is actually a function that is called when the module is
loaded. Its locals() are then used to create a type instance, i. e. the
class.

Say I wanted to keep the need for "staticmember" hidden from subclasses,
and do the staticmember() conversion in the base class methods. I've
tried 2 ways of doing this (no subclassing here, just to keep it simple):


If I try method 1 (assigning to the class object - ignore for a moment
the problem of only doing this once!) I get a set of very surprising
results:


note that assigning the staticmember() result to B.fn does NOT change
the type of B.fn!! And foo is treated as a member function.

So if I try method 2 (using staticmethod() at runtime):


fn2 is a static method, as I'd expect, but it is somehow not callable?

Your problems stem from the fact that attribute assignment to a class
doesn't always roundtrip:
(<unbound method A.f>, <function f at 0x4028eae4>)

If a class attribute is a descriptor i. e. it has a __get__() method (which
pure python functions do) A.method is not just a dictionary lookup but a
call

A.__dict__["method"].__get__(None, A)

This is slightly simplified as it doesn't take inheritance into account.
__get__() is free to apply whatever magic, but staticmethod just gives back
the original function whereas a some_func.__get__(None, SomeClass) gives
you an unbound method:
<unbound method A.f>

Is there a way to get the original function back out of the unbound method?
Let's see:
['__call__', '__class__', '__cmp__', '__delattr__', '__doc__', '__get__',
'__getattribute__', '__hash__', '__init__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__str__', 'im_class',
'im_func', 'im_self']

im_func seems promising:

Can someone explain what is going on here? Pointers to language spec,
code, PEPs etc gladly accepted.

Greg,
caught in a twisty little maze of descriptors, all different!

The two relevant documents are

http://users.rcn.com/python/download/Descriptor.htm
http://www.python.org/2.2.1/descrintro.html

Peter