Behavior of staticmethod in Python 3

[Marco Buttu]

In Python 3 the following two classes should be equivalent:

$ cat foo.py
class Foo:
def foo():
pass
print(callable(foo))

class Foo:
@staticmethod
def foo():
pass
print(callable(foo))

But they do not:

$ python3 foo.py
True
False

How come the metaclass does not skip the staticmethod decorator?

 

[Peter Otten]

In Python 3 the following two classes should be equivalent:

Says who?

$ cat foo.py
class Foo:
def foo():
pass
print(callable(foo))

class Foo:
@staticmethod
def foo():
pass
print(callable(foo))

But they do not:

$ python3 foo.py
True
False

How come the metaclass does not skip the staticmethod decorator?

What? Your print()s are executed inside the class body, so the classes don't
come into play at all.

Your script is saying that a staticmethod instance is not a callable object.
It need not be because

Foo.foo()

doesn't call the Foo.foo attribute directly, it calls

Foo.foo.__get__(None, Foo)()
.... def __get__(self, *args): print(args)
........ foo = D()
....(None, <class '__main__.Foo'>)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable

Look for "descriptor protocol" to learn the details.

 

[Marco Buttu]

Says who?


Your script is saying that a staticmethod instance is not a callable object.
It need not be because

Foo.foo()

Yes, you are right about Python 3. But in Python 2, if I am not going
wrong, there is not solution, and I need to define a function outside
the class. For instance:

$ cat config.py
class Configuration(object):

def positiveCheck(value):
if not value > 0:
raise AttributeError('Must be a positive number')

attributes = {
# Attribute name: (type, checkrule)
'myattr': (int, positiveCheck),
}

def __setattr__(self, name, value):
if not name in Configuration.attributes:
raise AttributeError("Attribute `%s` non allowed." %name)

expected_type, checkrule = Configuration.attributes[name]
if not isinstance(value, expected_type):
raise TypeError('The value %s is not of type %s' \
%(value, expected_type.__name__))
if callable(checkrule):
print('calling %s(%s)' %(checkrule.__name__, value))
checkrule(value)

super(Configuration, self).__setattr__(name, value)

The positive check works fine:
calling positiveCheck(-10)
Traceback (most recent call last):
...
AttributeError: Must be a positive number

But I cannot use the method as a function:
Traceback (most recent call last):
...
Configuration instance as first argument (got int instance instead).

Furthemore, I cannot use the method as a staticmethod, becase otherwise
it will not be callable inside the class body.

 

[Marco Buttu]

Says who?


Your script is saying that a staticmethod instance is not a callable object.
It need not be because

Foo.foo()

Yes, you are right about Python 3. But in Python 2, if I am not going
wrong, there is not solution, and I need to define a function outside
the class. For instance:

$ cat config.py
class Configuration(object):

def positiveCheck(value):
if not value > 0:
raise AttributeError('Must be a positive number')

attributes = {
# Attribute name: (type, checkrule)
'myattr': (int, positiveCheck),
}

def __setattr__(self, name, value):
if not name in Configuration.attributes:
raise AttributeError("Attribute `%s` non allowed." %name)

expected_type, checkrule = Configuration.attributes[name]
if not isinstance(value, expected_type):
raise TypeError('The value %s is not of type %s' \
%(value, expected_type.__name__))
if callable(checkrule):
print('calling %s(%s)' %(checkrule.__name__, value))
checkrule(value)

super(Configuration, self).__setattr__(name, value)

The positive check works fine:
calling positiveCheck(-10)
Traceback (most recent call last):
...
AttributeError: Must be a positive number

But I cannot use the method as a function:
Traceback (most recent call last):
...
Configuration instance as first argument (got int instance instead).

Furthemore, I cannot use the method as a staticmethod, becase otherwise
it will not be callable inside the class body.

 

[Steven D'Aprano]

In Python 3 the following two classes should be equivalent:

They certainly are not equivalent in *any* version of Python, because
staticmethods are not equivalent to instance methods.

$ cat foo.py
class Foo:
def foo():
pass
print(callable(foo))

class Foo:
@staticmethod
def foo():
pass
print(callable(foo))

But they do not:

$ python3 foo.py
True
False

And Python 2 gives the same result for staticmethods:

[steve@ando ~]$ python2.7
Python 2.7.2 (default, May 18 2012, 18:25:10)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py>
py> class Test(object):
.... @staticmethod
.... def method():
.... pass
.... print callable(method)
....
False

How come the metaclass does not skip the staticmethod decorator?

What makes you think the metaclass gets the opportunity to skip the
decorator? By the time the metaclass gets called, the decorator has
already run.

You seem to be conflating behaviour in Python 2 that doesn't actually
occur. Staticmethods are not directly callable in any version of Python.

The problem you seem to have is that you want to call a method both
during and after construction:

class MyClass(object):
def helper(arg):
return arg + 1
x = helper(10)
y = helper(20)
def method(self, arg):
return self.helper(arg)


Here, the attributes x and y rely on calling helper as a function, where
it does not receive a self argument, but when calling helper from inside
the instance method, or when calling it like MyClass.helper(), it will
receive a self argument.

Unfortunately there is no good solution to this using just built-ins.
staticmethod doesn't work, as it's not callable. However, we can create
our own callable version of staticmethod:


class callable_staticmethod(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, cls=None):
return self.func
def __call__(self, *args, **kwargs):
return self.func(*args, **kwargs)


class MyClass(object):
@callable_staticmethod
def helper(arg):
return arg + 1
x = helper(10)
y = helper(20)
def method(self, arg):
return self.helper(arg)


This should now work exactly as you hope.

 

[Antoon Pardon]

Op 23-11-13 10:01, Peter Otten schreef:

Your script is saying that a staticmethod instance is not a callable object.
It need not be because

Foo.foo()

doesn't call the Foo.foo attribute directly, it calls

Foo.foo.__get__(None, Foo)()

I think you are burdening the programmer with implemantation details
that don't matter to him.

IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.

 

[Peter Otten]

Yes, you are right about Python 3. But in Python 2, if I am not going
wrong, there is not solution, and I need to define a function outside
the class. For instance:

$ cat config.py
class Configuration(object):

def positiveCheck(value):
if not value > 0:
raise AttributeError('Must be a positive number')

attributes = {
# Attribute name: (type, checkrule)
'myattr': (int, positiveCheck),
}

Just add

positiveCheck = staticmethod(positiveCheck)

at this point and everything should work in both Python 2 and 3.
Alternatively use Steven's callable_staticmethod as a decorator.

def __setattr__(self, name, value):
if not name in Configuration.attributes:
raise AttributeError("Attribute `%s` non allowed." %name)

expected_type, checkrule = Configuration.attributes[name]
if not isinstance(value, expected_type):
raise TypeError('The value %s is not of type %s' \
%(value, expected_type.__name__))
if callable(checkrule):
print('calling %s(%s)' %(checkrule.__name__, value))
checkrule(value)

super(Configuration, self).__setattr__(name, value)

The positive check works fine:
calling positiveCheck(-10)
Traceback (most recent call last):
...
AttributeError: Must be a positive number

But I cannot use the method as a function:
Traceback (most recent call last):
...
Configuration instance as first argument (got int instance instead).

Furthemore, I cannot use the method as a staticmethod, becase otherwise
it will not be callable inside the class body.

PS: AttributeErrors should be raised when an attribute does not exist or
cannot be set; I recommend that you raise a ValueError in positiveCheck().

 

[Chris Angelico]

IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.

I absolutely agree. But isn't that already the case? I seem to be
missing something here.
@staticmethod
def foo():
pass
True

ChrisA

 

[Peter Otten]

Op 23-11-13 10:01, Peter Otten schreef:


I think you are burdening the programmer with implemantation details
that don't matter to him.

IMO if Foo.foo() is legal then Foo.foo is callable. That the actual call
is delegated to Foo.foo.__get__(None, Foo) shouldn't matter.

If you read the original post -- I think in this case the details do matter.

What is your highlevel explanation for
.... @staticmethod
.... def foo(): pass
.... try: foo()
.... except Exception as err:
.... print(err)
....
'staticmethod' object is not callable
or maybe clearer:
.... def foo(): pass
........ foo = foo
.... bar = bar
....False

How would you explain that without mentioning the descriptor protocol?

 

[Antoon Pardon]

Op 23-11-13 22:51, Peter Otten schreef:

If you read the original post -- I think in this case the details do matter.

What is your highlevel explanation for

I don't care about what kind of explanation. I care about a correct answer to
the question whether a particular object is callable (from a programmers point
of view). I'm sure you can give a very comprehensive explanation for why in
this case we get an incorrect answer but that doesn't make the behaviour correct.

Foo.foo() is legal here. So Foo.foo is callable. So you starting with it needn't
be callable is using "callable" with a different meaning than the natural
interpretation. Al the rest is just an attempt in getting others to accept your
use of "callable" instead of the natural one.

Foo.foo() being legal and Foo.foo not being callable is IMO a bug in python. No matter
what explanation you have for the behaviour.

 

[Peter Otten]

Foo.foo() being legal and Foo.foo not being callable is IMO a bug in
python.

Foo.foo() is legal, and Foo.foo is callable.

 

[Ian Kelly]

Op 23-11-13 22:51, Peter Otten schreef:

I don't care about what kind of explanation. I care about a correct answer to
the question whether a particular object is callable (from a programmers point
of view). I'm sure you can give a very comprehensive explanation for why in
this case we get an incorrect answer but that doesn't make the behaviour correct.

Foo.foo() is legal here. So Foo.foo is callable. So you starting with it needn't
be callable is using "callable" with a different meaning than the natural
interpretation. Al the rest is just an attempt in getting others to accept your
use of "callable" instead of the natural one.

Foo.foo() being legal and Foo.foo not being callable is IMO a bug in python. No matter
what explanation you have for the behaviour.

Your supposition that Foo.foo is not considered callable by Python is
false, as Chris already demonstrated, and I don't see where anybody
here has stated otherwise. What Peter wrote was that "a staticmethod
instance is not a callable object", which is absolutely correct, and
these two facts are consistent because Foo.foo is not a staticmethod
instance (Foo.__dict__['foo'] on the other hand *is* a staticmethod
instance and *is not* callable, and Foo.__dict__['foo']() will
correspondingly raise a TypeError).

 

[Peter Otten]

Op 23-11-13 10:01, Peter Otten schreef:


I think you are burdening the programmer with implemantation details
that don't matter to him.

Replacing "you" in your statement with "python" I was about to suggest to
make staticmethod(func) callable when I found this had already been
rejected:

[Python-Dev] Making staticmethod objects callable?
Nicolas Fleury nidoizo at yahoo.com
Wed Mar 1 15:57:12 CET 2006

https://mail.python.org/pipermail/python-dev/2006-March/061948.html

If you think you have compelling arguments, go ahead and explain them --
preferably on python-ideas.

 

[Steven D'Aprano]

Foo.foo() is legal here. So Foo.foo is callable.

Incorrect. Foo.foo() is legal for *any* identifiers Foo and foo. Since
Python is an extremely dynamic language, the compiler cannot (easily, or
at all) prohibit "illegal" combinations. The only combinations which are
illegal are those which are not legal identifier, e.g.:

while.spam
abc$xyz.eggs

Otherwise, any pair of legal identifiers are legal and will be accepted
by the compiler. Only at runtime does the lookup succeed or fail:

str.length # likely to fail, unless str has been shadowed
str.find # likely to succeed, unless str has been shadowed

If the lookup has succeeded, then and only then does a second lookup
occur, namely:

type(str.find).__call__

and if that succeeds it is called.

It isn't helpful to talk about function or method calls being "legal" or
"illegal" in Python, since such things are normally determined in terms
of *success* or *failure*, not permitted versus prohibited. Either the
full chain of lookups and function call will succeed, or something will
raise an exception and it will fail.

Foo.foo() being legal and Foo.foo not being callable is IMO a bug in
python. No matter what explanation you have for the behaviour.

I don't know why you keep going on about this point, since this is NOT
the behaviour the original poster is talking about. With foo decorated as
a staticmethod in class Foo, Foo.foo *is* callable. It takes 30 seconds
to try it yourself:


py> class Foo(object): # inherit from object necessary in Python 2
.... @staticmethod
.... def foo():
.... return "Success!"
....
py>
py> Foo.foo()
'Success!'
py> Foo().foo()
'Success!'


This is not what the OP is talking about. Try this instead:

py> Foo.__dict__['foo']()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'staticmethod' object is not callable


What's going on here? Let's have a look:

py> Foo.__dict__['foo']
<staticmethod object at 0x9d83194>
py> Foo.foo
<function foo at 0x9d80294>


You CANNOT understand this behaviour without understanding the descriptor
protocol, which is *fundamental* to Python, and has been since Python
2.2. Regular instance methods, class methods, static methods and
properties are all defined in terms of descriptors. Trying to understand
them without knowledge of descriptors is like trying to understand
generators and iterators without knowledge of StopIteration -- you can go
only so far before you get stuck and confused.

The OP discovered that you can call a regular function inside a class
body, outside of a method, during the class statement:


class MyClass:
def function():
return "stuff"

attr = function()


This works, but then you can't call MyClass().function() as it will
raise. So the OP tried making it a staticmethod:

class MyClass:
@staticmethod
def function():
return "stuff"

attr = function()


but that fails, because *staticmethod instances are not callable*. They
are descriptors. Only after the descriptor protocol gets a chance to run
do you get a callable function.

*This is not a bug.* Making staticmethod instances callable is a feature
enhancement, not a bug fix. Or you could just define your own callable-
staticmethod descriptor, it's easy enough to do.

 

[Antoon Pardon]

Op 24-11-13 11:43, Peter Otten schreef:

Foo.foo() is legal, and Foo.foo is callable.

Indeed, I had a kink in my brain which made it difficult to see
where I was going wrong myself. Sorry about that.

 

[Antoon Pardon]

Op 24-11-13 12:03, Peter Otten schreef:

Op 23-11-13 10:01, Peter Otten schreef:


I think you are burdening the programmer with implemantation details
that don't matter to him.

Replacing "you" in your statement with "python" I was about to suggest to
make staticmethod(func) callable when I found this had already been
rejected:

[Python-Dev] Making staticmethod objects callable?
Nicolas Fleury nidoizo at yahoo.com
Wed Mar 1 15:57:12 CET 2006

https://mail.python.org/pipermail/python-dev/2006-March/061948.html

If you think you have compelling arguments, go ahead and explain them --
preferably on python-ideas.

I am under no illusion that compelling arguments will make any difference.
People had argued for a ternary operator for years and it all fell on
deaf ears until finally a python developer was bitten by a nasty bug
while using one of the alternative that was always suggested here.

When you asked that python should behave following the principle of
least surprise and thus showed consistency where possible, the
standard trope was that a foolish consistency was the hobgoblin
of little minds.

I see no reason why this should go any way differently now.