Decorating methods - where do my arguments go?

[Mikael Olofsson]

Hi all!

I have long tried to avoid decorators, but now I find myself in a
situation where I think they can help. I seem to be able to decorate
functions, but I fail miserably when trying to decorate methods. The
information I have been able to find on-line focuses on decorating
functions, and do not mention any special problem regarding methods.

Consider the following example. I start by defining a simple decorator:
.... def __init__(self,func):
.... self._func = func
.... def __call__(self, *args):
.... print 'Decorator:', args
.... self._func(*args)

Then I decorate a function:
.... def func(*args):
.... print 'Function: ', args

Let's try that:
Decorator: (1, 2, 3)
Function: (1, 2, 3)

OK! That was what I expected. Let's decorate a method:
.... @test_decorator
.... def meth(self,*args):
.... print 'Method: ', args

Then try that:
Decorator: (1, 2, 3)
Method: (2, 3)

Oops! That's weird. I had expected - or at least wanted - the same
result as for the function above.

Where did the first argument go? If it was sent to the original method
meth as the real first argument (self), then why did I not get an exception?

More importantly: How do I write a decorator that does not drop arguments?

I guess it all has to do with the fact that the returned callable isn't
a method of cls. Is it possible to write a decorator that returns a
callable that is a method of cls, when used on methods in cls?

/MiO

 

[MRAB]

Hi all!

I have long tried to avoid decorators, but now I find myself in a
situation where I think they can help. I seem to be able to decorate
functions, but I fail miserably when trying to decorate methods. The
information I have been able to find on-line focuses on decorating
functions, and do not mention any special problem regarding methods.

Consider the following example. I start by defining a simple decorator:

... def __init__(self,func):
... self._func = func
... def __call__(self, *args):
... print 'Decorator:', args
... self._func(*args)

Then I decorate a function:

... def func(*args):
... print 'Function: ', args

Let's try that:

Decorator: (1, 2, 3)
Function: (1, 2, 3)

OK! That was what I expected. Let's decorate a method:

... @test_decorator
... def meth(self,*args):
... print 'Method: ', args

Then try that:

Decorator: (1, 2, 3)
Method: (2, 3)

Oops! That's weird. I had expected - or at least wanted - the same
result as for the function above.

Where did the first argument go? If it was sent to the original method
meth as the real first argument (self), then why did I not get an
exception?

More importantly: How do I write a decorator that does not drop arguments?

I guess it all has to do with the fact that the returned callable isn't
a method of cls. Is it possible to write a decorator that returns a
callable that is a method of cls, when used on methods in cls?

@test_decorator
def meth(self,*args):
print 'self: ', self
print 'Method: ', args
Decorator: (1, 2, 3)
self: 1
Method: (2, 3)

 

[Peter Otten]

Hi all!

I have long tried to avoid decorators, but now I find myself in a
situation where I think they can help. I seem to be able to decorate
functions, but I fail miserably when trying to decorate methods. The
information I have been able to find on-line focuses on decorating
functions, and do not mention any special problem regarding methods.

Consider the following example. I start by defining a simple decorator:

... def __init__(self,func):
... self._func = func
... def __call__(self, *args):
... print 'Decorator:', args
... self._func(*args)

Then I decorate a function:

... def func(*args):
... print 'Function: ', args

Let's try that:

Decorator: (1, 2, 3)
Function: (1, 2, 3)

OK! That was what I expected. Let's decorate a method:

... @test_decorator
... def meth(self,*args):
... print 'Method: ', args

Then try that:

Decorator: (1, 2, 3)
Method: (2, 3)

Oops! That's weird. I had expected - or at least wanted - the same
result as for the function above.

Where did the first argument go? If it was sent to the original method
meth as the real first argument (self), then why did I not get an
exception?

More importantly: How do I write a decorator that does not drop arguments?

I guess it all has to do with the fact that the returned callable isn't
a method of cls. Is it possible to write a decorator that returns a
callable that is a method of cls, when used on methods in cls?

/MiO

You have to turn your decorator into a descriptor by providing a __get__()
method. A primitive example:

class test_decorator(object):
def __init__(self,func):
self._func = func
def __call__(self, *args):
print 'Decorator:', args
self._func(self.inst, *args)
def __get__(self, inst, cls):
self.inst = inst
return self

Read more about the details at

http://users.rcn.com/python/download/Descriptor.htm

Peter

 

[George Sakkis]

Hi all!

I have long tried to avoid decorators, but now I find myself in a
situation where I think they can help. I seem to be able to decorate
functions, but I fail miserably when trying to decorate methods. The
information I have been able to find on-line focuses on decorating
functions, and do not mention any special problem regarding methods.

Consider the following example. I start by defining a simple decorator:

... def __init__(self,func):
... self._func = func
... def __call__(self, *args):
... print 'Decorator:', args
... self._func(*args)

Then I decorate a function:

... def func(*args):
... print 'Function: ', args

Let's try that:

Decorator: (1, 2, 3)
Function: (1, 2, 3)

OK! That was what I expected. Let's decorate a method:

... @test_decorator
... def meth(self,*args):
... print 'Method: ', args

Then try that:

Decorator: (1, 2, 3)
Method: (2, 3)

Oops! That's weird. I had expected - or at least wanted - the same
result as for the function above.

Where did the first argument go? If it was sent to the original method
meth as the real first argument (self), then why did I not get an exception?

More importantly: How do I write a decorator that does not drop arguments?

I guess it all has to do with the fact that the returned callable isn't
a method of cls. Is it possible to write a decorator that returns a
callable that is a method of cls, when used on methods in cls?

Yes, just return an actual function from the decorator instead of a
callable object:

def test_decorator2(func):
def wrapper(*args):
print 'Decorator2:', args
func(*args)
return wrapper


class cls(object):
@test_decorator
def meth(self,*args):
print 'Method: ', args

@test_decorator2
def meth2(self,*args):
print 'Method2: ', args
Decorator2: (<__main__.cls object at 0x8766ecc>, 1, 2, 3)
Method2: (1, 2, 3)

The reason this works is that functions are already descriptors, so
you don't have to explicitly define __get__() as you would for a new
callable class (see Peter's example).

HTH,
George

 

[Michele Simionato]

 >>> class test_decorator(object):
...     def __init__(self,func):
...         self._func = func
...     def __call__(self, *args):
...         print 'Decorator:', args
...         self._func(*args)

Or you could use the decorator module (http://pypi.python.org/pypi/
decorator):

from decorator import decorator

@decorator
def test_decorator(func, *args, **kw):
print 'Decorator:', args
return func(*args)

 

[Mikael Olofsson]

You have to turn your decorator into a descriptor by providing a __get__()
method. A primitive example:

class test_decorator(object):
def __init__(self,func):
self._func = func
def __call__(self, *args):
print 'Decorator:', args
self._func(self.inst, *args)
def __get__(self, inst, cls):
self.inst = inst
return self

Thanks! Works perfectly for methods, as far as I can see. That's
perfectly OK, since that is what I was asking for. What I failed to
include in my original post is that I intend to use the decorator for
functions as well. The above fails for functions on the second row of
__call__ with

AttributeError: 'test_decorator' object has no attribute 'inst'

It seems to me like __get__ is not called when decorating a function. I
guess there's no reasonable value for inst in that case. It might be the
function itself, but it doesn't make sense to call the function with
itself as its first argument, when it isn't supposed to be called that way.

George Sakkis decorator function solution seems to work equally well for
functions and methods. However, I prefer the cleaner encapsulation given
by a class. Based on those observations, I think I will use the
following approach:
.... _inst = None
.... def __init__(self,func):
.... self._func = func
.... def __call__(self, *args):
.... print 'Decorator:', args
.... if self._inst is None:
.... print 'We seem to be decorating a function.'
.... self._func(*args)
.... else:
.... print 'We seem to be decorating a method.'
.... self._func(self._inst,*args)
.... def __get__(self, inst, cls):
.... self._inst = inst
.... return self
........ def func(*args):
.... print 'Function: ', args
....Decorator: (1, 2, 3)
We seem to be decorating a function.
Function: (1, 2, 3).... @test_decorator3
.... def meth(self,*args):
.... print 'Method: ', args
....Decorator: (1, 2, 3)
We seem to be decorating a method.
Method: (1, 2, 3)

If there are any drawbacks with this approach that I fail to see, please
enlighten me.

/MiO

 

[Mikael Olofsson]

Yes, just return an actual function from the decorator instead of a
callable object:

def test_decorator2(func):
def wrapper(*args):
print 'Decorator2:', args
func(*args)
return wrapper


class cls(object):
@test_decorator
def meth(self,*args):
print 'Method: ', args

@test_decorator2
def meth2(self,*args):
print 'Method2: ', args

Thanks! This has the merit over Peter's solution that it seems to work
for both functions and methods. However, as you can see from my answer
to Peter, I think I will go for a class based approach anyway. Still,
your answer helped me grasping the concepts.

/MiO

 

[Mikael Olofsson]

The __get__ method should be returning a new object, NOT modifying the
state of the decorator. As written it will break badly and unexpectedly
in a variety of situations:

[snip good examples of things going bad]

Ouch! So, does that mean that George's solution based on a function is
the way to go, or does that mean that my __call__ should analyze the
passed callable?

/MiO

 

[Peter Otten]

The __get__ method should be returning a new object, NOT modifying the
state of the decorator. As written it will break badly and unexpectedly
in a variety of situations:

[snip good examples of things going bad]

Ouch! So, does that mean that George's solution based on a function is
the way to go, or does that mean that my __call__ should analyze the
passed callable?

I usually use decorator functions, but I think the following should work,
too:

class deco(object):
def __init__(self, func):
self._func = func
def __call__(self, *args):
print "Decorator:", args
self._func(*args)
def __get__(self, *args):
return deco(self._func.__get__(*args))

Peter

 

[Mikael Olofsson]

I usually use decorator functions, but I think the following should work,
too:

class deco(object):
def __init__(self, func):
self._func = func
def __call__(self, *args):
print "Decorator:", args
self._func(*args)
def __get__(self, *args):
return deco(self._func.__get__(*args))

This looks like a winner to me. It is elegant, works for functions as
well as for methods, and does not mess with the internals of the
decorator which avoids the problems that Duncan pointed out.

Thanks Peter, and thanks everyone else that chimed in and contributed.

/MiO

 

[Michele Simionato]

This looks like a winner to me. It is elegant, works for functions as
well as for methods, and does not mess with the internals of the
decorator which avoids the problems that Duncan pointed out.

Still it turns something which is a function into an object and you
lose the
docstring and the signature. pydoc will not be too happy with this
approach.

 

[Mikael Olofsson]

> Still it turns something which is a function into an object and
> you lose the docstring and the signature. pydoc will not be too
> happy with this approach.

I don't know why Mikael wants to use a class rather than a function
but if he wants to save state between calls this isn't going to help.

I've never used decorators before, and that's the main reason for my
confusion. I have found myself in a situation where I think that
decorators is the way to go. So I tried to find information about how to
write them the usual way: Googling and browsing documentation. I found
examples using both functions and classes, but not much explanation of
what's going on, especially not in the class case. I didn't find any
arguments for favouring any of those alternatives. Actually, most
examples I found did not even include the definition of the actual
decorator or any information about where they might come from, only the
@-row.

Based on the above observations, I assumed that both alternatives were
equally possible. The first answers I got here also led me to believe
that that was the case. My understanding is that in most cases, you can
use a callable object instead of a function with little or no practical
difference, except the syntax of its definition. In my eyes, a class
definition looks nicer and is more readable than a nested function.
Therefore, I tend to define a class with a __call__ method and helper
methods instead of nested functions, if I find the need.

Practicality beats purity, and it seem like using a class for decoration
is a lot more hassle than using a function. I fold.

/MiO