Decorating functions without losing their signatures


R

Rotwang

Hi all,

Here's a Python problem I've come up against and my crappy solution.
Hopefully someone here can suggest something better. I want to decorate
a bunch of functions with different signatures; for example, I might
want to add some keyword-only arguments to all functions that return
instances of a particular class so that the caller can create instances
with additional attributes. So I do something like this:

import functools

def mydecorator(f):
@functools.wraps(f)
def wrapped(*args, attribute = None, **kwargs):
result = f(*args, **kwargs)
result.attribute = attribute
return result
return wrapped

@mydecorator
def f(x, y = 1, *a, z = 2, **k):
return something

The problem with this is, when I subsequently type 'f(' in IDLE, the
signature prompt that appears is not very useful; it looks like this:

(*args, attribute=None, **kwargs)

whereas I'd like it to look like this:

(x, y=1, *a, z=2, attribute=None, **k)


After thinking about it for a while I've come up with the following
abomination:

import inspect

def sigwrapper(sig):
if not isinstance(sig, inspect.Signature):
sig = inspect.signature(sig)
def wrapper(f):
ps = 'args = []\n\t\t'
ks = 'kwargs = {}\n\t\t'
for p in sig.parameters.values():
if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
ps = '%sargs.append(%s)\n\t\t' % (ps, p.name)
elif p.kind == p.VAR_POSITIONAL:
ps = '%sargs.extend(%s)\n\t\t' % (ps, p.name)
elif p.kind == p.KEYWORD_ONLY:
ks = '%skwargs[%r] = %s\n\t\t' % (ks, p.name, p.name)
elif p.kind == p.VAR_KEYWORD:
ks = '%skwargs.update(%s)\n\t\t' % (ks, p.name)
loc = {'wrapped': f}
defstring = ('def wrapouter(wrapped = wrapped):'
'\n\tdef wrapinner%s:'
'\n\t\t%s%sreturn wrapped(*args, **kwargs)'
'\n\treturn wrapinner' % (sig, ps, ks))
exec(defstring, f.__globals__, loc)
return loc['wrapouter']()
return wrapper

The function sigwrapper() may be passed an inspect.Signature object sig
(or function, if that function has the right signature) and returns a
decorator that gives any function the signature sig. I can then replace
my original decorator with something like

def mydecorator(f):
sig = inspect.signature(f)
sig = do_something(sig) # add an additional kw-only argument to sig
@functools.wraps(f)
@sigwrapper(sig)
def wrapped(*args, attribute = None, **kwargs):
result = f(*args, **kwargs)
result.attribute = attribute
return result
return wrapped

It seems to work, but I don't like it. Does anyone know of a better way
of doing the same thing?
 
Ad

Advertisements

S

Steven D'Aprano

Hi all,

Here's a Python problem I've come up against and my crappy solution.
Hopefully someone here can suggest something better. I want to decorate
a bunch of functions with different signatures; [...]
After thinking about it for a while I've come up with the following
abomination: [...]
It seems to work, but I don't like it. Does anyone know of a better way
of doing the same thing?


Wait until Python 3.4 or 3.5 (or Python 4000?) when functools.wraps
automatically preserves the function signature?

Alas, I think this is a hard problem to solve with current Python. You
might like to compare your solution with that of Michele Simionato's
"decorator" module:

http://micheles.googlecode.com/hg/decorator/documentation.html


See this for some other ideas:

http://numericalrecipes.wordpress.com/2009/05/25/signature-preserving-
function-decorators/



Good luck!
 
J

Jan Riechers

Hi all,

Here's a Python problem I've come up against and my crappy solution.
Hopefully someone here can suggest something better. I want to decorate
a bunch of functions with different signatures; for example, I might
want to add some keyword-only arguments to all functions that return
instances of a particular class so that the caller can create instances
with additional attributes. So I do something like this: [...]
It seems to work, but I don't like it. Does anyone know of a better way
of doing the same thing?

Hi,

I think you might want to check out that Pycon2013 Video about Metaclass
Prgoramming of David Beazley:
http://www.pyvideo.org/video/1716/python-3-metaprogramming

He explains how to passing attributes, such creating custom classes on
demand and returning there signatures even when wrapped.

I think that was what you wanted to archive?

Regards
Jan
 
M

Michele Simionato

After thinking about it for a while I've come up with the following

abomination


Alas, there is actually no good way to implement this feature in pure Python without abominations. Internally the decorator module does something similar to what you are doing. However, instead of cooking up yourself your custom solution, it is probably better if you stick to the decorator module which has been used in production for several years and has hundreds of thousands of downloads. I am not claiming that it is bug free, but it is stable,bug reports come very rarely and it works for all versions of Python from 2.5 to 3.3.
 
R

Rotwang

[...]

After thinking about it for a while I've come up with the following
abomination:

import inspect

def sigwrapper(sig):
if not isinstance(sig, inspect.Signature):
sig = inspect.signature(sig)
def wrapper(f):
ps = 'args = []\n\t\t'
ks = 'kwargs = {}\n\t\t'
for p in sig.parameters.values():
if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
ps = '%sargs.append(%s)\n\t\t' % (ps, p.name)
elif p.kind == p.VAR_POSITIONAL:
ps = '%sargs.extend(%s)\n\t\t' % (ps, p.name)
elif p.kind == p.KEYWORD_ONLY:
ks = '%skwargs[%r] = %s\n\t\t' % (ks, p.name, p.name)
elif p.kind == p.VAR_KEYWORD:
ks = '%skwargs.update(%s)\n\t\t' % (ks, p.name)
loc = {'wrapped': f}
defstring = ('def wrapouter(wrapped = wrapped):'
'\n\tdef wrapinner%s:'
'\n\t\t%s%sreturn wrapped(*args, **kwargs)'
'\n\treturn wrapinner' % (sig, ps, ks))
exec(defstring, f.__globals__, loc)
return loc['wrapouter']()
return wrapper

Oops! Earlier I found out the hard way that this fails when the
decorated function has arguments called 'args' or 'kwargs'. Here's a
modified version that fixes said bug, but presumably not the many others
I haven't noticed yet:

def sigwrapper(sig):
if not isinstance(sig, inspect.Signature):
sig = inspect.signature(sig)
n = 0
while True:
pn = 'p_%i' % n
kn = 'k_%i' % n
if pn not in sig.parameters and kn not in sig.parameters:
break
n += 1
ps = '%s = []\n\t\t' % pn
ks = '%s = {}\n\t\t' % kn
for p in sig.parameters.values():
if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD):
ps = '%s%s.append(%s)\n\t\t' % (ps, pn, p.name)
elif p.kind == p.VAR_POSITIONAL:
ps = '%s%s.extend(%s)\n\t\t' % (ps, pn, p.name)
elif p.kind == p.KEYWORD_ONLY:
ks = '%s%s[%r] = %s\n\t\t' % (ks, kn, p.name, p.name)
elif p.kind == p.VAR_KEYWORD:
ks = '%s%s.update(%s)\n\t\t' % (ks, kn, p.name)
defstring = ('def wrapouter(wrapped = wrapped):'
'\n\tdef wrapinner%s:'
'\n\t\t%s%sreturn wrapped(*%s, **%s)'
'\n\treturn wrapinner' % (sig, ps, ks, pn, kn))
def wrapper(f):
loc = {'wrapped': f}
exec(defstring, f.__globals__, loc)
return loc['wrapouter']()
return wrapper
 
R

Rotwang

Hi all,

Here's a Python problem I've come up against and my crappy solution.
Hopefully someone here can suggest something better. I want to decorate
a bunch of functions with different signatures; [...]
After thinking about it for a while I've come up with the following
abomination: [...]
It seems to work, but I don't like it. Does anyone know of a better way
of doing the same thing?


Wait until Python 3.4 or 3.5 (or Python 4000?) when functools.wraps
automatically preserves the function signature?

Alas, I think this is a hard problem to solve with current Python. You
might like to compare your solution with that of Michele Simionato's
"decorator" module:

http://micheles.googlecode.com/hg/decorator/documentation.html


See this for some other ideas:

http://numericalrecipes.wordpress.com/2009/05/25/signature-preserving-
function-decorators/



Good luck!

Thanks. It'll take me a while to fully absorb the links, but it looks
like both are similarly based on abusing the exec function.

Thanks to Jan too.
 
Ad

Advertisements

R

Rotwang

Alas, there is actually no good way to implement this feature in pure
Python without abominations. Internally the decorator module does
something similar to what you are doing. However, instead of cooking up
yourself your custom solution, it is probably better if you stick to
the decorator module which has been used in production for several
years and has hundreds of thousands of downloads. I am not claiming
that it is bug free, but it is stable, bug reports come very rarely and
it works for all versions of Python from 2.5 to 3.3.

Thanks, I'll check it out. Looking at the link Steven provided, I didn't
see an easy way to add additional keyword-only arguments to a function's
signature, though (but then I've yet to figure out how the FunctionMaker
class works).
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top