Wrapping methods of built-in dict

[shailesh]

Hello,

I'm trying to write a class decorator which takes a function as an
argument and wraps instancemethods of the class with the function.

After reading a few examples on ActiveState and this blog post <http://
wordaligned.org/articles/echo>, my first attempt goes something like
this:

def tracing_wrapper(fn):
import functools
@functools.wraps(fn)
def wrapped(*args, **kwargs):
print "invoking %s(%s, %s)" % (fn.__name__, args, kwargs)
return fn(*args, **kwargs)
return wrapped

def factory(wrapper_fn):
def class_wrapper(klass):
import inspect
for _, method in inspect.getmembers(klass, inspect.ismethod):
setattr(klass, method.__name__, wrapper_fn(method))

for _, fn in inspect.getmembers(klass, inspect.isfunction):
setattr(klass, fn.__name__, staticmethod(wrapper_fn(fn)))

return klass
return class_wrapper

@factory(tracing_wrapper)
class MyObject(object):
def a(self):
print "a"

@factory(tracing_wrapper)
class MyDict(dict):
pass
invoking a((<__main__.MyObject object at 0xb7bb5bcc>,), {})
a

Naturally, when wrapping the built-in dict type, I don't get the
expected results.

md = MyDict()
md['hello'] = 1
md.get('hello')

1

The reason as far as I understand is that the methods on the built-in
dict are not of MethodType or FunctionType so they are not included in
the result of the inspect.getmembers call and are not wrapped.

Here I'm stuck, unsure how to fix this. Could anyone suggest a way to
wrap the methods of the built-in types? This recipe <http://
code.activestate.com/recipes/252151/> for generalized proxies seems
promising, but I'm not sure how to adapt it for use here.

Thanks in advance.

 

[Steven D'Aprano]

The reason as far as I understand is that the methods on the built-in
dict are not of MethodType or FunctionType

That seems to be true:

so they are not included in
the result of the inspect.getmembers call and are not wrapped.

But that isn't:

zip(*inspect.getmembers(dict))[0] # extract the names only

('__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
'__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__',
'__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
'__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys',
'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys',
'pop', 'popitem', 'setdefault', 'update', 'values')


So the problem isn't directly with getmembers, but with the predicate
functions you have passed to it (inspect.isfunction and inspect.method).
Try inspect.ismethoddescriptor instead.


Hint: dir(inspect) and help(inspect.whatever) are useful

 

[shailesh]

The reason as far as I understand is that the methods on the built-in
dict are not of MethodType or FunctionType

That seems to be true:

so they are not included in
the result of the inspect.getmembers call and are not wrapped.

But that isn't:

zip(*inspect.getmembers(dict))[0]  # extract the names only

('__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
'__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__',
'__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
'__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys',
'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys',
'pop', 'popitem', 'setdefault', 'update', 'values')

So the problem isn't directly with getmembers, but with the predicate
functions you have passed to it (inspect.isfunction and inspect.method).
Try inspect.ismethoddescriptor instead.

Thanks for that. Between ismethod, isfunction, ismethoddescriptor and
isbuiltin I think I can cover all the types needed for wrapping
classes.

I began writing an equivalent version for wrapping objects instead of
classes. The problem I run into is that there isn't a predicate for
some methods of dict instances

all = set([ name for (name, method) in inspect.getmembers({}) ]) # get all the methods
builtin = set([ name for (name, method) in inspect.getmembers({}, inspect.isbuiltin) ]) # get the builtin methods
for m in all.difference(builtin): # print the types of what's left over

.... print m, type(getattr({}, m))
....
__ne__ <type 'method-wrapper'>
__setattr__ <type 'method-wrapper'>
__hash__ <type 'NoneType'>
__delitem__ <type 'method-wrapper'>
__str__ <type 'method-wrapper'>
__getattribute__ <type 'method-wrapper'>
__class__ <type 'type'>
__cmp__ <type 'method-wrapper'>
__delattr__ <type 'method-wrapper'>
__iter__ <type 'method-wrapper'>
__le__ <type 'method-wrapper'>
__len__ <type 'method-wrapper'>
__gt__ <type 'method-wrapper'>
__setitem__ <type 'method-wrapper'>
__lt__ <type 'method-wrapper'>
__ge__ <type 'method-wrapper'>
__eq__ <type 'method-wrapper'>
__doc__ <type 'str'>
__init__ <type 'method-wrapper'>

[ attr for attr in dir(inspect) if attr.startswith('is') ] # list of predicates supported by inspect

['isabstract', 'isbuiltin', 'isclass', 'iscode', 'isdatadescriptor',
'isframe', 'isfunction', 'isgenerator', 'isgeneratorfunction',
'isgetsetdescriptor', 'ismemberdescriptor', 'ismethod',
'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback']

inspect.getmembers({}, inspect.ismethod) []
inspect.getmembers({}, inspect.isfunction) []
inspect.getmembers({}, inspect.ismemthoddescriptor)

[]

There doesn't seem to be a predicate returning method wrappers. Is
there an alternate way to query an object for attributes that are of
method wrappers?

This exercise also makes me question if I'm going about this
correctly. If I want to add functionality to the methods of a class or
an object are decorators and the inspect module the pythonic way to go
about it? I can think of alternative implementations either through
metaclasses or proxy objects.

Thanks again.

 

[George Sakkis]

There doesn't seem to be a predicate returning method wrappers. Is
there an alternate way to query an object for attributes that are of
method wrappers?
Sure:

MethodWrapper = type({}.__init__)
isinstance([].__len__, MethodWrapper)

True

But you're better off catching everything by checking with callable()
(or equivalently hasattr(obj, '__call__')).

This exercise also makes me question if I'm going about this
correctly. If I want to add functionality to the methods of a class or
an object are decorators and the inspect module the pythonic way to go
about it? I can think of alternative implementations either through
metaclasses or proxy objects.

In my experience, it's quite unlikely to really want to decorate
indiscriminately *all* methods of a class/instance, let alone all the
special methods (e.g. __getattribute__ very rarely needs to be
overridden). Do you have an actual use case or are you just playing
around ?

George

 

[shailesh]

There doesn't seem to be a predicate returning method wrappers. Is
there an alternate way to query an object for attributes that are of
method wrappers?

Sure:>>> MethodWrapper = type({}.__init__)

isinstance([].__len__, MethodWrapper)

True

But you're better off catching everything by checking with callable()
(or equivalently hasattr(obj, '__call__')).

This exercise also makes me question if I'm going about this
correctly. If I want to add functionality to the methods of a class or
an object are decorators and the inspect module the pythonic way to go
about it? I can think of alternative implementations either through
metaclasses or proxy objects.

In my experience, it's quite unlikely to really want to decorate
indiscriminately *all* methods of a class/instance, let alone all the
special methods (e.g. __getattribute__ very rarely needs to be
overridden). Do you have an actual use case or are you just playing
around ?

The use case I'm exploring is automatic lock acquisition and release.
I've been trying to create a decorator which protects objects against
concurrent modification by placing all attribute behind a mutex. More
fine grained locking will probably perform better, but the convenience
and reliability of auto-locking is nice. Suggestions for alternate
implementations are most welcome.

- Shailesh