Wrapping methods of built-in dict

Discussion in 'Python' started by shailesh, May 21, 2009.

  1. shailesh

    shailesh Guest

    Hello,

    I'm trying to write a class decorator which takes a function as an
    argument and wraps instancemethods of the class with the function.

    After reading a few examples on ActiveState and this blog post <http://
    wordaligned.org/articles/echo>, my first attempt goes something like
    this:

    def tracing_wrapper(fn):
    import functools
    @functools.wraps(fn)
    def wrapped(*args, **kwargs):
    print "invoking %s(%s, %s)" % (fn.__name__, args, kwargs)
    return fn(*args, **kwargs)
    return wrapped

    def factory(wrapper_fn):
    def class_wrapper(klass):
    import inspect
    for _, method in inspect.getmembers(klass, inspect.ismethod):
    setattr(klass, method.__name__, wrapper_fn(method))

    for _, fn in inspect.getmembers(klass, inspect.isfunction):
    setattr(klass, fn.__name__, staticmethod(wrapper_fn(fn)))

    return klass
    return class_wrapper

    @factory(tracing_wrapper)
    class MyObject(object):
    def a(self):
    print "a"

    @factory(tracing_wrapper)
    class MyDict(dict):
    pass

    >>> a = MyObject()
    >>> a.a()

    invoking a((<__main__.MyObject object at 0xb7bb5bcc>,), {})
    a

    Naturally, when wrapping the built-in dict type, I don't get the
    expected results.
    >>> md = MyDict()
    >>> md['hello'] = 1
    >>> md.get('hello')

    1

    The reason as far as I understand is that the methods on the built-in
    dict are not of MethodType or FunctionType so they are not included in
    the result of the inspect.getmembers call and are not wrapped.

    Here I'm stuck, unsure how to fix this. Could anyone suggest a way to
    wrap the methods of the built-in types? This recipe <http://
    code.activestate.com/recipes/252151/> for generalized proxies seems
    promising, but I'm not sure how to adapt it for use here.

    Thanks in advance.
    shailesh, May 21, 2009
    #1
    1. Advertising

  2. On Wed, 20 May 2009 18:42:38 -0700, shailesh wrote:

    > The reason as far as I understand is that the methods on the built-in
    > dict are not of MethodType or FunctionType



    That seems to be true:

    >>> type({}.get)

    <type 'builtin_function_or_method'>
    >>> type(dict.get)

    <type 'method_descriptor'>


    > so they are not included in
    > the result of the inspect.getmembers call and are not wrapped.


    But that isn't:

    >>> zip(*inspect.getmembers(dict))[0] # extract the names only

    ('__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
    '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__',
    '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
    '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
    '__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys',
    'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys',
    'pop', 'popitem', 'setdefault', 'update', 'values')


    So the problem isn't directly with getmembers, but with the predicate
    functions you have passed to it (inspect.isfunction and inspect.method).
    Try inspect.ismethoddescriptor instead.


    Hint: dir(inspect) and help(inspect.whatever) are useful :)



    --
    Steven
    Steven D'Aprano, May 21, 2009
    #2
    1. Advertising

  3. shailesh

    shailesh Guest

    On May 20, 7:31 pm, Steven D'Aprano
    <> wrote:
    > On Wed, 20 May 2009 18:42:38 -0700, shailesh wrote:
    > > The reason as far as I understand is that the methods on the built-in
    > > dict are not of MethodType or FunctionType

    >
    > That seems to be true:
    >
    > >>> type({}.get)

    > <type 'builtin_function_or_method'>
    >
    >>> type(dict.get>

    > <type 'method_descriptor'>
    >
    > > so they are not included in
    > > the result of the inspect.getmembers call and are not wrapped.

    >
    > But that isn't:
    >
    > >>> zip(*inspect.getmembers(dict))[0]  # extract the names only

    >
    > ('__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
    > '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__',
    > '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__',
    > '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
    > '__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys',
    > 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys',
    > 'pop', 'popitem', 'setdefault', 'update', 'values')
    >
    > So the problem isn't directly with getmembers, but with the predicate
    > functions you have passed to it (inspect.isfunction and inspect.method).
    > Try inspect.ismethoddescriptor instead.


    Thanks for that. Between ismethod, isfunction, ismethoddescriptor and
    isbuiltin I think I can cover all the types needed for wrapping
    classes.

    I began writing an equivalent version for wrapping objects instead of
    classes. The problem I run into is that there isn't a predicate for
    some methods of dict instances

    >>> all = set([ name for (name, method) in inspect.getmembers({}) ]) # get all the methods
    >>> builtin = set([ name for (name, method) in inspect.getmembers({}, inspect.isbuiltin) ]) # get the builtin methods
    >>> for m in all.difference(builtin): # print the types of what's left over

    .... print m, type(getattr({}, m))
    ....
    __ne__ <type 'method-wrapper'>
    __setattr__ <type 'method-wrapper'>
    __hash__ <type 'NoneType'>
    __delitem__ <type 'method-wrapper'>
    __str__ <type 'method-wrapper'>
    __getattribute__ <type 'method-wrapper'>
    __class__ <type 'type'>
    __cmp__ <type 'method-wrapper'>
    __delattr__ <type 'method-wrapper'>
    __iter__ <type 'method-wrapper'>
    __le__ <type 'method-wrapper'>
    __len__ <type 'method-wrapper'>
    __gt__ <type 'method-wrapper'>
    __setitem__ <type 'method-wrapper'>
    __lt__ <type 'method-wrapper'>
    __ge__ <type 'method-wrapper'>
    __eq__ <type 'method-wrapper'>
    __doc__ <type 'str'>
    __init__ <type 'method-wrapper'>
    __repr__ <type 'method-wrapper'>

    >>> [ attr for attr in dir(inspect) if attr.startswith('is') ] # list of predicates supported by inspect

    ['isabstract', 'isbuiltin', 'isclass', 'iscode', 'isdatadescriptor',
    'isframe', 'isfunction', 'isgenerator', 'isgeneratorfunction',
    'isgetsetdescriptor', 'ismemberdescriptor', 'ismethod',
    'ismethoddescriptor', 'ismodule', 'isroutine', 'istraceback']

    >>> inspect.getmembers({}, inspect.ismethod)

    []
    >>> inspect.getmembers({}, inspect.isfunction)

    []
    >>> inspect.getmembers({}, inspect.ismemthoddescriptor)

    []

    There doesn't seem to be a predicate returning method wrappers. Is
    there an alternate way to query an object for attributes that are of
    method wrappers?

    This exercise also makes me question if I'm going about this
    correctly. If I want to add functionality to the methods of a class or
    an object are decorators and the inspect module the pythonic way to go
    about it? I can think of alternative implementations either through
    metaclasses or proxy objects.

    Thanks again.
    shailesh, May 21, 2009
    #3
  4. On May 21, 5:55 pm, shailesh <> wrote:

    > There doesn't seem to be a predicate returning method wrappers. Is
    > there an alternate way to query an object for attributes that are of
    > method wrappers?


    Sure:
    >>> MethodWrapper = type({}.__init__)
    >>> isinstance([].__len__, MethodWrapper)

    True

    But you're better off catching everything by checking with callable()
    (or equivalently hasattr(obj, '__call__')).

    > This exercise also makes me question if I'm going about this
    > correctly. If I want to add functionality to the methods of a class or
    > an object are decorators and the inspect module the pythonic way to go
    > about it? I can think of alternative implementations either through
    > metaclasses or proxy objects.


    In my experience, it's quite unlikely to really want to decorate
    indiscriminately *all* methods of a class/instance, let alone all the
    special methods (e.g. __getattribute__ very rarely needs to be
    overridden). Do you have an actual use case or are you just playing
    around ?

    George
    George Sakkis, May 22, 2009
    #4
  5. shailesh

    shailesh Guest

    On May 21, 10:13 pm, George Sakkis <> wrote:
    > On May 21, 5:55 pm, shailesh <> wrote:
    >
    > > There doesn't seem to be a predicate returning method wrappers. Is
    > > there an alternate way to query an object for attributes that are of
    > > method wrappers?

    >
    > Sure:>>> MethodWrapper = type({}.__init__)
    > >>> isinstance([].__len__, MethodWrapper)

    >
    > True
    >
    > But you're better off catching everything by checking with callable()
    > (or equivalently hasattr(obj, '__call__')).
    >
    > > This exercise also makes me question if I'm going about this
    > > correctly. If I want to add functionality to the methods of a class or
    > > an object are decorators and the inspect module the pythonic way to go
    > > about it? I can think of alternative implementations either through
    > > metaclasses or proxy objects.

    >
    > In my experience, it's quite unlikely to really want to decorate
    > indiscriminately *all* methods of a class/instance, let alone all the
    > special methods (e.g. __getattribute__ very rarely needs to be
    > overridden). Do you have an actual use case or are you just playing
    > around ?


    The use case I'm exploring is automatic lock acquisition and release.
    I've been trying to create a decorator which protects objects against
    concurrent modification by placing all attribute behind a mutex. More
    fine grained locking will probably perform better, but the convenience
    and reliability of auto-locking is nice. Suggestions for alternate
    implementations are most welcome.

    - Shailesh
    shailesh, Jun 3, 2009
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Skip Montanaro
    Replies:
    0
    Views:
    412
    Skip Montanaro
    Aug 15, 2003
  2. Alexander Kozlovsky

    dict!ident as equivalent of dict["ident"]

    Alexander Kozlovsky, May 21, 2006, in forum: Python
    Replies:
    5
    Views:
    357
    Alexander Kozlovsky
    May 22, 2006
  3. Paul Melis

    dict.has_key(x) versus 'x in dict'

    Paul Melis, Dec 6, 2006, in forum: Python
    Replies:
    48
    Views:
    1,318
    Kent Johnson
    Dec 15, 2006
  4. Almad
    Replies:
    8
    Views:
    397
    Terry Reedy
    Dec 14, 2006
  5. Drew
    Replies:
    19
    Views:
    1,345
    Duncan Booth
    Mar 15, 2007
Loading...

Share This Page