Observer-Pattern by (simple) decorator

[Wildemar Wildenburger]

Hello folks
This has got to be an FAQ, but somehow I can't seem to find an answer
that I understand ...

I thought: I'll just write a decorator that lets me react to method
calls easily (the ever so popular observer-pattern). I've looked at some
recepies, but I just don't get them (I'm feeling kinda dumb today, sorry).
What I'd *like* to do is this:

def observable(func):
# magic code

class SomeActor(object):
@observable
def meth(self, foo):
print foo

def callback(instance):
print "Yippie, I've been called on", instance
instance.bar = True

sa = SomeActor()
sa.meth.add_callback(callback)

# now this
sa.meth("I'm the boring old argument")

# I would like to result in this# or so



This is more complicated than expected mainly for two reasons:

* I can't find a way to pass the proper 'instance' argument to
callback, that is, I don't know how to retrieve the instance that
meth() was called on, because the observable decorator only gets
the *function* object but not the *method*. (I hope this was clear
enough)
* Also, I don't see how I could add the add_callback() method to the
meth object. That doesn't seem possible. I can add it to meth's
function object just fine in the definition of observable, but I
thats not what I really want. This is probably just a cosmetic
issue because I don't like the idea of calling
sa.meth.im_func.add_callback(callback).


Any ideas? Would be great.
/W

 

[Steven Bethard]

I thought: I'll just write a decorator that lets me react to method
calls easily (the ever so popular observer-pattern). I've looked at some
recepies, but I just don't get them (I'm feeling kinda dumb today, sorry). [snip]
This is more complicated than expected mainly for two reasons:

* I can't find a way to pass the proper 'instance' argument to
callback, that is, I don't know how to retrieve the instance that
meth() was called on, because the observable decorator only gets
the *function* object but not the *method*. (I hope this was clear
enough)
* Also, I don't see how I could add the add_callback() method to the
meth object. That doesn't seem possible. I can add it to meth's
function object just fine in the definition of observable, but I
thats not what I really want. This is probably just a cosmetic
issue because I don't like the idea of calling
sa.meth.im_func.add_callback(callback).

I think you want to define __get__ on your Observable class so that it
can do the right thing when the method is bound to the instance:
.... def __init__(self, func, instance=None, observers=None):
.... if observers is None:
.... observers = []
.... self.func = func
.... self.instance = instance
.... self.observers = observers
.... def __get__(self, obj, cls=None):
.... if obj is None:
.... return self
.... else:
.... func = self.func.__get__(obj, cls)
.... return Observable(func, obj, self.observers)
.... def __call__(self, *args, **kwargs):
.... result = self.func(*args, **kwargs)
.... for observer in self.observers:
.... observer(self.instance)
.... return result
.... def add_callback(self, callback):
.... self.observers.append(callback)
........ @Observable
.... def meth(self, foo):
.... print foo
........ print "Yippie, I've been called on", instance
.... instance.bar = True
....I'm the boring old argument
True


STeVe

 

[Wildemar Wildenburger]

I think you want to define __get__ on your Observable class so that it
can do the right thing when the method is bound to the instance:
[snip]
... def __get__(self, obj, cls=None):
... if obj is None:
... return self
... else:
... func = self.func.__get__(obj, cls)
... return Observable(func, obj, self.observers)
[snip]

Great, that does it! Thanks a billion

It took me quite some time understanding what the __get__ method does,
but I think now I figured it out. I've made a quantum leap in my
understanding of Python (esp. method binding) with this! Awesome!


/W

 

[Gabriel Genellina]

En Fri, 01 Jun 2007 21:25:50 -0300, Wildemar Wildenburger

Steven Bethard wrote:

It took me quite some time understanding what the __get__ method does,
but I think now I figured it out. I've made a quantum leap in my
understanding of Python (esp. method binding) with this! Awesome!

To proceed to next energy level, read some articles from
http://www.python.org/doc/newstyle/

 

[David Wahler]

I think you want to define __get__ on your Observable class so that it
can do the right thing when the method is bound to the instance:
... def __init__(self, func, instance=None, observers=None):
... if observers is None:
... observers = []
... self.func = func
... self.instance = instance
... self.observers = observers
... def __get__(self, obj, cls=None):
... if obj is None:
... return self
... else:
... func = self.func.__get__(obj, cls)
... return Observable(func, obj, self.observers)
... def __call__(self, *args, **kwargs):
... result = self.func(*args, **kwargs)
... for observer in self.observers:
... observer(self.instance)
... return result
... def add_callback(self, callback):
... self.observers.append(callback)
...... @Observable
... def meth(self, foo):
... print foo
...... print "Yippie, I've been called on", instance
... instance.bar = True
...

Is this desired behavior?
42
Yippie, I've been called on <__main__.SomeActor object at 0x00C23550>

-- David

 

[Wildemar Wildenburger]

Is this desired behavior?


True

No it's not, but I get
False


I don't see how your behaviour should come about ... a new observer-list
is created for every decorated method, so there is no problem.

Now I'm confused ?-|

/W

 

[Peter Otten]

No it's not, but I get

False

Then you have modified the code posted by Steven Bethard.

I don't see how your behaviour should come about ... a new observer-list
is created for every decorated method, so there is no problem.

Yes, but that list is shared across instances of SomeActor.

Now I'm confused ?-|

You start with one Observable per method:
True

This Observable knows nothing about a SomeActor instance (because there is
none).
True

Everytime you access meth from a SomeActor instance the __get__() method is
invoked and a new Observable is created, this time with an instance:
<tmp.SomeActor object at 0x401d56ec>

But all of these Observables share the same observables list
True

If you want per-instance callbacks you have to store the observers list (or
the bound method) in the instance:
.... def __init__(self):
.... self.meth = Observable(self.meth, self)
.... def meth(self, foo): print foo
........ def f(instance):
.... print s, instance
.... return f
....

1

2
beta <__main__.SomeActor object at 0x401d5ccc>

Note that with this approach Observable need not be a descriptor; I was just
too lazy to rewrite it.

Peter

 

[David Wahler]

Then you have modified the code posted by Steven Bethard.


Yes, but that list is shared across instances of SomeActor.


You start with one Observable per method:


<tmp.Observable object at 0x401d5b0c>>>> SomeActor.meth is SomeActor.meth

True

This Observable knows nothing about a SomeActor instance (because there is
none).


True

Everytime you access meth from a SomeActor instance the __get__() method is
invoked and a new Observable is created, this time with an instance:


<tmp.SomeActor object at 0x401d56ec>

But all of these Observables share the same observables list


True

If you want per-instance callbacks you have to store the observers list (or
the bound method) in the instance:


... def __init__(self):
... self.meth = Observable(self.meth, self)
... def meth(self, foo): print foo
...>>> a = SomeActor()

... def f(instance):
... print s, instance
... return f
...>>> a.meth.add_callback(make_callback("alpha"))

1
alpha <__main__.SomeActor object at 0x401d5c6c>>>> b.meth(2)

2
beta <__main__.SomeActor object at 0x401d5ccc>

Note that with this approach Observable need not be a descriptor; I was just
too lazy to rewrite it.

Here's my attempt at an implementation that works as a decorator. It
stores the observers in a dictionary attribute of the instance, using
the method name as a key. For example:

a = SomeActor()
a.meth.add_callback(callback)
a._observers

a.meth.observers is a._observers['meth']

True

######################################################################

class Observable(object):
def __init__(self, func, instance=None):
self.func = func
self.instance = instance
if instance is not None and not hasattr(self.instance,
'_observers'):
self.instance._observers = {}

def __get__(self, obj, cls=None):
if obj is None:
return self
else:
func = self.func.__get__(obj, cls)
return Observable(func, obj)

def __call__(self, *args, **kwargs):
result = self.func(*args, **kwargs)
for observer in self.observers:
observer(self.instance)
return result

@property
def observers(self):
func_name = self.func.im_func.func_name
return self.instance._observers.setdefault(func_name,[])

def add_callback(self, callback):
self.observers.append(callback)

######################################################################

This implementation has a few drawbacks that I can see:

* It doesn't allow for the syntax SomeActor.meth(instance, *args).
I haven't spent the time to make it work on unbound as well as
bound methods.

* It may not play well with inheritance. If the subclass overrides
an Observable method of the superclass, and declares its override
Observable as well, the callback will be called twice. I'm not
sure exactly how to handle this.

* Perhaps it would be better to use the function object itself as
the dictionary key, rather than its name?

Anyway, these are just my initial thoughts -- I don't have the time to
really think this through thoroughly.

-- David

 

[Wildemar Wildenburger]

Then you have modified the code posted by Steven Bethard.

Oops. Indeed I did.
I changed this
.... def __init__(self, func, instance=None, observers=None):
.... if observers is None:
.... observers = []
.... self.func = func
.... self.instance = instance
.... self.observers = observers


to this
.... def __init__(self, func, instance=None, observers=None):
.... self.func = func
.... self.instance = instance
.... self.observers = observers or []

Yes, but that list is shared across instances of SomeActor.

Oh! I thought a new Observable-object is created for every *instance* of
SomeActor, where there is actually only one Observable per method that
decorates *all* methods (all meth() methods, that is). Am I right?

And I seem to have accidentally "solved" this by rewriting the
__init__() method the way I did. Now it creates a new list even when it
gets passed an empty list. How very smart of me ...

Looking at the way it is going to be used, I don't see a problem with
this. Or am I missing something that could stab me in the back here?


Thanks for the thorough explanation. But you *did* mock me with that
"SomeActor.meth is SomeActor.meth"-line, right?

If you want per-instance callbacks you have to store the observers list (or
the bound method) in the instance:


... def __init__(self):
... self.meth = Observable(self.meth, self)
... def meth(self, foo): print foo

Seems good, but I'm trying to take as much hassle out of the programmer
(probably me, mostly) as possible in order to watch method calls. Simply
sticking a decorator in front of a methos, with no arguments, no
nothing, seems less prone to errors.
Also, reading "self.meth = Observable(self.meth, self)" kinda makes
sense now, but I *know* I'd be scratching my head about this in no less
than a few weeks. BTW, that might be my single favorite line of python
code so far ...

/W

 

[Wildemar Wildenburger]

Here's my attempt at an implementation that works as a decorator. It
stores the observers in a dictionary attribute of the instance, using
the method name as a key. For example:


[snip: another nice aproach]
Anyway, these are just my initial thoughts -- I don't have the time to
really think this through thoroughly.

-- David

Neither do I at the moment, but thats an interesting idea, still. I'll
keep it in the back of me head. For now I'll keep testing Steven's
version, see how far that gets me. Thanks for the list of possible
problems as well; inheritance is something I hadn't thought about yet ...

Argh! Considering that this is only the *first* step towards my awesome
app ...

/W

 

[Peter Otten]

Thanks for the thorough explanation. But you did mock me with that
"SomeActor.meth is SomeActor.meth"-line, right?

Acually, no:
.... def __get__(*args): return object()
........ meth = Descriptor()
....False

Peter

 

[Steven Bethard]

I think you want to define __get__ on your Observable class so that it
can do the right thing when the method is bound to the instance:

[snip]
Is this desired behavior?
42
Yippie, I've been called on <__main__.SomeActor object at 0x00C23550>

Yeah, I wasn't sure whether the observers were meant to be shared or
not. Yes, they are shared in the previous code. If you don't want them
to be shared, you can just use a WeakKeyDictionary to map instances to
their appropriate observer lists::
.... _observer_map = weakref.WeakKeyDictionary()
.... def __init__(self, func, instance=None):
.... if instance is None:
.... observers = []
.... else:
.... observers = self._observer_map.setdefault(instance, [])
.... self.func = func
.... self.instance = instance
.... self.observers = observers
.... def __get__(self, obj, cls=None):
.... if obj is None:
.... return self
.... else:
.... func = self.func.__get__(obj, cls)
.... return Observable(func, obj)
.... def __call__(self, *args, **kwargs):
.... result = self.func(*args, **kwargs)
.... for observer in self.observers:
.... observer(self.instance)
.... return result
.... def add_callback(self, callback):
.... self.observers.append(callback)
........ @Observable
.... def meth(self, foo):
.... print foo
........ print "Yippie, I've been called on", instance
.... instance.bar = True
....boring old argument
boring old argument


STeVe

 

[Wildemar Wildenburger]

... def __get__(*args): return object()
...

... meth = Descriptor()
...

False

Of course ...
Man, this language is breaking my head, seriously! &->


/W

 

[Steven Bethard]

... def __init__(self, func, instance=None, observers=None):
... self.func = func
... self.instance = instance
... self.observers = observers or []

Unless you also changed code in __get__, this means you'll get a new
list every time you access the "meth" attribute since the __get__ method
is called anew for every attribute access::
.... @Observable
.... def meth(self, foo):
.... print foo
........ print "Yippie, I've been called on", instance
.... instance.bar = True
....a1
a1

See my other post for how to get instance-level observer lists using a
WeakKeyDictionary.

STeVe

 

[Wildemar Wildenburger]

class Observable(object):

... def __init__(self, func, instance=None, observers=None):
... self.func = func
... self.instance = instance
... self.observers = observers or []

Unless you also changed code in __get__, this means you'll get a new
list every time you access the "meth" attribute since the __get__ method
is called anew for every attribute access::

Man! Nothing is easy ...



/W