decorator and API

[Lee Harr]

I have a class with certain methods from which I want to select
one at random, with weighting.

The way I have done it is this ....



import random

def weight(value):
def set_weight(method):
method.weight = value
return method
return set_weight

class A(object):
def actions(self):
'return a list of possible actions'

return [getattr(self, method)
for method in dir(self)
if method.startswith('action_')]

def action(self):
'Select a possible action using weighted choice'

actions = self.actions()
weights = [method.weight for method in actions]
total = sum(weights)

choice = random.randrange(total)

while choice> weights[0]:
choice -= weights[0]
weights.pop(0)
actions.pop(0)

return actions[0]


@weight(10)
def action_1(self):
print "A.action_1"

@weight(20)
def action_2(self):
print "A.action_2"


a = A()
a.action()()




The problem I have now is that if I subclass A and want to
change the weighting of one of the methods, I am not sure
how to do that.

One idea I had was to override the method using the new
weight in the decorator, and then call the original method:

class B(A):
@weight(50)
def action_1(self):
A.action_1(self)


That works, but it feels messy.


Another idea was to store the weightings as a dictionary
on each instance, but I could not see how to update that
from a decorator.

I like the idea of having the weights in a dictionary, so I
am looking for a better API, or a way to re-weight the
methods using a decorator.

Any suggestions appreciated.

_________________________________________________________________
Explore the seven wonders of the world
http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=QBRE

 

[Aaron \Castironpi\ Brady]

I have a class with certain methods from which I want to select
one at random, with weighting.

The way I have done it is this ....

import random

def weight(value):
    def set_weight(method):
        method.weight = value
        return method
    return set_weight

class A(object):
    def actions(self):
        'return a list of possible actions'

        return [getattr(self, method)
                    for method in dir(self)
                    if method.startswith('action_')]

    def action(self):
        'Select a possible action using weighted choice'

        actions = self.actions()
        weights = [method.weight for method in actions]
        total = sum(weights)

        choice = random.randrange(total)

        while choice> weights[0]:
            choice -= weights[0]
            weights.pop(0)
            actions.pop(0)

        return actions[0]

    @weight(10)
    def action_1(self):
        print "A.action_1"

    @weight(20)
    def action_2(self):
        print "A.action_2"

a = A()
a.action()()

The problem I have now is that if I subclass A and want to
change the weighting of one of the methods, I am not sure
how to do that.

One idea I had was to override the method using the new
weight in the decorator, and then call the original method:

class B(A):
    @weight(50)
    def action_1(self):
        A.action_1(self)

That works, but it feels messy.

Another idea was to store the weightings as a dictionary
on each instance, but I could not see how to update that
from a decorator.

I like the idea of having the weights in a dictionary, so I
am looking for a better API, or a way to re-weight the
methods using a decorator.

Any suggestions appreciated.

_________________________________________________________________
Explore the seven wonders of the worldhttp://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=QBRE

What about a function, 'reweight', which wraps the original, and sets
a weight on the wrapper?

def reweight(value):
def reset_weight(method):
#@wraps(method) #optional
def new_method( *ar, **kw ):
return method( *ar, **kw )
new_method.weight = value
return new_method
return reset_weight

Call like this:

class B(A):
action_1= reweight( 50 )( A.action_1 )

You could pass them both in to reweight with two parameters:

class B(A):
action_1= reweight( 50, A.action_1 )

It's about the same. Variable-signature functions have limits.

Otherwise, you can keep dictionaries by name, and checking membership
in them in superclasses that have them by hand. Then you'd need a
consistent name for the dictionary. That option looks like this
(unproduced):

class A:
__weights__= {}
@weight( __weights__, 10 ) ...
@weight( __weights__, 20 ) ...

class B( A ):
__weights__= {} #new instance so you don't change the original
@weight( __weights__, 50 ) ...

B.__weight__ could be an object that knows what it's overriding
(unproduced):

class A:
weights= WeightOb() #just a dictionary, mostly
@weights( 10 ) ...
@weights( 20 ) ...

class B( A ):
weights= WeightOb( A.weights ) #new, refs "super-member"
@weights( 50 ) ...

If either of the last two options look promising, I think I can
produce the WeightOb class. It has a '__call__' method.

 

[Aaron \Castironpi\ Brady]

(snip)







class A:
   weights= WeightOb() #just a dictionary, mostly
   @weights( 10 ) ...
   @weights( 20 ) ...

class B( A ):
   weights= WeightOb( A.weights ) #new, refs "super-member"
   @weights( 50 ) ...

Lee,

Probably overkill. Here's a solution like above.

class WeightOb( object ):
def __init__( self, *supers ):
self.weights= {}
self.supers= supers
def set( self, weight ):
def __callset__( fun ):
self.weights[ fun.func_name ]= weight
return fun
return __callset__
def reset( self, weight, fun ):
self.weights[ fun.func_name ]= weight
return fun
#search parent 'weight' objects
#return 'child-most' weight of 'name'
def get_weight( self, name ):
if name in self.weights:
return self.weights[ name ]
else:
for x in self.supers:
try:
return x.get_weight( name )
except KeyError: #not found
pass
raise KeyError
#returns a dictionary mapping bound instances to weights
#(hence the second parameter)
def contents( self, inst ):
d= {}
for x in reversed( self.supers ):
d.update( x.contents( inst ) )
d.update( dict( [
( getattr( inst, k ), v ) for k, v in self.weights.iteritems( ) ] ) )
return d


class A( object ):
weights= WeightOb( )
@weights.set( 10 )
def action_1( self ):
print 'action_1'
@weights.set( 20 )
def action_2( self ):
print 'action_2'
#WeightOb.contents needs to know which instance to bind
#functions to. Get weights from an instance that has them.
def getweights( self ):
return self.weights.contents( self )

class B( A ):
weights= WeightOb( A.weights )
action_2= weights.reset( 50, A.action_2 )

a= A()
b= B()
print a.weights.get_weight( 'action_1' )
print a.weights.get_weight( 'action_2' )
print b.weights.get_weight( 'action_1' )
print b.weights.get_weight( 'action_2' )
print a.getweights( )
print b.getweights( )


/Output:

10
20
10
50
{<bound method A.action_2 of <__main__.A object at 0x00A04070>>: 20,
<bound meth
od A.action_1 of <__main__.A object at 0x00A04070>>: 10}
{<bound method B.action_2 of <__main__.B object at 0x00A04090>>: 50,
<bound meth
od B.action_1 of <__main__.B object at 0x00A04090>>: 10}

 

[Steven D'Aprano]

I have a class with certain methods from which I want to select one at
random, with weighting.

The way I have done it is this ....

[snip]


You are coupling the weights, the actions, and the object which chooses
an action all in the one object. I find that harder to wrap my brain
around than a more loosely coupled system. Make the chooser independent
of the things being chosen:


def choose_with_weighting(actions, weights=None):
if weights is None:
weights = [1]*len(actions) # equal weights
# Taken virtually unchanged from your code.
# I hope it does what you want it to do!
assert len(weights) == len(actions)
total = sum(weights)
choice = random.randrange(total)
while choice > weights[0]:
choice -= weights[0]
weights.pop(0)
actions.pop(0)
return actions[0]


Loosely couple the actions from their weightings, so you can change them
independently. Here's a decorator that populates a dictionary with
weights and actions:

def weight(value, storage):
def set_weight(method):
storage[method.__name__] = value
return method
return set_weight


Here's how to use it:

class A(object):
weights = {}
def __init__(self):
self.weights = self.__class__.weights.copy()
@weight(10, weights)
def action_1(self):
print "A.action_1"
@weight(20, weights)
def action_2(self):
print "A.action_2"


The class is now populated with a set of default weights, which is then
copied to the instance. If you want to over-ride a particular weight, you
don't need to make a subclass, you just change the instance:

obj = A()
obj.weights["action_1"] = 30

method = choose_with_weighting(obj.weights.keys(), obj.weights.values())
getattr(obj, method)() # call the method



Hope this helps,

 

[George Sakkis]

I have a class with certain methods from which I want to select
one at random, with weighting.

The way I have done it is this ....

import random

def weight(value):
def set_weight(method):
method.weight = value
return method
return set_weight

class A(object):
def actions(self):
'return a list of possible actions'

return [getattr(self, method)
for method in dir(self)
if method.startswith('action_')]

def action(self):
'Select a possible action using weighted choice'

actions = self.actions()
weights = [method.weight for method in actions]
total = sum(weights)

choice = random.randrange(total)

while choice> weights[0]:
choice -= weights[0]
weights.pop(0)
actions.pop(0)

return actions[0]

@weight(10)
def action_1(self):
print "A.action_1"

@weight(20)
def action_2(self):
print "A.action_2"

a = A()
a.action()()

The problem I have now is that if I subclass A and want to
change the weighting of one of the methods, I am not sure
how to do that.

One idea I had was to override the method using the new
weight in the decorator, and then call the original method:

class B(A):
@weight(50)
def action_1(self):
A.action_1(self)

That works, but it feels messy.

Another idea was to store the weightings as a dictionary
on each instance, but I could not see how to update that
from a decorator.

I like the idea of having the weights in a dictionary, so I
am looking for a better API, or a way to re-weight the
methods using a decorator.

Any suggestions appreciated.

Below is a lightweight solution that uses a descriptor. Also the
random action function has been rewritten more efficiently (using
bisect).

George

#======== usage ===========================

class A(object):

# actions don't have to follow a naming convention

@weighted_action(weight=4)
def foo(self):
print "A.foo"

@weighted_action() # default weight=1
def bar(self):
print "A.bar"


class B(A):
# explicit copy of each action with new weight
foo = A.foo.copy(weight=2)
bar = A.bar.copy(weight=4)

@weighted_action(weight=3)
def baz(self):
print "B.baz"

# equivalent to B, but update all weights at once in one statement
class B2(A):
@weighted_action(weight=3)
def baz(self):
print "B2.baz"
update_weights(B2, foo=2, bar=4)


if __name__ == '__main__':
for obj in A,B,B2:
print obj
for action in iter_weighted_actions(obj):
print ' ', action

a = A()
for i in xrange(10): take_random_action(a)
print
b = B()
for i in xrange(12): take_random_action(b)

#====== implementation =======================

class _WeightedActionDescriptor(object):
def __init__(self, func, weight):
self._func = func
self.weight = weight
def __get__(self, obj, objtype):
return self
def __call__(self, *args, **kwds):
return self._func(*args, **kwds)
def copy(self, weight):
return self.__class__(self._func, weight)
def __str__(self):
return 'WeightedAction(%s, weight=%s)' % (self._func,
self.weight)

def weighted_action(weight=1):
return lambda func: _WeightedActionDescriptor(func,weight)

def update_weights(obj, **name2weight):
for name,weight in name2weight.iteritems():
action = getattr(obj,name)
assert isinstance(action,_WeightedActionDescriptor)
setattr(obj, name, action.copy(weight))

def iter_weighted_actions(obj):
return (attr for attr in
(getattr(obj, name) for name in dir(obj))
if isinstance(attr, _WeightedActionDescriptor))

def take_random_action(obj):
from random import random
from bisect import bisect
actions = list(iter_weighted_actions(obj))
weights = [action.weight for action in actions]
total = float(sum(weights))
cum_norm_weights = [0.0]*len(weights)
for i in xrange(len(weights)):
cum_norm_weights = cum_norm_weights[i-1] + weights/total
return actions[bisect(cum_norm_weights, random())](obj)

 

[Peter Otten]

Steven D'Aprano wrote:

I agree with you that the simple explicit approach is better.
Now, to answer the question the OP didn't ask:

def choose_with_weighting(actions, weights=None):
    if weights is None:
        weights = [1]*len(actions)  # equal weights
    # Taken virtually unchanged from your code.
    # I hope it does what you want it to do!

It probably doesn't.

    assert len(weights) == len(actions)
    total = sum(weights)
    choice = random.randrange(total)
    while choice > weights[0]:
        choice -= weights[0]
        weights.pop(0)
        actions.pop(0)
    return actions[0]

Assume two actions with equal weights [1, 1]. total becomes 2, and choice is
either 0 or 1, but never > weights[0].

While this can be fixed by changing the while condition to

while choice >= weights[0]: #...

I prefer an approach that doesn't destroy the actions and weights lists,
something like

import bisect

def choose_with_weighting(actions, weights=None, acc_weights=None):
if acc_weights is None:
if weights is None:
return random.choice(actions)
else:
sigma = 0
acc_weights = []
for w in weights:
sigma += w
acc_weights.append(sigma)
return actions[bisect.bisect(acc_weights,
random.randrange(acc_weights[-1]))]

especially if you prepare the acc_weights list once outside the function.

Peter