Access to static members from inside a method decorator?

[glen.coates.bigworld]

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.

At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMethods(), a la:

class Exposed:
@staticmethod
def expose( f ):
...

def getExposedMethods( self ):
...

class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...

I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMethods() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there). The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.

So how can this be done? It doesn't seem like it's possible to pass a
reference to the enclosing class into the decorator, which in turn
means that static tracking of the list of exposed methods is impossible
(at least, if I want to use decorators).

Any ideas that will enable my initial design, or suggestions for an
elegant, workable alternative would be much appreciated.

Cheers,
Glen

 

[Bruno Desthuilliers]

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.

At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMethods(), a la:

class Exposed:
@staticmethod
def expose( f ):
...

def getExposedMethods( self ):
...

class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...

I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMethods() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there).

Not yet. When your decorator is called, the class object is not yet
created, and what you are decorating is a plain function.

The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.
Exactly.

So how can this be done?

The simplest thing is to use a two-stages scheme : mark the functions as
exposed, then collect them:

def expose(func):
func._exposed = True
return func

def exposed(obj):
return callable(obj) and getattr(obj, '_exposed', False)

class Exposing(object):
@classmethod
def get_exposed_methods(cls):
try:
exposeds = cls._exposed_methods
except AttributeError:
exposeds = []
for name in dir(cls):
obj = getattr(cls, name)
if exposed(obj):
exposeds.append(obj)
cls._exposed_methods = exposeds
return exposeds

class Parrot(Exposing):
@expose
def parrot(self, what):
return "%s says %s" % (self, str(what))



HTH

 

[glen.coates.bigworld]

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.

At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMethods(), a la:

class Exposed:
@staticmethod
def expose( f ):
...

def getExposedMethods( self ):
...

class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...

I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMethods() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there).

Not yet. When your decorator is called, the class object is not yet
created, and what you are decorating is a plain function.

The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.
Exactly.

So how can this be done?

The simplest thing is to use a two-stages scheme : mark the functions as
exposed, then collect them:

def expose(func):
func._exposed = True
return func

def exposed(obj):
return callable(obj) and getattr(obj, '_exposed', False)

class Exposing(object):
@classmethod
def get_exposed_methods(cls):
try:
exposeds = cls._exposed_methods
except AttributeError:
exposeds = []
for name in dir(cls):
obj = getattr(cls, name)
if exposed(obj):
exposeds.append(obj)
cls._exposed_methods = exposeds
return exposeds

class Parrot(Exposing):
@expose
def parrot(self, what):
return "%s says %s" % (self, str(what))



HTH

Thanks Bruno. I came up with a similar solution today at work, which
involves an 'init' method which is called at the bottom of each module
that defines subclasses of Exposed and sets up static mappings for the
exposed methods. I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

Thanks for the help though,
Glen

 

[Maric Michaud]

Le jeudi 05 octobre 2006 17:18, (e-mail address removed) a écrit :

I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

You can always replace the need of the init method on classes using a
metaclass.

This demonstrates it with Bruno's expose decorator, but it can be done with
your actual init func too.

In [6]: class m(type) :
...: def __init__(self, *a,**kw) :
...: for name, meth in self.__dict__.items() :
...: if getattr(meth, '_exposed', False) :
...: print 'exposed :', name
...:
...:

In [7]: class a(object):
...: __metaclass__ = m
...: def f(self) ass
...: @expose
...: def g(self) ass
...:
...:
exposed : g

In [8]: class b(a) :
...: @expose
...: def h(self) ass
...:
...:
exposed : h




--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Bruno Desthuilliers]

Bruno Desthuilliers wrote: (snip)

class Exposing(object):
@classmethod
def get_exposed_methods(cls):
try:
exposeds = cls._exposed_methods
except AttributeError:
exposeds = []
for name in dir(cls):
obj = getattr(cls, name)
if exposed(obj):
exposeds.append(obj)
cls._exposed_methods = exposeds
return exposeds

class Parrot(Exposing):
@expose
def parrot(self, what):
return "%s says %s" % (self, str(what))

Thanks Bruno. I came up with a similar solution today at work, which
involves an 'init' method which is called at the bottom of each module
that defines subclasses of Exposed and sets up static mappings for the
exposed methods. I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with,

This is only a problem in a framework-style use.

however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

I think you have missed a point in my solution : the 'collect' pass
happens only once for each class - the first time the
get_exposed_methods is called on the class (or any instance of). It's
then memoized for latter calls.

 

[Bruno Desthuilliers]

Le jeudi 05 octobre 2006 17:18, (e-mail address removed) a écrit :

I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

You can always replace the need of the init method on classes using a
metaclass.

This demonstrates it with Bruno's expose decorator, but it can be done with
your actual init func too.

In [6]: class m(type) :
...: def __init__(self, *a,**kw) :
...: for name, meth in self.__dict__.items() :

NB : This will only get methods exposeds in this class - not the one
exposeds in the parent classes...

...: if getattr(meth, '_exposed', False) :
...: print 'exposed :', name
...:
...:

(snip)

 

[Steve Holden]

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.

At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMethods(), a la:

class Exposed:
@staticmethod
def expose( f ):
...

def getExposedMethods( self ):
...

class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...

I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMethods() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there).

Not yet. When your decorator is called, the class object is not yet
created, and what you are decorating is a plain function.

The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.
Exactly.


So how can this be done?

The simplest thing is to use a two-stages scheme : mark the functions as
exposed, then collect them:

def expose(func):
func._exposed = True
return func

def exposed(obj):
return callable(obj) and getattr(obj, '_exposed', False)

class Exposing(object):
@classmethod
def get_exposed_methods(cls):
try:
exposeds = cls._exposed_methods
except AttributeError:
exposeds = []
for name in dir(cls):
obj = getattr(cls, name)
if exposed(obj):
exposeds.append(obj)
cls._exposed_methods = exposeds
return exposeds

class Parrot(Exposing):
@expose
def parrot(self, what):
return "%s says %s" % (self, str(what))



HTH

Thanks Bruno. I came up with a similar solution today at work, which
involves an 'init' method which is called at the bottom of each module
that defines subclasses of Exposed and sets up static mappings for the
exposed methods. I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

Surely the right place to handle "collection" is in a metaclass, where
the metaclass's __call__() method can scan the __dict__ and take
appropriate action on the marked methods? That way it's done just once,
at class definition time, as it should be.

regards
Steve

 

[glen.coates.bigworld]

Thanks for all the help guys ... in almost every way using a metaclass
seems to be the right solution for what I'm trying to do here. I say
almost because there is one thing that is still confusing me: what is
the most elegant way to provide base-class implementations of methods
that are expected to be overriden by some of the derived classes (or in
the case of metaclasses, the classes that either declare __metaclass__
= Exposed or are derived from such classes).

Here's what I've just knocked out:

import sys

class Exposed( type ):

def __init__( cls, *args, **kw ):

# Track marked exposed methods
cls.s_exposedMethods = []
for superclass in cls.__mro__:
for name, meth in superclass.__dict__.items():
if hasattr( meth, "exposed" ):
cls.s_exposedMethods.append( name )

# Add getExposedMethods method
cls.getExposedMethods = lambda self: self.s_exposedMethods
cls.bar = lambda self: sys.stdout.write( "bar\n" )

@staticmethod
def expose( f ):
f.exposed = True
return f


class Process( object ):

__metaclass__ = Exposed

@Exposed.expose
def foo( self ):
pass


def bar( self ):
print "BAR"
pass


class BotProcess( Process ):

@Exposed.expose
def addBots( self ):
pass

p = Process()
p.bar()

#############

The problem here is that the implementation of 'bar' inside
Exposed.__init__ overrides the implementation of bar() in Process,
which makes sense I guess seeing as Exposed.__init__() is called after
the class has been initialised. It's not a problem for
getExposedMethods() seeing as it's not overriden by any derived
classes.

So what is the accepted way of doing this? Do I need two Exposed
classes, one is the metaclass that handles all the static mapping
stuff, and another provides base implementations of methods and is what
is actually derived from? E.g.:

class ExposedMeta( type ):
...

class Exposed( object ):
...

class Process( Exposed ):
__metaclass__ = ExposedMeta
...

class BotProcess( Process ):
...

Hmmm ... that seems a bit ugly too. If I change the assignments in
Exposed.__init__() I guess I can avoid the two-class thing:

cls.getExposedMethods = getattr( cls, "getExposedMethods", lambda
self: self.s_exposedMethods )
cls.bar = getattr( cls, "bar", lambda self: sys.stdout.write( "bar\n"
) )

That's better, but still ugly. Is there a better way?

Thanks for all the help thus far guys,
Glen

Le jeudi 05 octobre 2006 17:18, (e-mail address removed) a écrit :

I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

You can always replace the need of the init method on classes using a
metaclass.

This demonstrates it with Bruno's expose decorator, but it can be done with
your actual init func too.

In [6]: class m(type) :
...: def __init__(self, *a,**kw) :
...: for name, meth in self.__dict__.items() :
...: if getattr(meth, '_exposed', False) :
...: print 'exposed :', name
...:
...:

In [7]: class a(object):
...: __metaclass__ = m
...: def f(self) ass
...: @expose
...: def g(self) ass
...:
...:
exposed : g

In [8]: class b(a) :
...: @expose
...: def h(self) ass
...:
...:
exposed : h




--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Peter Otten]

Thanks for all the help guys ... in almost every way using a metaclass
seems to be the right solution for what I'm trying to do here. I say
almost because there is one thing that is still confusing me: what is
the most elegant way to provide base-class implementations of methods
that are expected to be overriden by some of the derived classes (or in
the case of metaclasses, the classes that either declare __metaclass__
= Exposed or are derived from such classes).

Here's what I've just knocked out:

[snip example]

The problem here is that the implementation of 'bar' inside
Exposed.__init__ overrides the implementation of bar() in Process,
which makes sense I guess seeing as Exposed.__init__() is called after
the class has been initialised. It's not a problem for
getExposedMethods() seeing as it's not overriden by any derived
classes.

So what is the accepted way of doing this? Do I need two Exposed
classes, one is the metaclass that handles all the static mapping
stuff, and another provides base implementations of methods and is what
is actually derived from? E.g.:

You define one base type with a custom metaclass and inherit from that. Your
example then becomes:

import sys

class ExposedType( type ):
def __init__( cls, *args, **kw ):
# Track marked exposed methods
cls.s_exposedMethods = []
for superclass in cls.__mro__:
for name, meth in superclass.__dict__.items():
if hasattr( meth, "exposed" ):
cls.s_exposedMethods.append( name )

class Exposed:
__metaclass__ = ExposedType
def getExposedMethods(self):
return self.s_exposedMethods
def bar(self):
print "bar\n"
@staticmethod
def expose( f ):
f.exposed = True
return f

class Process( Exposed):
@Exposed.expose
def foo( self ):
pass
def bar( self ):
print "BAR"


class BotProcess( Process ):
@Exposed.expose
def addBots( self ):
pass

p = Process()
p.bar()

This prints "BAR" as expected.

Peter

 

[urielka]

no need for all that,i wrote a basic Ajax framework for cherrypy that
features a Ajax.Net feature,exposing functions to JavaScript via
attributes(or in python via decorators),here is a decorator that run
one time(i.e. before running the actual code) and get the name of the
class

def AddFunction(func):
        stack=inspect.stack()
        ClsName=stack[1][3]#get the class name from the stack
        #now you both have the ClassName and the func object,so you can
populate your list
        return func#return the decorated function

it use the module inspect,to inspect the stack.

 

[glen.coates.bigworld]

(e-mail address removed) wrote:

You define one base type with a custom metaclass and inherit from that. Your
example then becomes:

import sys

class ExposedType( type ):
def __init__( cls, *args, **kw ):
# Track marked exposed methods
cls.s_exposedMethods = []
for superclass in cls.__mro__:
for name, meth in superclass.__dict__.items():
if hasattr( meth, "exposed" ):
cls.s_exposedMethods.append( name )

class Exposed:
__metaclass__ = ExposedType
def getExposedMethods(self):
return self.s_exposedMethods
def bar(self):
print "bar\n"
@staticmethod
def expose( f ):
f.exposed = True
return f

class Process( Exposed):
@Exposed.expose
def foo( self ):
pass
def bar( self ):
print "BAR"


class BotProcess( Process ):
@Exposed.expose
def addBots( self ):
pass

p = Process()
p.bar()

This prints "BAR" as expected.

Peter

Thanks Peter. Yeah I had thought of that earlier, but wasn't sure if
this is a standard design pattern for what I'm trying to achieve. It
seems ugly to me to use 2 classes when you are essentially describing a
single type. Is the Splat/SplatType pairing really a common design
pattern when trying to use metaclasses in this way?

Also ... as for the 'inspect' based solution, yeah I can see that would
work, but it seems very hacky to me and my gut reaction is to avoid
that kind of thing ...

Cheers,
Glen

 

[Peter Otten]

Thanks Peter. Yeah I had thought of that earlier, but wasn't sure if
this is a standard design pattern for what I'm trying to achieve. It
seems ugly to me to use 2 classes when you are essentially describing a
single type.

To me both Exposed and ExposedType look more idiomatic than your combined
approach.

Is the Splat/SplatType pairing really a common design
pattern when trying to use metaclasses in this way?

I think so, but look for yourself:

http://www.google.com/codesearch?q=lang:python+__metaclass__&btnG=Search+Code

Refinements to that query welcome...

Peter