Re: properties + types, implementing meta-class desciptors elegantly?

Discussion in 'Python' started by Michele Simionato, Jul 19, 2003.

  1. (Bengt Richter) wrote in message news:<bfa97g$s5j$0@216.39.172.122>...
    > Not sure exactly what you are doing, but what does
    >
    > object.__setattr__(client, self.name, value)
    >
    > do in your context in place of
    >
    > client.__dict__[ self.name ] = value
    >
    > ?
    >
    > Regards,
    > Bengt Richter


    I agree with Bengt, from your traceback it seems you are assigning directly
    to client.__dict__, but you cannot do that (I think because client.__dict__
    is a dictproxy object and not a real dictionary). The right way to
    go is via object.__setattr__ , or type.__setattr__ in the case of
    metaclasses.

    I guess you are aware of the metaclass+properties recipes in the
    on-line cookbook, but just in case ...

    HTH,


    Michele
    Michele Simionato, Jul 19, 2003
    #1
    1. Advertising

  2. Michele Simionato wrote:
    ....

    >I agree with Bengt, from your traceback it seems you are assigning directly
    >to client.__dict__, but you cannot do that (I think because client.__dict__
    >is a dictproxy object and not a real dictionary). The right way to
    >go is via object.__setattr__ , or type.__setattr__ in the case of
    >metaclasses.
    >

    Try this to see what I'm getting at:

    >>> class o(object):

    .... class p( object ):
    .... def __set__( self, client, value, *args, **named ):
    .... print '__set__', self, client, value, args, named
    .... # now what do you do here to set without triggering
    ourselves
    .... # (without having to code diff versions of the
    descriptor class
    .... # for each possible type of client object (and for that
    matter, just
    .... # making it possible for (meta-)types with properties
    w/out requiring
    .... # e.g. a new dict object in each such serviced type))?.
    .... return object.__setattr__( client, "v", value)
    .... v = p()
    ....
    >>> s = o()
    >>> s.v = 3


    You'll notice you go into an infinite loop. What I'm looking for
    (really a thinly veiled attempt to get other people to demand it from
    Guido so he doesn't think I'm a lone crackpot ;) ) is a function/method
    that provides the default implementation of the get/set functionality
    *below* the level of the descriptor hooks, but high enough that it can
    deal with the differences between classes, types, instances, and
    instances-with-__slots__ (I realise that last one probably isn't going
    to work, BTW).

    Thought of another way, it's asking for a superclass of descriptors
    which provides this logic in such a way that, by sub-classing from them,
    you can readily gain access to this descriptor-building functionality
    without needing to always use it, and without introducing unwanted
    restrictions (such as requiring a property-class initialisation).

    Thought of another way, it's asking for a further rationalisation of the
    get/set hooks so that there's a lower level at which you can either
    insert storage code *or* use the default storage code.

    >I guess you are aware of the metaclass+properties recipes in the
    >on-line cookbook, but just in case ...
    >

    Hadn't looked at them, have now, there's nothing which addresses this
    particular issue that I can see with a search for metaclass. Willing to
    be proved wrong :) .

    Have fun,
    Mike

    _______________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://members.rogers.com/mcfletch/
    Mike C. Fletcher, Jul 19, 2003
    #2
    1. Advertising

  3. On Sat, 19 Jul 2003 11:49:25 -0400, "Mike C. Fletcher" <> wrote:

    >Michele Simionato wrote:
    >...
    >
    >>I agree with Bengt, from your traceback it seems you are assigning directly
    >>to client.__dict__, but you cannot do that (I think because client.__dict__
    >>is a dictproxy object and not a real dictionary). The right way to
    >>go is via object.__setattr__ , or type.__setattr__ in the case of
    >>metaclasses.
    >>

    >Try this to see what I'm getting at:
    >
    > >>> class o(object):

    >... class p( object ):
    >... def __set__( self, client, value, *args, **named ):
    >... print '__set__', self, client, value, args, named
    >... # now what do you do here to set without triggering
    >ourselves
    >... # (without having to code diff versions of the
    >descriptor class
    >... # for each possible type of client object (and for that
    >matter, just
    >... # making it possible for (meta-)types with properties
    >w/out requiring
    >... # e.g. a new dict object in each such serviced type))?.
    >... return object.__setattr__( client, "v", value)

    client.__dict__['v'] = value
    the above line should work for this example, so it must be different from
    what you were doing before? Perhaps the client object before was a class?
    I guess I'm not getting the big picture yet of your design intent.

    Are you trying to make a base class that has properties that can
    set same-named properties in the subclass via attributes
    of objects instantiated from the subclass? Or as attributes of the
    subclass? Do you need the property ability to munge its arguments dynamically?
    Otherwise a plain old attribute assignment to the subclass per se should install
    the property for its instances, no?

    >... v = p()
    >...
    > >>> s = o()
    > >>> s.v = 3

    >
    >You'll notice you go into an infinite loop. What I'm looking for

    For this example that's certainly what you'd expect.
    >(really a thinly veiled attempt to get other people to demand it from
    >Guido so he doesn't think I'm a lone crackpot ;) ) is a function/method
    >that provides the default implementation of the get/set functionality
    >*below* the level of the descriptor hooks, but high enough that it can
    >deal with the differences between classes, types, instances, and
    >instances-with-__slots__ (I realise that last one probably isn't going
    >to work, BTW).
    >
    >Thought of another way, it's asking for a superclass of descriptors
    >which provides this logic in such a way that, by sub-classing from them,
    >you can readily gain access to this descriptor-building functionality
    >without needing to always use it, and without introducing unwanted
    >restrictions (such as requiring a property-class initialisation).
    >
    >Thought of another way, it's asking for a further rationalisation of the
    >get/set hooks so that there's a lower level at which you can either
    >insert storage code *or* use the default storage code.
    >

    More simple examples like the above might help me understand, but I haven't
    got the picture yet, I am afraid ;-)

    Regards,
    Bengt Richter
    Bengt Richter, Jul 19, 2003
    #3
  4. Okay, rather than respond to the fragments first, which just seems to be
    confusing people, as they aren't getting the previous discussion.
    Here's a quick sketch of what I'm creating:

    class Meta( type ):
    tableName = common.StringProperty(
    "tableName", """The table in which this class stores it's primary
    data""",
    defaultValue = "",
    )
    implementationBase = common.ClassByNameProperty(
    "implementation", """Base class for implementation""",
    defaultValue = "wxpytable.dbresultset.DBResultSet",
    setDefaultOnGet = 0,
    )
    ...

    Now, those objects from "common" are part of a fairly involved hierarchy
    of descriptor classes which provide type coercion, default-value
    retrieval, automated lookup of factory functions, managed list-of-type
    support, and a dozen other features. Here's the framework of what a
    "basicproperty" does:

    * hooks the __set__ event (and __get__ and __delete__)
    o is thereby a descriptor
    o can be introspected from the object with the properties
    + has documentation
    + has meta-data for coercion, type-checking, default
    values, operations for choosing common values,
    etceteras, etceteras
    + can be readily sub-classed to produce new effects,
    such as using weak references or storing values in a
    database based on schema objects, or calling methods
    on a client...
    * checks the data type of the value
    o if necessary, coerces the value (normally defering to the
    "baseType" for the property)
    * finally, stores the value
    o tries to do what would have been done if there were no
    descriptor (with the new, coerced value)
    o does *not* create new names in the object's namespace (all
    names are documented w/ descriptors, there's not a lot of
    '_' prefixed names cluttering the namespace)
    o does *not* require a new dictionary/storage-object attribute
    for the object (the descriptor works like any other
    descriptor, a *stand-alone* object that replaces a regular
    attribute)

    It's that last part that I'd like to have a function/method to
    accomplish. That is, store (and obviously retrieve as well) attribute
    values for objects, instances, types, classes, __slot__'d instances,
    etceteras under a given name without triggering any of the __setattr__
    machinery which defers to the __set__ method of the associated descriptor.

    I can work around the problem by violating either of those two bullet
    points under "finally, stores the value", but I'm looking for something
    that *doesn't* violate them. See below for responses to particular
    points...

    Bengt Richter wrote:
    ....

    > client.__dict__['v'] = value
    >the above line should work for this example, so it must be different from
    >what you were doing before? Perhaps the client object before was a class?
    >I guess I'm not getting the big picture yet of your design intent.
    >

    That's what the base properties do, but they just can't work when doing
    a meta-class, as it's __dict__ is a dict-proxy, which doesn't allow
    assignment. I'm looking for a method to tell the Python system "Okay,
    what would you have done if there were no descriptors? Here's the key
    and value to set, go set them." That sample was just to demonstrate why
    calling object.__setattr__ wouldn't work (it recursively calls the
    descriptor).

    >Are you trying to make a base class that has properties that can
    >set same-named properties in the subclass via attributes
    >of objects instantiated from the subclass? Or as attributes of the
    >subclass? Do you need the property ability to munge its arguments dynamically?
    >Otherwise a plain old attribute assignment to the subclass per se should install
    >the property for its instances, no?
    >
    >

    Nope, I'm trying to make meta-classes which have rich properties. The
    particular project is a plug-in system, where the classes (the
    metaclass-instances) want to have all sorts of rich metadata associated
    with them in a way which meshes with the rest of the system (i.e. using
    a descriptor-based introspection mechanism).
    ....

    Enjoy,
    Mike
    _______________________________________

    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://members.rogers.com/mcfletch/
    Mike C. Fletcher, Jul 19, 2003
    #4
  5. Michele Simionato

    Aahz Guest

    In article <>,
    Mike C. Fletcher <> wrote:
    >
    >Nope, I'm trying to make meta-classes which have rich properties. The
    >particular project is a plug-in system, where the classes (the
    >metaclass-instances) want to have all sorts of rich metadata associated
    >with them in a way which meshes with the rest of the system (i.e. using
    >a descriptor-based introspection mechanism).
    >...


    Keeping in mind that I'm really not following this discussion all that
    closely (because metaclasses give me headaches), are you trying to make
    a *metaclass* that has properties or a *metaclass instance* (i.e. a
    class) that has properties?
    --
    Aahz () <*> http://www.pythoncraft.com/

    This is Python. We don't care much about theory, except where it intersects
    with useful practice. --Aahz
    Aahz, Jul 20, 2003
    #5
  6. Aahz wrote:

    >In article <>,
    >Mike C. Fletcher <> wrote:
    >
    >
    >>Nope, I'm trying to make meta-classes which have rich properties. The
    >>particular project is a plug-in system, where the classes (the
    >>metaclass-instances) want to have all sorts of rich metadata associated
    >>with them in a way which meshes with the rest of the system (i.e. using
    >>a descriptor-based introspection mechanism).
    >>...
    >>
    >>

    >
    >Keeping in mind that I'm really not following this discussion all that
    >closely (because metaclasses give me headaches), are you trying to make
    >a *metaclass* that has properties or a *metaclass instance* (i.e. a
    >class) that has properties?
    >
    >

    Just the meta-class instances (classes). Wasn't sufficiently precise in
    my description, the properties are normally declared in the meta-class,
    but they are actually properties of the meta-class instances.

    Sorry about that,
    Mike

    _______________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://members.rogers.com/mcfletch/
    Mike C. Fletcher, Jul 20, 2003
    #6
  7. Michele Simionato

    Aahz Guest

    In article <>,
    Mike C. Fletcher <> wrote:
    >Aahz wrote:
    >>In article <>,
    >>Mike C. Fletcher <> wrote:
    >>>
    >>>Nope, I'm trying to make meta-classes which have rich properties. The
    >>>particular project is a plug-in system, where the classes (the
    >>>metaclass-instances) want to have all sorts of rich metadata associated
    >>>with them in a way which meshes with the rest of the system (i.e. using
    >>>a descriptor-based introspection mechanism).

    >>
    >>
    >>Keeping in mind that I'm really not following this discussion all that
    >>closely (because metaclasses give me headaches), are you trying to make
    >>a *metaclass* that has properties or a *metaclass instance* (i.e. a
    >>class) that has properties?

    >
    >Just the meta-class instances (classes). Wasn't sufficiently precise in
    >my description, the properties are normally declared in the meta-class,
    >but they are actually properties of the meta-class instances.


    Next question: are you also trying to access the properties on class
    instances?
    --
    Aahz () <*> http://www.pythoncraft.com/

    This is Python. We don't care much about theory, except where it intersects
    with useful practice. --Aahz
    Aahz, Jul 20, 2003
    #7
  8. Mike C. Fletcher wrote on 2003-07-19:

    > * finally, stores the value
    > o tries to do what would have been done if there were no
    > descriptor (with the new, coerced value)
    > o does *not* create new names in the object's namespace (all
    > names are documented w/ descriptors, there's not a lot of
    > '_' prefixed names cluttering the namespace)
    > o does *not* require a new dictionary/storage-object attribute
    > for the object (the descriptor works like any other
    > descriptor, a *stand-alone* object that replaces a regular
    > attribute)
    >
    > It's that last part that I'd like to have a function/method to
    > accomplish. That is, store (and obviously retrieve as well) attribute
    > values for objects, instances, types, classes, __slot__'d instances,
    > etceteras under a given name without triggering any of the __setattr__
    > machinery which defers to the __set__ method of the associated descriptor.
    >
    > I can work around the problem by violating either of those two bullet
    > points under "finally, stores the value", but I'm looking for something
    > that *doesn't* violate them. See below for responses to particular
    > points...
    >

    I didn't grasp your precise problem but I've been having some similar
    problems (without metaclasses however). Two tricks:

    - Insert a dummy class into you heirarchy between object (or whatever
    other superclass that doesn't have the properties yet and the class
    where you install the properties. Then you might be able to use
    this class to access the underliying class dict without being
    intercepted by the properties. [I recall some problems with
    properties vs. super, didn't follow them closely].

    - If you use __slots__, they are also implemented as properties. So
    if you define properties with the same name in the same class, you
    lose all access to the raw slots. The solution is similar: define
    __slots__ in a superclass, hide them with properties in a sublass.
    See http://www.technion.ac.il/~cben/python/streams.py for an
    example.

    HTH,
    Beni Cherniavsky <>
    Beni Cherniavsky, Jul 20, 2003
    #8
  9. Aahz wrote:

    >In article <>,
    >Mike C. Fletcher <> wrote:
    >
    >
    >>Aahz wrote:
    >>
    >>
    >>>In article <>,
    >>>Mike C. Fletcher <> wrote:
    >>>
    >>>
    >>>>Nope, I'm trying to make meta-classes which have rich properties. The
    >>>>particular project is a plug-in system, where the classes (the
    >>>>metaclass-instances) want to have all sorts of rich metadata associated
    >>>>with them in a way which meshes with the rest of the system (i.e. using
    >>>>a descriptor-based introspection mechanism).
    >>>>
    >>>>
    >>>Keeping in mind that I'm really not following this discussion all that
    >>>closely (because metaclasses give me headaches), are you trying to make
    >>>a *metaclass* that has properties or a *metaclass instance* (i.e. a
    >>>class) that has properties?
    >>>
    >>>

    >>Just the meta-class instances (classes). Wasn't sufficiently precise in
    >>my description, the properties are normally declared in the meta-class,
    >>but they are actually properties of the meta-class instances.
    >>
    >>

    >
    >Next question: are you also trying to access the properties on class
    >instances?
    >

    Not really, I want them to be properties of the class, and would prefer
    that they *not* show up in the instances. That said, I can deal with it
    if they do show up in the instances.

    Enjoy,
    Mike

    _______________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://members.rogers.com/mcfletch/
    Mike C. Fletcher, Jul 20, 2003
    #9
  10. Beni Cherniavsky wrote:

    >Mike C. Fletcher wrote on 2003-07-19:
    >
    >
    >
    >> * finally, stores the value
    >> o tries to do what would have been done if there were no
    >> descriptor (with the new, coerced value)
    >> o does *not* create new names in the object's namespace (all
    >> names are documented w/ descriptors, there's not a lot of
    >> '_' prefixed names cluttering the namespace)
    >> o does *not* require a new dictionary/storage-object attribute
    >> for the object (the descriptor works like any other
    >> descriptor, a *stand-alone* object that replaces a regular
    >> attribute)
    >>
    >>It's that last part that I'd like to have a function/method to
    >>accomplish. That is, store (and obviously retrieve as well) attribute
    >>values for objects, instances, types, classes, __slot__'d instances,
    >>etceteras under a given name without triggering any of the __setattr__
    >>machinery which defers to the __set__ method of the associated descriptor.
    >>
    >>I can work around the problem by violating either of those two bullet
    >>points under "finally, stores the value", but I'm looking for something
    >>that *doesn't* violate them. See below for responses to particular
    >>points...
    >>
    >>
    >>

    >I didn't grasp your precise problem but I've been having some similar
    >problems (without metaclasses however). Two tricks:
    >
    >- Insert a dummy class into you heirarchy between object (or whatever
    > other superclass that doesn't have the properties yet and the class
    > where you install the properties. Then you might be able to use
    > this class to access the underliying class dict without being
    > intercepted by the properties. [I recall some problems with
    > properties vs. super, didn't follow them closely].
    >

    This violates the "does *not* require a new dictionary/storage-object
    attribute" constraint. The super-class is just another way of spelling
    "special place to store data" within the class instance, albeit a nicely
    evil one :) . I'm looking for something that doesn't require modifying
    classes to be property-aware just to add the properties, that is, a
    stand-alone property object should be able to do this stuff w/out
    requiring that the client objects know about the properties at all.

    >- If you use __slots__, they are also implemented as properties. So
    > if you define properties with the same name in the same class, you
    > lose all access to the raw slots. The solution is similar: define
    > __slots__ in a superclass, hide them with properties in a sublass.
    > See http://www.technion.ac.il/~cben/python/streams.py for an
    > example.
    >

    Will look it up. Basically, I'm not too worried about slotted instances
    yet. I just think it should be part of whatever general solution Guido
    figures is best.

    Have fun,
    Mike

    _______________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://members.rogers.com/mcfletch/
    Mike C. Fletcher, Jul 20, 2003
    #10
  11. Michele Simionato

    Aahz Guest

    In article <>,
    Mike C. Fletcher <> wrote:
    >
    > * finally, stores the value
    > o tries to do what would have been done if there were no
    > descriptor (with the new, coerced value)
    > o does *not* create new names in the object's namespace (all
    > names are documented w/ descriptors, there's not a lot of
    > '_' prefixed names cluttering the namespace)
    > o does *not* require a new dictionary/storage-object attribute
    > for the object (the descriptor works like any other
    > descriptor, a *stand-alone* object that replaces a regular
    > attribute)


    But this is a recipe for name clashes. If you follow the first bullet,
    it's a normal attribute, but everything else says you want a property.
    Properties are themselves objects that are attached to names in an
    object. You can't have the same name bound to two different objects.
    --
    Aahz () <*> http://www.pythoncraft.com/

    This is Python. We don't care much about theory, except where it intersects
    with useful practice. --Aahz
    Aahz, Jul 20, 2003
    #11
  12. Michele Simionato wrote:

    >"Mike C. Fletcher" <> wrote in message news:<>...
    ><snip>
    >
    >As others before me, I am not sure of what you are trying to do
    >and why you are doing so. I use descriptors in this way:
    >
    >class o(object):
    > class p( object ): # descriptor class
    > def __set__( self, client, value, *args, **named ):
    > print '__set__', self, client, value, args, named
    > self.v=value
    > def __get__(self,obj,cls):
    > return self.v
    > v = p()
    >

    You are aware that you have just created a class-attribute, rather than
    an instance attribute? The property/descriptor object exists within the
    class 'o' namespace, not the instance namespace. Try creating two
    instances of your o class and setting v on both of them. There is one
    'p' instance "self" to which you are assigning/retrieving an attribute
    for all instances of o. Creating class variables is certainly a valid
    use for descriptors, but I'm not sure if that's really what you were
    trying to do.

    What I'm looking for, in terms of your code, is a method/function which
    would do the proper thing instead of using "self.v=value", (or rather
    client.__dict__['v'] = value (i.e. store the value in the client
    dictionary)) for all of the major built-in types, such as
    object-instances, classes (and object-instances with slots would be
    nice). To the best of my knowledge, such a function does not exist
    within Python; for instances, simply doing instance.__dict__[ key ] =
    value is sufficient, but classes do not have a method exposed AFAIK
    which allows an equivalent setting of a value *without* triggering the
    descriptor machinery for the given key.

    I'm creating a slightly more involved pattern, by the way:

    class plugin( type ):
    someHelperClass = common.ClassByNameProperty(
    "someHelperClass", """Documentation for this meta-property""",
    defaultValue = "some.package.module.ClassName",
    setDefaultOnGet = 0,
    )

    class MyWorkingPlugIn( myBaseImplementation ):
    __metaclass__ = plugin

    instance = MyWorkingPlugIn()

    Where the properties of the plugin meta-class are providing all sorts of
    services for the MyWorkingPlugIn class, such as allowing it to find
    "someHelperClass" related to the plug-in system while allowing that
    property to be set manually if desired, or automatically calculating a
    global identifier if the plug-in doesn't currently have a global
    identifier. The value is primarily that the meta-class will use exactly
    the same mechanisms as the rest of the system, and so will be readily
    dealt with by the property-based meta-application system.

    There are certainly other ways to get around the particular problem (I
    have already done that), I'm looking for the *elegant* solution to the
    *general* case.

    Enjoy,
    Mike

    _______________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://members.rogers.com/mcfletch/
    Mike C. Fletcher, Jul 20, 2003
    #12
  13. On 20 Jul 2003 09:39:51 -0400, (Aahz) wrote:

    >In article <>,
    >Mike C. Fletcher <> wrote:
    >>
    >> * finally, stores the value
    >> o tries to do what would have been done if there were no
    >> descriptor (with the new, coerced value)
    >> o does *not* create new names in the object's namespace (all
    >> names are documented w/ descriptors, there's not a lot of
    >> '_' prefixed names cluttering the namespace)
    >> o does *not* require a new dictionary/storage-object attribute
    >> for the object (the descriptor works like any other
    >> descriptor, a *stand-alone* object that replaces a regular
    >> attribute)

    >
    >But this is a recipe for name clashes. If you follow the first bullet,
    >it's a normal attribute, but everything else says you want a property.
    >Properties are themselves objects that are attached to names in an
    >object. You can't have the same name bound to two different objects.


    I was wondering if Mike was wanting to have class C(...) with property p
    (meaning the property object is an attribute of class C or a base), and
    wanting also to have the instance dict contain the data as c.p rather
    than, say, c._p.

    IOW, does he want a sneaky way to avoid seeing the underscored or otherwise
    modified names in the object, so as to have a clean dir(c)?

    Other than the problem of getting the class defined in a dynamically configurable way
    via metaclasses or other machinery, c.__dict__['p'] = value_from_property_fset should
    work, IWT. The code below does that, though a little more parameterized ;-)

    What I'm not getting is why a dict proxy is getting in the way, unless there is an
    attempt to use the property mechanism to store properties-as-values in a class-as-instance.

    If so, does he (continuing 3rd person since I'm replying to AAhz. Hi Mike ;-) really want
    all that dynamic meta-stuff to happen every time a class is instatiated, or should the
    classes-to-use be created once at some configuration time (maybe system start or other
    special times), and thereafter appear to be equivalent to plain valnilla manually written
    classes with properties?

    If the latter, some kind of factory module that manufactures configured classes with
    configured properties and puts them in its global name space to be used in apparently
    normal "import m; inst=m.C()" fashion would seem better than metaclass inheritance magic behind C.
    If there are other configuration times than first-import, the module could have a special
    factory functions for that, IWT (as in cpc.py below). The properties of the final
    configured/manufactured classes could of course act with any degree of dynamism desired
    when accessed as attributes of the final class instances.

    BTW, do I recall somewhere seeing that defining __dict__ as a slot induces creation of
    a writable dict, and could that be used to get a dict in place of proxy for the trouble spot?

    For Mike:

    What does the following not address (I assume you can rearrange things to your taste ;-)
    that you want to do? (InfoSource is my placeholder as an example for whatever dynamic
    configuration of the ultimate class you want to do).


    ====< cpc.py >=================================================================
    def getCustPropClass(infoSource):
    class MC_CustPropBase(type):
    """Metaclass to provide basic customized property methods and data"""
    def __new__(cls, name, bases, cdict):
    if name == 'CustPropBase': return type.__new__(cls, name, bases, cdict)
    cdict['__name__'] = name
    cdict['__doc__'] = infoSource[name, '__doc__']
    # was raw_input('Base Doc string for property %r: '%name)
    # let subclasses "underride" default and config
    if 'default' not in cdict:
    cdict['default'] = infoSource[name, 'default']
    # was raw_input('Base Default for property %r: '%name)
    if 'config' not in cdict:
    cdict['config'] = infoSource[name, 'config']
    # was raw_input('Base Config data for property %r: '%name)
    def __set__(self, client, val):
    client.__dict__[self.__name__] = val
    def __get__(self, client, ctype=None):
    if client is None: return self
    if self.__name__ not in client.__dict__: return self.default
    return client.__dict__[self.__name__]
    def __delete__(self, client):
    if self.__name__ not in client.__dict__:
    raise AttributeError, 'Property %r has no instance data to delete.' % self.__name__
    del client.__dict__[self.__name__]
    for method in '__set__ __get__ __delete__'.split():
    if method in cdict: continue # don't override
    cdict[method] = vars()[method]
    return type.__new__(cls, name, bases, cdict)

    class CustPropBase(object): __metaclass__ = MC_CustPropBase

    class MC_ConfProps(type):
    """
    Configurable property definitions to be instantiated in
    the instances of (this meta) class.
    """
    def __new__(cls, name, bases, cdict):
    docs = filter(None, [cdict.get('__doc__', '')])
    for k,v in MC_ConfProps.__dict__.items(): #walk prop defs in this class
    if isinstance(v, type) and (hasattr(v,'__set__') or hasattr(v, '__get__')):
    cdict[k] = v()
    docs.append('\n see also %s.%s.__doc__ for property %r'%(name, k, k))
    cdict['__doc__'] = ''.join(docs)
    return type.__new__(cls, name, bases, cdict)

    class p(CustPropBase):
    """property p direct doc string will be overwritten by CustPropBase"""
    default = infoSource['p','default']
    # was raw_input('Property p: "Underride" of Base default value: ')
    conf = infoSource['p','conf']
    # was raw_input('Property p: Special Config data: ')
    def __set__(self, client, val): # special set method differs from CustPropBase
    client.__dict__['p'] = (
    '(%r) modified w/ overridden base default(%r) data plus\n'
    'base config(%r) & special config(%r) data.' %
    ( val, self.default, self.config, self.conf))

    class q(CustPropBase): pass # totally default

    class r(CustPropBase): # totally default except read-only
    def __set__(self, client, val):
    raise AttributeError, 'Property %r is read-only' % self.__name__

    class ConfiguredProperties(object): __metaclass__ = MC_ConfProps

    class C(ConfiguredProperties):
    """Empty test class C inheriting from ConfiguredProperties"""
    return C

    class InteractiveInfo(object):
    def __getitem__(self, ixtup): # expect (propname, dataname) tuple as key
    return raw_input('Enter value for property %r attribute %r: '% ixtup) #(propname,dataname)

    testInfo = {
    ('p','__doc__'): '<p doc string>',
    ('p','default'): '<p default value>',
    ('p','config'): '<p config value>',
    ('p','conf'): '<p special conf value>',
    ('q','__doc__'): '<q doc string>',
    ('q','default'): ('<q default value>', (lambda x: 123456789),'(not just strings ;-)'),
    ('q','config'): '<q config value>',
    ('r','__doc__'): '<r doc string>',
    ('r','default'): '<r default value>',
    ('r','config'): '<r config value>',
    }

    def test():
    for info_source in (testInfo, InteractiveInfo()):
    C = getCustPropClass(info_source)
    c = C()
    print 'intitial p:', c.p
    print 'intitial q:', c.q
    print 'intitial r:', c.r
    print 'try to set them to 123 and retrieve'
    for prop in 'pqr':
    print 'setting %s to 123'%prop
    try:
    setattr(c, prop, 123)
    except Exception, e:
    print '%s: %s'% (e.__class__.__name__, e)
    print 'getting %s:\n----\n%s\n----' %(prop, getattr(c,prop,'???'))
    print 'try to delete them and retrieve'
    for prop in 'pqr':
    print 'deleting %s'%prop
    try:
    delattr(c, prop)
    except Exception, e:
    print '%s: %s'% (e.__class__.__name__, e)
    print 'after deleting %s:\n----\n%s\n----' %(prop, getattr(c,prop,'???'))
    print 'c.__doc__\n', c.__doc__
    for prop in 'pqr':
    print getattr(C, prop).__doc__

    if __name__ == '__main__':
    test()
    ===============================================================================
    An interactive run (with automatic test data first):

    [15:46] C:\pywk\clp\fletcher>cfc.py
    intitial p: <p default value>
    intitial q: ('<q default value>', <function <lambda> at 0x00800730>, '(not just strings ;-)')
    intitial r: <r default value>
    try to set them to 123 and retrieve
    setting p to 123
    getting p:
    ----
    (123) modified w/ overridden base default('<p default value>') data plus
    base config('<p config value>') & special config('<p special conf value>') data.
    ----
    setting q to 123
    getting q:
    ----
    123
    ----
    setting r to 123
    AttributeError: Property 'r' is read-only
    getting r:
    ----
    <r default value>
    ----
    try to delete them and retrieve
    deleting p
    after deleting p:
    ----
    <p default value>
    ----
    deleting q
    after deleting q:
    ----
    ('<q default value>', <function <lambda> at 0x00800730>, '(not just strings ;-)')
    ----
    deleting r
    AttributeError: Property 'r' has no instance data to delete.
    after deleting r:
    ----
    <r default value>
    ----
    c.__doc__
    Empty test class C inheriting from ConfiguredProperties
    see also C.q.__doc__ for property 'q'
    see also C.p.__doc__ for property 'p'
    see also C.r.__doc__ for property 'r'
    <p doc string>
    <q doc string>
    <r doc string>
    Enter value for property 'p' attribute 'default': [p default]
    Enter value for property 'p' attribute 'conf': [p conf]
    Enter value for property 'p' attribute '__doc__': [p doc]
    Enter value for property 'p' attribute 'config': [p config]
    Enter value for property 'q' attribute '__doc__': [q doc]
    Enter value for property 'q' attribute 'default': [q default]
    Enter value for property 'q' attribute 'config': [q config]
    Enter value for property 'r' attribute '__doc__': [r doc]
    Enter value for property 'r' attribute 'default': [r default]
    Enter value for property 'r' attribute 'config': [r config]
    intitial p: [p default]
    intitial q: [q default]
    intitial r: [r default]
    try to set them to 123 and retrieve
    setting p to 123
    getting p:
    ----
    (123) modified w/ overridden base default('[p default]') data plus
    base config('[p config]') & special config('[p conf]') data.
    ----
    setting q to 123
    getting q:
    ----
    123
    ----
    setting r to 123
    AttributeError: Property 'r' is read-only
    getting r:
    ----
    [r default]
    ----
    try to delete them and retrieve
    deleting p
    after deleting p:
    ----
    [p default]
    ----
    deleting q
    after deleting q:
    ----
    [q default]
    ----
    deleting r
    AttributeError: Property 'r' has no instance data to delete.
    after deleting r:
    ----
    [r default]
    ----
    c.__doc__
    Empty test class C inheriting from ConfiguredProperties
    see also C.q.__doc__ for property 'q'
    see also C.p.__doc__ for property 'p'
    see also C.r.__doc__ for property 'r'
    [p doc]
    [q doc]
    [r doc]

    Regards,
    Bengt Richter
    Bengt Richter, Jul 20, 2003
    #13
  14. "Mike C. Fletcher" <> wrote in message news:<>...
    > Michele Simionato wrote:
    >
    > >"Mike C. Fletcher" <> wrote in message news:<>...
    > ><snip>
    > >
    > >As others before me, I am not sure of what you are trying to do
    > >and why you are doing so. I use descriptors in this way:
    > >
    > >class o(object):
    > > class p( object ): # descriptor class
    > > def __set__( self, client, value, *args, **named ):
    > > print '__set__', self, client, value, args, named
    > > self.v=value
    > > def __get__(self,obj,cls):
    > > return self.v
    > > v = p()
    > >

    > You are aware that you have just created a class-attribute, rather than
    > an instance attribute? The property/descriptor object exists within the
    > class 'o' namespace, not the instance namespace. Try creating two
    > instances of your o class and setting v on both of them. There is one
    > 'p' instance "self" to which you are assigning/retrieving an attribute
    > for all instances of o. Creating class variables is certainly a valid
    > use for descriptors, but I'm not sure if that's really what you were
    > trying to do.


    I have just copied your code, I thought you wanted a class attribute.
    It is is trivial to do the same for instance attributes:

    class o(object):
    class p( object ): # descriptor class
    def __set__( self, client, value, *args, **named ):
    print '__set__', self, client, value, args, named
    self.value=value
    def __get__(self,obj,cls):
    return self.value

    def __init__(self):
    self.v = self.p()

    I understand that you aware of this, but you don't like it. Still I
    do not understand why do you feel it to be ugly and/or inelegant. It
    seems to me quite idiomatic.

    > What I'm looking for, in terms of your code, is a method/function which
    > would do the proper thing instead of using "self.v=value", (or rather
    > client.__dict__['v'] = value (i.e. store the value in the client
    > dictionary)) for all of the major built-in types, such as
    > object-instances, classes (and object-instances with slots would be
    > nice). To the best of my knowledge, such a function does not exist
    > within Python; for instances, simply doing instance.__dict__[ key ] =
    > value is sufficient, but classes do not have a method exposed AFAIK
    > which allows an equivalent setting of a value *without* triggering the
    > descriptor machinery for the given key.


    Yes, I understand you want to be able to set class dictionaries just
    as object dictionaries, bypassing descriptors. Still, I am not sure
    if this would be a good idea.

    > I'm creating a slightly more involved pattern, by the way:
    >
    > class plugin( type ):
    > someHelperClass = common.ClassByNameProperty(
    > "someHelperClass", """Documentation for this meta-property""",
    > defaultValue = "some.package.module.ClassName",
    > setDefaultOnGet = 0,
    > )
    >
    > class MyWorkingPlugIn( myBaseImplementation ):
    > __metaclass__ = plugin
    >
    > instance = MyWorkingPlugIn()
    >
    > Where the properties of the plugin meta-class are providing all sorts of
    > services for the MyWorkingPlugIn class, such as allowing it to find
    > "someHelperClass" related to the plug-in system while allowing that
    > property to be set manually if desired, or automatically calculating a
    > global identifier if the plug-in doesn't currently have a global
    > identifier. The value is primarily that the meta-class will use exactly
    > the same mechanisms as the rest of the system, and so will be readily
    > dealt with by the property-based meta-application system.
    >
    > There are certainly other ways to get around the particular problem (I
    > have already done that), I'm looking for the *elegant* solution to the
    > *general* case.
    >


    Elegance is in the eye of the beholder ;)

    > Enjoy,
    > Mike
    >
    > _______________________________________
    > Mike C. Fletcher
    > Designer, VR Plumber, Coder
    > http://members.rogers.com/mcfletch/


    Good luck with your project,

    Michele
    Michele Simionato, Jul 21, 2003
    #14
  15. Michele Simionato wrote:

    >"Mike C. Fletcher" <> wrote in message news:<>...
    >
    >

    ....

    >>You are aware that you have just created a class-attribute, rather than
    >>an instance attribute? The property/descriptor object exists within the
    >>class 'o' namespace, not the instance namespace. Try creating two
    >>instances of your o class and setting v on both of them. There is one
    >>'p' instance "self" to which you are assigning/retrieving an attribute
    >>for all instances of o. Creating class variables is certainly a valid
    >>use for descriptors, but I'm not sure if that's really what you were
    >>trying to do.
    >>
    >>

    >
    >I have just copied your code, I thought you wanted a class attribute.
    >

    I should have been more precise. Your value is stored in the
    descriptor, so it is not a per-class attribute, it is an attribute
    stored in one place for *all* instances of all sub-classes of the
    defined class, it is not a property of the class, but an effectively
    static variable in the class definition. That is, each instance
    modifies the one copy of the value, regardless of the instance' class.
    I'm looking for a meta-property, a property of a class, rather than a
    property of a class-instance. Changing an instance's value for that
    name shouldn't affect this value. Changing the meta-property value
    should only occur when you do setattr( cls, key, value ), and should
    store the value in the *particular* concrete class having the value set,
    not on some ancestor of that class, just as if you were to do setattr(
    cls, key, value) without any meta-descriptor being present.

    >It is is trivial to do the same for instance attributes:
    >
    >class o(object):
    > class p( object ): # descriptor class
    > def __set__( self, client, value, *args, **named ):
    > print '__set__', self, client, value, args, named
    > self.value=value
    > def __get__(self,obj,cls):
    > return self.value
    >
    > def __init__(self):
    > self.v = self.p()
    >
    >I understand that you aware of this, but you don't like it. Still I
    >do not understand why do you feel it to be ugly and/or inelegant. It
    >seems to me quite idiomatic.
    >
    >

    It's just not what I'm trying to do (create active properties on
    meta-classes). It's a perfectly fine way to create class-static
    variables shared among a hierarchy of instance objects.

    >Yes, I understand you want to be able to set class dictionaries just
    >as object dictionaries, bypassing descriptors. Still, I am not sure
    >if this would be a good idea.
    >

    Well, just to clean up the descriptor API it's probably worth it IMO,
    but I only seem to have Michael to back me up on that so far :) .
    There's a missing level of functionality/hooks that haven't been exposed
    and should be exposed to allow more general & orthogonal operation of
    descriptor code.

    Basically, it should be possible to mimic the built-in behaviour of
    setattr/getattr w/out resorting to horrific hacks just to get that
    behaviour. In other words, it should be possible to interpose an
    operation in the descriptor-processing stream in much the same way that
    super() allows for co-operative mix-in classes to inter-operate by
    saying 'okay, do what would have been done if this "filter" method
    didn't exist'.

    >Elegance is in the eye of the beholder ;)
    >

    Indeed.

    >Good luck with your project,
    >

    Thanks. Enjoy,
    Mike
    Mike C. Fletcher, Jul 22, 2003
    #15
  16. Michele Simionato

    Aahz Guest

    In article <>,
    Mike C. Fletcher <> wrote:
    >Aahz wrote:
    >>In article <>,
    >>Mike C. Fletcher <> wrote:
    >>>
    >>> * finally, stores the value
    >>> o tries to do what would have been done if there were no
    >>> descriptor (with the new, coerced value)
    >>> o does *not* create new names in the object's namespace (all
    >>> names are documented w/ descriptors, there's not a lot of
    >>> '_' prefixed names cluttering the namespace)
    >>> o does *not* require a new dictionary/storage-object attribute
    >>> for the object (the descriptor works like any other
    >>> descriptor, a *stand-alone* object that replaces a regular
    >>> attribute)

    >>
    >>But this is a recipe for name clashes. If you follow the first bullet,
    >>it's a normal attribute, but everything else says you want a property.
    >>Properties are themselves objects that are attached to names in an
    >>object. You can't have the same name bound to two different objects.

    >
    >It's true that:
    >
    > * your class has a namespace, and there are objects stored in that
    > namespace
    > o the objects stored in that namespace often are descriptors
    > o they are available to the instances which do not shadow them
    > (as I said, I can deal with this, and it's cleaner anyway).
    > + if they are descriptors, they modify instance access
    > to attributes


    That's technically inaccurate as phrased. First of all, not all
    descriptors get activated through attribute access, only data
    descriptors do. Data descriptors are descriptors that define a ``set``
    attribute (which is essentially a method of the descriptor object).
    Secondly, it muddies understanding to say that the descriptor modifies
    access to attributes, as if the descriptor is a proxy for an attribute.
    Data descriptors *replace* attributes, and if you want to store a value,
    it's your responsibility to determine where that value is stored (and
    how to retrieve it).

    > * the metaclass instance's property *values* would be stored in the
    > metaclass instance's dictionary, just as is normally done (the
    > metaclass' property *descriptors* would be stored in the
    > metaclass' dictionary), there's no new conflicts created here,
    > everything just works like it does now, a lookup would look
    > something like this:
    >
    > >>> x.y

    >x doesn't have a __getattribute__, so see if it has a descriptor for 'y'


    Actually, x is the *last* object checked, and __getattribute__() is
    never called on instances.

    > get type(x).y
    >
    > type(x) doesn't have a __getattribute__, so see if it has a
    > descriptor for 'y'
    >
    > get type(type(x)).y -> this is just a simple descriptor, as
    > there's no higher-level descriptor, has a __get__ method,
    > calls it to retrieve type(x).y


    That should be type(type(x).y), I think. (I'm not testing this in
    code.)

    > returns the currently stored entry 'y' in type(x)'s dictionary,
    > which happens to be a descriptor
    >
    > has type(x).y, a descriptor with a __get__ method, calls it, gets
    > back value stored in instance's dictionary


    That's correct if type(x).y is a data descriptor; other types of
    descriptors do get overridden by the instance.

    >You wind up with some names that are not available for use *as
    >properties* on instances *iff* you set a simple value for the properties
    >of the meta-class instance, but you solve that the regular way, by
    >making the value stored by the meta-class instance's property a
    >descriptor, rather than a simple value. That's just the way classes
    >work. Yes, it's a "name clash", but everything in classes/instances is
    >name clashes <shrug>.


    Instances don't have properties, only types do. A metaclass is "just" a
    class that creates classes (which are themselves types with new-style
    classes). Problems only show up when you want to access class behavior
    *through* a class instance.

    >Not a problem I can see in the problem I outlined just yet, there's
    >still no low-level value-setting mechanism exposed,


    The point is that there's no low-level value. You always have to
    explicitly specify the name and namespace when you use properties.
    --
    Aahz () <*> http://www.pythoncraft.com/

    This is Python. We don't care much about theory, except where it intersects
    with useful practice. --Aahz
    Aahz, Jul 26, 2003
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ken Fine
    Replies:
    0
    Views:
    327
    Ken Fine
    Jul 19, 2003
  2. Michael Hudson
    Replies:
    3
    Views:
    363
    Bengt Richter
    Jul 22, 2003
  3. Dave
    Replies:
    2
    Views:
    376
    Paul Rubin
    Aug 26, 2003
  4. Saqib Ali
    Replies:
    7
    Views:
    279
    Steve Holden
    Oct 14, 2004
  5. Lorenzo Chomp

    Unix file desciptors

    Lorenzo Chomp, Mar 15, 2006, in forum: Ruby
    Replies:
    0
    Views:
    83
    Lorenzo Chomp
    Mar 15, 2006
Loading...

Share This Page