Descriptors and side effects

Discussion in 'Python' started by mrkafk@gmail.com, Nov 5, 2007.

  1. Guest

    Hello everyone,

    I'm trying to do seemingly trivial thing with descriptors: have
    another attribute updated on dot access in object defined using
    descriptors.

    For example, let's take a simple example where you set an attribute s
    to a string and have another attribute l set automatically to its
    length.

    >>> class Desc(str):

    def __init__(self,val):
    self.s=val
    self.l=len(val)
    print "creating value: ", self.s
    print "id(self.l)", id(self.l)
    def __set__(self, obj, val):
    self.s=val
    self.l=len(val)
    print "setting value:", self.s, "length:", self.l
    def __get__(self, obj, type=None):
    print "getting value:", self.s, "length:", self.l
    return self.l


    >>> class some(str):

    m=Desc('abc')
    l=m.l


    creating value: abc
    id(self.l) 10049688
    >>> ta=some()
    >>> ta.m='test string'

    setting value: test string length: 11

    However, the attribute ta.l didn't get updated:

    >>> ta.l

    3

    This is so much weirder that object id of ta.l is the same as id of
    instance of descriptor:

    >>> id(ta.l)

    10049688

    A setter function should have updated self.l just like it updated
    self.s:

    def __set__(self, obj, val):
    self.s=val
    self.l=len(val)
    print "setting value:", self.s, "length:", self.l

    Yet it didn't happen.

    >From my POV, the main benefit of a descriptor lies in its side effect:

    on dot access (getting/setting) I can get other attributes updated
    automatically: say, in class of Squares I get area automatically
    updated on updating side, etc.

    Yet, I'm struggling with getting it done in Python. Descriptors are a
    great idea, but I would like to see them implemented in Python in a
    way that makes it easier to get desireable side effects.
    , Nov 5, 2007
    #1
    1. Advertising

  2. a écrit :
    > Hello everyone,
    >
    > I'm trying to do seemingly trivial thing with descriptors: have
    > another attribute updated on dot access in object defined using
    > descriptors.
    >
    > For example, let's take a simple example where you set an attribute s
    > to a string and have another attribute l set automatically to its
    > length.
    >
    >>>> class Desc(str):

    > def __init__(self,val):
    > self.s=val
    > self.l=len(val)
    > print "creating value: ", self.s
    > print "id(self.l)", id(self.l)
    > def __set__(self, obj, val):
    > self.s=val
    > self.l=len(val)
    > print "setting value:", self.s, "length:", self.l
    > def __get__(self, obj, type=None):
    > print "getting value:", self.s, "length:", self.l
    > return self.l
    >
    >
    >>>> class some(str):

    > m=Desc('abc')
    > l=m.l


    First point : I don't get why Desc and some derive from str. Second
    point: I don't get why you're storing the value and it's length in the
    descriptor itself - obviously, you can't expect one descriptor instance
    to be mapped to 2 distinct attributes. Third point: you understand that,
    the way you wrote it, your descriptor will behave as a class (ie:shared)
    attribute, don't you. Fourth point: if you hope some.l to be rebound
    when m.l is, then you should learn Python basics before trying to jump
    into descriptors.

    The obvious, simple way to some your problem is to use a couple of
    properties:

    class Some(object):
    @apply
    def m():
    def fget(self):
    return self._m
    def fset(self, val):
    self._m = val
    self._l = len(val)
    return property(**locals())
    @apply
    def l():
    def fget(self):
    return self._l
    def fset(self):
    raise AttributeError("%s.l is readonly" % self)
    def __init__(self, m):
    self.m = m

    Now if you absolutely insist on using custom descriptors, you'll need
    two of them: one to manage access to s, and the second to manage access
    to l (which btw is a very bad name):

    class DescS(object):
    def __init__(self, val):
    self._default = val

    def __set___(self, obj, val):
    obj._s = val
    obj._l = len(val)

    def __get__(self, obj, cls):
    if obj is None:
    return self # or self._default, or whatever
    try:
    return obj._s
    except AttributeError:
    return self._default


    class DescL(object):
    def __init__(self, descS):
    self._descS = descS
    def __get__(self, obj, cls):
    if obj is None:
    return self # or self._default, or whatever
    try:
    return obj._l
    except AttributeError:
    return len(self._descS._default)


    class Test(object):
    m = DescS('abc')
    l = DescL(m)

    (nb : not tested)
    Bruno Desthuilliers, Nov 5, 2007
    #2
    1. Advertising

  3. Rich Harkins Guest

    wrote:
    > Hello everyone,
    >
    > I'm trying to do seemingly trivial thing with descriptors: have
    > another attribute updated on dot access in object defined using
    > descriptors.


    [snip]

    > A setter function should have updated self.l just like it updated
    > self.s:
    >
    > def __set__(self, obj, val):
    > self.s=val
    > self.l=len(val)
    > print "setting value:", self.s, "length:", self.l
    >
    > Yet it didn't happen.
    >

    [snip]

    I noticed that Python will block all attribute overrides (either via
    __dict__ through setattr) if the property has a __set__ method. The
    standard property has this method and there is no way that I can find to
    defeat it. So, here is what I use:

    class ConstProperty(object):
    """
    Provides a property that keeps its return value. The function will
    only be called on the first access. After that the same value can
    be used over and over again with no function call penalty. If the
    cached value needs to be cleared, simply del the attribute.

    >>> class MyClass(object):

    ... def __init__(self, x):
    ... self.x = x
    ... @ConstProperty
    ... def y(self):
    ... print "HERE"
    ... return self.x ** 2
    ...
    >>> obj = MyClass(5)
    >>> obj.y

    HERE
    25
    >>> obj.y

    25
    """

    def __init__(self, fn):
    self.fn = fn

    def __get__(self, target, cls=None):
    if target is None:
    return self.fn # Helps pydoc
    else:
    obj = self.fn(target)
    setattr(target, self.fn.__name__, obj)
    return obj

    This is a little different than what you originally posted, but
    hopefully it is close enough to be helpful.

    Cheers!
    Rich
    Rich Harkins, Nov 5, 2007
    #3
  4. Rich Harkins a écrit :
    > wrote:
    >> Hello everyone,
    >>
    >> I'm trying to do seemingly trivial thing with descriptors: have
    >> another attribute updated on dot access in object defined using
    >> descriptors.

    >
    > [snip]
    >
    >> A setter function should have updated self.l just like it updated
    >> self.s:
    >>
    >> def __set__(self, obj, val):
    >> self.s=val
    >> self.l=len(val)
    >> print "setting value:", self.s, "length:", self.l
    >>
    >> Yet it didn't happen.
    >>

    > [snip]
    >
    > I noticed that Python will block all attribute overrides (either via
    > __dict__ through setattr) if the property has a __set__ method.


    It doesn't "block", it controls access to... Of course, if the __set__
    method is a no-op, then nothing will happen.

    > The
    > standard property has this method and there is no way that I can find to
    > defeat it.


    "defeat" ? Why don't you just pass the appropriate fset function to
    property ?

    > So, here is what I use:
    >
    > class ConstProperty(object):
    > """
    > Provides a property that keeps its return value. The function will
    > only be called on the first access. After that the same value can
    > be used over and over again with no function call penalty. If the
    > cached value needs to be cleared, simply del the attribute.
    >
    > >>> class MyClass(object):

    > ... def __init__(self, x):
    > ... self.x = x
    > ... @ConstProperty
    > ... def y(self):
    > ... print "HERE"
    > ... return self.x ** 2
    > ...
    > >>> obj = MyClass(5)
    > >>> obj.y

    > HERE
    > 25
    > >>> obj.y

    > 25
    > """
    >
    > def __init__(self, fn):
    > self.fn = fn
    >
    > def __get__(self, target, cls=None):
    > if target is None:
    > return self.fn # Helps pydoc
    > else:
    > obj = self.fn(target)
    > setattr(target, self.fn.__name__, obj)
    > return obj




    >>> m = MyClass(5)
    >>> m.__dict__

    {'x': 5}
    >>> m.y

    HERE
    25
    >>> m.__dict__

    {'y': 25, 'x': 5}
    >>> m.x = 42
    >>> m.y

    25
    >>> m.__dict__

    {'y': 25, 'x': 42}
    >>>



    I'm sorry, but this looks like a very complicated way to do a simple thing:

    class MySimpleClass(object):
    def __init__(self, x):
    self.x = x
    self.y = x ** 2
    Bruno Desthuilliers, Nov 5, 2007
    #4
  5. Rich Harkins Guest

    Bruno Desthuilliers wrote:
    [snip]
    > I'm sorry, but this looks like a very complicated way to do a simple thing:
    >
    > class MySimpleClass(object):
    > def __init__(self, x):
    > self.x = x
    > self.y = x ** 2
    >
    >


    Sure, for the absurdly simplified case I posed as an example. ;)

    Here's another:

    class Path(tuple):
    @ConstProperty
    def pathstr(self):
    print "DEBUG: Generating string"
    return '/'.join(self)

    def __add__(self, other):
    if isinstance(other, tuple):
    return Path(tuple.__add__(self, other))
    else:
    return Path(tuple.__add__(self, (other,)))

    >>> ROOT = Path(())
    >>> path = ROOT + 'x' + 'y' + 'z'
    >>> path.pathstr

    DEBUG: Generating string
    /x/y/z
    >>> path.pathstr

    /x/y/z

    Basically, you can use ConstProperty above for items you don't want to
    calculate automatically, but only when someone actually WANTS it. After
    it is applied, then the penalties for function call of the property and
    the computation are wiped out once the second access is requested.

    Now, in the original example, len() might be considered too little for
    this use and should be just generated in the constructor "for free".
    OTOH, that assumes that __len__ hasn't been overridden to do something
    more complicated and time consuming. If the antecedent object is
    static, and the derivative consequent is also static, then ConstProperty
    works very well and shouldn't cost more on the first access than any
    other built-in property function.

    BTW, another use is to avoid creating lots of unnecessary objects for
    free unless they are accessed. Another quickie example:

    class Node(object):
    hasChildList = False
    hasAttributesDict = False

    @ConstProperty
    def children(self):
    self.hasChildList = True
    return []

    @ConstProperty
    def attributes(self):
    self.hasAttributesDict = True
    return {}

    The extra class/object attributes can be used to test for whether the
    associated objects were created. When used in a large tree, not
    creating a lot of extra lists and dictionaries can save a lot of memory
    and CPU as the children and attributes are not created or explored
    unless they were manipulated.

    Rich
    Rich Harkins, Nov 5, 2007
    #5
  6. Rich Harkins Guest

    Bruno Desthuilliers wrote:
    > Rich Harkins a écrit :
    >> wrote:
    >>> Hello everyone,
    >>>
    >>> I'm trying to do seemingly trivial thing with descriptors: have
    >>> another attribute updated on dot access in object defined using
    >>> descriptors.

    >> [snip]
    >>
    >>> A setter function should have updated self.l just like it updated
    >>> self.s:
    >>>
    >>> def __set__(self, obj, val):
    >>> self.s=val
    >>> self.l=len(val)
    >>> print "setting value:", self.s, "length:", self.l
    >>>
    >>> Yet it didn't happen.
    >>>

    >> [snip]
    >>
    >> I noticed that Python will block all attribute overrides (either via
    >> __dict__ through setattr) if the property has a __set__ method.

    >
    > It doesn't "block", it controls access to... Of course, if the __set__
    > method is a no-op, then nothing will happen.
    >
    >> The
    >> standard property has this method and there is no way that I can find to
    >> defeat it.

    >
    > "defeat" ? Why don't you just pass the appropriate fset function to
    > property ?
    >
    >> So, here is what I use:
    >>
    >> class ConstProperty(object):
    >> """
    >> Provides a property that keeps its return value. The function will
    >> only be called on the first access. After that the same value can
    >> be used over and over again with no function call penalty. If the
    >> cached value needs to be cleared, simply del the attribute.
    >>
    >> >>> class MyClass(object):

    >> ... def __init__(self, x):
    >> ... self.x = x
    >> ... @ConstProperty
    >> ... def y(self):
    >> ... print "HERE"
    >> ... return self.x ** 2
    >> ...
    >> >>> obj = MyClass(5)
    >> >>> obj.y

    >> HERE
    >> 25
    >> >>> obj.y

    >> 25
    >> """
    >>
    >> def __init__(self, fn):
    >> self.fn = fn
    >>
    >> def __get__(self, target, cls=None):
    >> if target is None:
    >> return self.fn # Helps pydoc
    >> else:
    >> obj = self.fn(target)
    >> setattr(target, self.fn.__name__, obj)
    >> return obj

    >
    >
    >
    > >>> m = MyClass(5)
    > >>> m.__dict__

    > {'x': 5}
    > >>> m.y

    > HERE
    > 25
    > >>> m.__dict__

    > {'y': 25, 'x': 5}
    > >>> m.x = 42
    > >>> m.y

    > 25
    > >>> m.__dict__

    > {'y': 25, 'x': 42}
    > >>>

    >
    >
    > I'm sorry, but this looks like a very complicated way to do a simple thing:
    >
    > class MySimpleClass(object):
    > def __init__(self, x):
    > self.x = x
    > self.y = x ** 2
    >
    >
    Rich Harkins, Nov 5, 2007
    #6
  7. Rich Harkins a écrit :
    > Bruno Desthuilliers wrote:
    > [snip]
    >
    >>I'm sorry, but this looks like a very complicated way to do a simple thing:
    >>
    >>class MySimpleClass(object):
    >> def __init__(self, x):
    >> self.x = x
    >> self.y = x ** 2
    >>
    >>

    >
    >
    > Sure, for the absurdly simplified case I posed as an example. ;)
    >
    > Here's another:
    >
    > class Path(tuple):


    Ok, for an immutable type, it might eventually work.

    > @ConstProperty
    > def pathstr(self):
    > print "DEBUG: Generating string"
    > return '/'.join(self)


    import os.path
    help(os.path)

    > def __add__(self, other):
    > if isinstance(other, tuple):
    > return Path(tuple.__add__(self, other))
    > else:
    > return Path(tuple.__add__(self, (other,)))
    >
    >
    >>>>ROOT = Path(())
    >>>>path = ROOT + 'x' + 'y' + 'z'
    >>>>path.pathstr

    >
    > DEBUG: Generating string
    > /x/y/z
    >
    >>>>path.pathstr

    >
    > /x/y/z


    >>> p = Path(('home', 'bruno'))
    >>> p += ['toto', 'tata']
    >>> p.pathstr

    DEBUG: Generating string
    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "/usr/tmp/python-8690chu", line 31, in __get__
    File "/usr/tmp/python-8690chu", line 40, in pathstr
    TypeError: sequence item 2: expected string, list found
    >>>



    > Basically, you can use ConstProperty above for items you don't want to
    > calculate automatically, but only when someone actually WANTS it.


    Which is easy to do with properties too.

    > After
    > it is applied, then the penalties for function call of the property and
    > the computation are wiped out once the second access is requested.


    Agreed. But I wouldn't use such a scheme for mutable types - which are
    still the common case.

    > Now, in the original example, len() might be considered too little for
    > this use and should be just generated in the constructor "for free".
    > OTOH, that assumes that __len__ hasn't been overridden to do something
    > more complicated and time consuming. If the antecedent object is
    > static, and the derivative consequent is also static,


    You mean 'immutable', I assume...

    > then ConstProperty
    > works very well and shouldn't cost more on the first access than any
    > other built-in property function.
    >
    > BTW, another use is to avoid creating lots of unnecessary objects for
    > free unless they are accessed. Another quickie example:
    >
    > class Node(object):
    > hasChildList = False
    > hasAttributesDict = False
    >
    > @ConstProperty
    > def children(self):
    > self.hasChildList = True
    > return []
    >
    > @ConstProperty
    > def attributes(self):
    > self.hasAttributesDict = True
    > return {}


    Hmm... Perhaps not such a bad idea after all !-)
    Bruno Desthuilliers, Nov 5, 2007
    #7
  8. Rich Harkins Guest

    Bruno Desthuilliers wrote:
    > Which is easy to do with properties too.


    True enough. It's the caching of the return value that's the value add
    of course. ;)

    >
    >> After
    >> it is applied, then the penalties for function call of the property and
    >> the computation are wiped out once the second access is requested.

    >
    > Agreed. But I wouldn't use such a scheme for mutable types - which are
    > still the common case.
    >


    In many cases, yeah. Though I use a lot of immutable stuff in some of
    my pet projects and such. ConstProperty is definitely not meant as a
    replacement for property, only when something constant can be derived
    from something else constant, especially when the derivation is expensive.

    >> Now, in the original example, len() might be considered too little for
    >> this use and should be just generated in the constructor "for free".
    >> OTOH, that assumes that __len__ hasn't been overridden to do something
    >> more complicated and time consuming. If the antecedent object is
    >> static, and the derivative consequent is also static,

    >
    > You mean 'immutable', I assume...


    Yeah, that's probably the better term.

    [snip]

    Again, I've used it quite a bit for various things and it's worked well
    for the sort of thing the OP was requesting. Of course, your mileage
    may vary. :)

    Cheers!
    Rich

    PS: Sorry about the weird reposts. Thunderbird chaos.
    Rich Harkins, Nov 5, 2007
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    17
    Views:
    561
    Tim Rentsch
    Oct 12, 2005
  2. copx

    macros and side effects

    copx, May 4, 2007, in forum: C Programming
    Replies:
    19
    Views:
    631
  3. , India
    Replies:
    2
    Views:
    760
    James Kanze
    Dec 25, 2009
  4. moerchendiser2k3

    Py_single_input and the side-effects...

    moerchendiser2k3, May 29, 2010, in forum: Python
    Replies:
    9
    Views:
    428
    Mark Lawrence
    Jun 18, 2010
  5. regis
    Replies:
    63
    Views:
    543
    Seebs
    Nov 21, 2013
Loading...

Share This Page