reuse validation logic with descriptors

[David S.]

I am looking for a way to implement the same simple validation on many
instance attributes and I thought descriptors
(http://users.rcn.com/python/download/Descriptor.htm) looked like the
right tool.

But I am confused by their behavior on instance of my class.
I can only get the approximate behavior by using class variables.

I am looking for something like:

class SingleChar(object):
def init(self):
self._char = None

def __set__(self, instance, value):
if not len(value) == 1:
raise ValueError
self._char = value

def __get__(self, instance, owner):
return self._char

class Flags(object):
def __init__(self):
self.a = SingleChar()
self.b = SingleChar()

f = Flags()
f.a = "a"
f.b = "bb"
exceptions.ValueError
ValueError:

What I actually get when I try this is f.a and f.b become str instances.

Meanwhile, I can get this to work, except that a and b are now just class
attributes.

class CFlags(object):
a = SingleChar()
b = SingleChar()

What is the proper and clean way to accomplish this sort of thing, so that you
can reuse the logic in for many instance attributes across multiple classes?

Thanks, David S.

 

[Steven Bethard]

I am looking for a way to implement the same simple validation on many
instance attributes and I thought descriptors
(http://users.rcn.com/python/download/Descriptor.htm) looked like the
right tool.

But I am confused by their behavior on instance of my class.
I can only get the approximate behavior by using class variables.

I am looking for something like:

class SingleChar(object):
def init(self):
self._char = None

def __set__(self, instance, value):
if not len(value) == 1:
raise ValueError
self._char = value

def __get__(self, instance, owner):
return self._char

class Flags(object):
def __init__(self):
self.a = SingleChar()
self.b = SingleChar()

f = Flags()
f.a = "a"
f.b = "bb"
exceptions.ValueError
ValueError:

What I actually get when I try this is f.a and f.b become str instances.

Meanwhile, I can get this to work, except that a and b are now just class
attributes.

class CFlags(object):
a = SingleChar()
b = SingleChar()

What is the proper and clean way to accomplish this sort of thing, so that you
can reuse the logic in for many instance attributes across multiple classes?

Looks like you're trying to reinvent the property descriptor. Try using
the builtin property instead:

py> def getchar(self):
.... if not hasattr(self, '_char'):
.... self._char = None
.... return self._char
....
py> def setchar(self, value):
.... if not len(value) == 1:
.... raise ValueError
.... self._char = value
....
py> singlechar = property(getchar, setchar)
py> class Flags(object):
.... a = singlechar
.... b = singlechar
....
py> f = Flags()
py> f.a = "a"
py> f.b = "bb"
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 3, in setchar
ValueError

STeVe

 

[David S.]

Looks like you're trying to reinvent the property descriptor. Try using
the builtin property instead:

py> def getchar(self):
... if not hasattr(self, '_char'):
... self._char = None
... return self._char
...
py> def setchar(self, value):
... if not len(value) == 1:
... raise ValueError
... self._char = value
...
py> singlechar = property(getchar, setchar)
py> class Flags(object):
... a = singlechar
... b = singlechar
...
py> f = Flags()
py> f.a = "a"
py> f.b = "bb"
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 3, in setchar
ValueError

This still fails to work for instances variables of the class. That is
if I use your property in the following:
py> ...class Flags(object):
.... def __init__(self):
.... a = singlechar
....
py> f = Flags()
py> f.a = "a"

Now f.a.__class__.__name__ returns 'str'. So the property was not
used at all.

Also, it seems that using a property, I can not do the other useful
things I can do with a proper class, like provide an __init__, __str__,
or __repr__.

Again, thanks,
David S.

 

[Michael Spencer]

This still fails to work for instances variables of the class. That is
if I use your property in the following:
py> ...class Flags(object):
... def __init__(self):
... a = singlechar
...

you should write that as:
class Flags(object):
a = singlechar
def __init__(self):
a = "a"

py> f = Flags()
py> f.a = "a"

Now f.a.__class__.__name__ returns 'str'. So the property was not
used at all.

Also, it seems that using a property, I can not do the other useful
things I can do with a proper class, like provide an __init__, __str__,
or __repr__.

If you want "other useful things" then you can write a custom descriptor, like:

from weakref import WeakKeyDictionary

class SingleChar(object):
def __init__(self):
"""raises ValueError if attribute is set to something
other than a single char"""
self.objdict = WeakKeyDictionary()
def __get__(self, obj, cls):
if isinstance(obj, cls):
try:
return self.objdict[obj]
except KeyError:
raise AttributeError, "property not set"
else:
return self
def __set__(self, obj, value):
if isinstance(value, str) and len(value) == 1:
self.objdict[obj] = value
else:
raise ValueError, value

class Flags(object):
a = SingleChar()
b = SingleChar()


See also: http://groups-beta.google.com/group/comp.lang.python/msg/30c61a30a90133d2

for another example of this approach

Michael

 

[Steven Bethard]

This still fails to work for instances variables of the class. That is
if I use your property in the following:
py> ...class Flags(object):
... def __init__(self):
... a = singlechar
...
py> f = Flags()
py> f.a = "a"

Yes, you need to assign it at the class level, as you will for any
descriptor. Descriptors only function as attributes of type objects.
But note that as I've used them above, they do work on a per-instance
basis. What is it you're trying to do by assigning them in __init__?
Do you want different instances of Flags to have different descriptors?

Also, it seems that using a property, I can not do the other useful
things I can do with a proper class, like provide an __init__, __str__,
or __repr__.

Well, you can write your own descriptors that do things much like
property, but note that __init__ will only be invoked when you first
create them, and __str__ and __repr__ will only be invoked when you
actually return the descriptor object itself. For example:

py> class Descr(object):
.... def __init__(self):
.... self.value = None
.... def __repr__(self):
.... return 'Descr(value=%r)' % self.value
....
py> class DescrSelf(Descr):
.... def __get__(self, obj, type=None):
.... return self
....
py> class DescrObj(Descr):
.... def __get__(self, obj, type=None):
.... return obj
....
py> class DescrValue(Descr):
.... def __get__(self, obj, type=None):
.... return obj.value
....
py> class C(object):
.... s = DescrSelf()
.... o = DescrObj()
.... v = DescrValue()
....
py> C.s
Descr(value=None)
py> print C.o
None
py> C.v
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 3, in __get__
AttributeError: 'NoneType' object has no attribute 'value'
py> c = C()
py> c.s
Descr(value=None)
py> c.o
<__main__.C object at 0x011B65F0>
py> c.v
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 3, in __get__
AttributeError: 'C' object has no attribute 'value'
py> c.value = False
py> c.s
Descr(value=None)
py> c.o
<__main__.C object at 0x011B65F0>
py> c.v
False

The point here is that, if you define __repr__ for a descriptor, it will
only be invoked when the descriptor itself is returned. But you're
storing your string as an attribute of the descriptor (like 'value'
above), so you want the __repr__ on this attribute, not on the
descriptor itself. As you'll note from the code above, the only time
__repr__ is called is when the descriptor returns 'self'. But for
almost all purposes, you're going to want to do something like
DescrValue does (where you return an attribute of the object, not of the
descriptor).

If you want a single-character string type as an attribute, why not
subclass str and use a property?

py> class SingleChar(str):
.... def __new__(cls, s):
.... if len(s) != 1:
.... raise ValueError
.... return super(SingleChar, cls).__new__(cls, s)
.... def __repr__(self):
.... return 'SingleChar(%s)' % super(SingleChar, self).__repr__()
....
py> def getchar(self):
.... return self._char
....
py> def setchar(self, value):
.... self._char = SingleChar(value)
....
py> singlechar = property(getchar, setchar)
py> class Flags(object):
.... a = singlechar
.... b = singlechar
....
py> f = Flags()
py> f.a = "a"
py> f.a
SingleChar('a')
py> f.b = "bb"
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 2, in setchar
File "<interactive input>", line 4, in __new__
ValueError

Note that now (unlike if you'd used only a descriptor), the __repr__ is
correctly invoked.

STeVe

P.S. If you haven't already, you should read
http://users.rcn.com/python/download/Descriptor.htm a couple of times.
It took me until about the third time I read it to really understand
what descriptors were doing. The big thing to remember is that for an
instance b,
b.x
is equivalent to
type(b).__dict__['x'].__get__(b, type(b))
and for a class B,
B.x
is equivalent to
B.__dict__['x'].__get__(None, B)
Note that 'x' is always retrieved from the *type* __dict__, not from the
*instance* __dict__.

 

[David S.]

P.S. If you haven't already, you should read
http://users.rcn.com/python/download/Descriptor.htm a couple of times.
It took me until about the third time I read it to really understand
what descriptors were doing. The big thing to remember is that for an
instance b,
b.x
is equivalent to
type(b).__dict__['x'].__get__(b, type(b))
and for a class B,
B.x
is equivalent to
B.__dict__['x'].__get__(None, B)
Note that 'x' is always retrieved from the *type* __dict__, not from the
*instance* __dict__.

Steve, and others, thanks for the help. This and Michael Spencer's reply
at http://article.gmane.org/gmane.comp.python.general/390478 have been very
helpful in getting the descriptor definition clear. For me, it has taken
reading http://users.rcn.com/python/download/Descriptor.htm about 4 times
along with your help to get this straight.

Peace,
David S.

 

[Steve Holden]

This still fails to work for instances variables of the class. That is
if I use your property in the following:
py> ...class Flags(object):
... def __init__(self):
... a = singlechar
...
py> f = Flags()
py> f.a = "a"

Now f.a.__class__.__name__ returns 'str'. So the property was not
used at all.

You want assignment to a method-local variable to turn an attribute into
a property? That's programming with a magic wand ...

Also, it seems that using a property, I can not do the other useful
things I can do with a proper class, like provide an __init__, __str__,
or __repr__.

That will depend on the value returned by property access, surely?

I suspect you are a little confused about properties and descriptors.

regards
Steve

 

[Nick Coghlan]

Steve, and others, thanks for the help. This and Michael Spencer's reply
at http://article.gmane.org/gmane.comp.python.general/390478 have been very
helpful in getting the descriptor definition clear. For me, it has taken
reading http://users.rcn.com/python/download/Descriptor.htm about 4 times
along with your help to get this straight.

If it only takes 2 or 3 re-reads to get descriptors, does that mean Python's
black magic is really only a kind of off-white colour?

Cheers,
Nick.

 

[David S.]

You want assignment to a method-local variable to turn an attribute into
a property? That's programming with a magic wand ...

That will depend on the value returned by property access, surely?

I suspect you are a little confused about properties and descriptors.

regards
Steve

Quite confused, actually, which was the reason for my original post.
Thanks again to those who helped me and any other confused folks
understand this bit of Python that much better.
Peace,
David S.