Declarative properties

[Bruno Desthuilliers]

Dan Stromberg a écrit :

Python's current performance characteristics have no bearing on what is
good software engineering practice in general,

Neither useless complexity nor tryig to apply inappropriate idioms are
good practices.

(snip)

I'm not omniscient, and neither is anyone else; when one initially codes a
class, one doesn't know to what purposes it will need to be bent in the
future; using accessor methods instead of exposed attributes is
significantly more flexible for the future of your class.

Please remember that Python has support for computed attributes (you do
know what's a computed attribute, don't you ?), so you can turn a plain
attribute into a computed one at any time without breakink the API. IOW,
you don't need getters/setter a priori.

 

[Bruno Desthuilliers]

Dan Stromberg a écrit :
(snip)

My implementation may or may not be lacking (feel free to improve it - in
fact, please do!),

Since you ask for it:

def makeprop(name):
_name = '_' + name
def fget(self):
return getattr(self, _name, None)
def fset(self, val):
setattr(self, _name, val)
return property(fget, fset)

class AutoPropType(type):
def __init__(cls, clsname, bases, attribs):
for name in cls._autoprops:
if name not in attribs:
setattr(cls, name, makeprop(name))

class Demo(object):
__metaclass__ = AutoPropType
_autoprops = ('foo', 'bar', 'baaz')

def __init__(self, foo, bar=42, baaz='ni'):
self.foo = foo
self.bar = bar
self.baaz = baaz

d = Demo('yadda yadda')

> but that's irrelevant to the heart of the matter, which
> is "what you didn't think to plan for now most definitely can hurt you
> later".

Ahem... Please let me know, Mr Professor, what difference it would make,
from client code's POV, if we simply wrote this instead:

class Demo(object):
def __init__(self, foo, bar=42, baaz='ni'):
self.foo = foo
self.bar = bar
self.baaz = baaz


(snip more blah)

Little hackish tricks for performance's sake scattered throughout a
program are very wasteful of something more precious than CPU time.

Indeed. The point you're missing is that, in a language that supports
computed attributes, using plain attributes when that's all you need is
*not* a 'litlle hackish trick'. It's just common sense - and actually
saves on *both* programmer's and CPU time.

 

[Artur Siekielski]

Hi.

I would like to have declarative properties in Python, ie. something
like slots definitions in defclass in Common Lisp. It seems that even
Java will have it, using a library ( https://bean-properties.dev.java.net/
).

I know about 'property' function in Python, but it's normal usage
isn't declarative, because I have to code imperatively getters and
setters:

class Person(object):
def __init__(self, name):
self._name = name
def _get_name(self):
return self._name
def _set_name(self, new_name):
self._name = new_name
name = property(_get_name, _set_name)

I would like to have something like that:

class Person(object):
name = property('_name')

I assume that this causes "generation" of instance field '_name' and
default getters and setters. And if I would like to customize (g|
s)etters, I would write them by hand, or, better, use more declarative
approach (like @NotNull annotations in Java version).

I could probably code a solution to my problem with help of wonderful
introspection functionalities. But I'm looking for a standard and/or
proven way of doing this. Maybe some PEP is being prepared for this?

Regards,
Artur

 

[Marc 'BlackJack' Rintsch]

class Person(object):
def __init__(self, name):
self._name = name
def _get_name(self):
return self._name
def _set_name(self, new_name):
self._name = new_name
name = property(_get_name, _set_name)

This is more easily spelled:

class Person(object):
def __init__(self, name):
self.name = name

I would like to have something like that:

class Person(object):
name = property('_name')

I assume that this causes "generation" of instance field '_name' and
default getters and setters.

But why? Default getters and setters are unnecessary and if you need
something other than the default you need to write it anyway more
explicitly.

Ciao,
Marc 'BlackJack' Rintsch

 

[Artur Siekielski]

But why? Default getters and setters are unnecessary and if you need
something other than the default you need to write it anyway more
explicitly.

I see some problems with your approach:

1. If I use instance field 'name' which is accessed directly by other
classes,
and later I decide to implement nonstandard getter, I must refactor
'Person' class
and in some places change 'name' to '_name' (assuming this is now the
field's name).
The problem is that I cannot automatically change 'name' to '_name'
everywhere, because
in some places I want public property value (eg. validated and
formatted), and in other
places raw property value.

2. Properties define (a part of) public interface of a class. When
using fields for public
access, you must tell this explicitly in documentation, or use name
convention. And public
fields definition is mixed with private fields, which doesn't happen
when using properties.

3. Using properties as first-class objects gives possibilities to use
declarative approaches
for constructing them.

 

[Marc 'BlackJack' Rintsch]

I see some problems with your approach:

1. If I use instance field 'name' which is accessed directly by other
classes,
and later I decide to implement nonstandard getter, I must refactor
'Person' class
and in some places change 'name' to '_name' (assuming this is now the
field's name).
The problem is that I cannot automatically change 'name' to '_name'
everywhere, because
in some places I want public property value (eg. validated and
formatted), and in other
places raw property value.

So what? Otherwise you carry *always* the baggage of a public property
and a private attribute whether you need this or not. At least for me it
would be unnecessary in most cases.

2. Properties define (a part of) public interface of a class. When
using fields for public
access, you must tell this explicitly in documentation, or use name
convention. And public
fields definition is mixed with private fields, which doesn't happen
when using properties.

Attributes are public unless you start the name with an underscore which is
by convention a warning to not use that attribute unless you know what you
are doing. With properties all over the place you use that convention
anyway because the "real" value for `name` is bound as `_name`. So you
clutter your API with two attributes for non-method-attributes.

And a property is not automatically part of the public API, just if the
name does not start with an underscore.

3. Using properties as first-class objects gives possibilities to use
declarative approaches for constructing them.

No idea what you mean here. Everything in Python is an object.

Ciao,
Marc 'BlackJack' Rintsch

 

[sjdevnull]

I see some problems with your approach:

1. If I use instance field 'name' which is accessed directly by other
classes,
and later I decide to implement nonstandard getter, I must refactor
'Person' class
and in some places change 'name' to '_name' (assuming this is now the
field's name).
The problem is that I cannot automatically change 'name' to '_name'
everywhere, because
in some places I want public property value (eg. validated and
formatted), and in other
places raw property value.

In practice, it turns out to be a lot less work to deal with that
occasionally than to always deal with lugging around internal
attributes and external properties when they're really not needed. By
writing everything as properties up front you're just creating more
work for yourself, and it's generally considered bad practice in
Python to have default getters/setters (whether part of a property or
not).

2. Properties define (a part of) public interface of a class. When
using fields for public
access, you must tell this explicitly in documentation, or use name
convention.

Property vs. attribute doesn't make any difference here: both of them
are public, and part of the external interface, unless they're named
with a leading underscore.

Making something a property doesn't make it any more or less a part of
the public interface.

3. Using properties as first-class objects gives possibilities to use
declarative approaches
for constructing them.

This doesn't make any sense. Properties and attributes are both
objects in python.

 

[Artur Siekielski]

In practice, it turns out to be a lot less work to deal with that
occasionally than to always deal with lugging around internal
attributes and external properties when they're really not needed. By
writing everything as properties up front you're just creating more
work for yourself, and it's generally considered bad practice in
Python to have default getters/setters (whether part of a property or
not).

Ok, I appreciate your and Marc's argument. So I see that a pythonic
way is
using public attributes, and it works in real life. Automatic
"generation" of
getters/setters is a nice thing when using Java style of properties,
ie. no
public fields.

Property vs. attribute doesn't make any difference here: both of them
are public, and part of the external interface, unless they're named
with a leading underscore.

Nice thing about properites is that they are defined in more
declarative way -
in class body. Attributes are "defined" in a piece of code (which can
be long
and can contain loops etc.) by assigning a value to 'self'.

This doesn't make any sense. Properties and attributes are both
objects in python.

Attribute is an object which has value specific for the time it's get
from object.
It is separated from containing object - relation to it is lost.
Python property is an object that describes how to get/set a specific
attribute of
containing object - any time. It describes relationship between the
two.

By declarative method I mean something like a function that takes a
property object
and returns modified property object:

def not_none(prop):
def new_fset(self, obj):
if obj is not None:
prop.fset(self, obj)
else:
raise 'Cannot set None value'
return property(prop.fget, new_fset, prop.fdel)

And then:

class Person(object):
....
name = not_none( property(_get_name, _set_name) )

 

[Marc 'BlackJack' Rintsch]

Nice thing about properites is that they are defined in more declarative
way - in class body. Attributes are "defined" in a piece of code (which
can be long and can contain loops etc.) by assigning a value to 'self'.

The convention is to bind all attributes in `__init__()`, possibly to
`None` if the real value is not available at that time, and document at
least the public ones in the class' docstring. Tools like `pylint` check
for attributes that are introduced in other methods than `__init__()` and
give a warning.

Ciao,
Marc 'BlackJack' Rintsch

 

[Bruno Desthuilliers]

Artur Siekielski a écrit :

Hi.

I would like to have declarative properties in Python, ie. something
like slots definitions in defclass in Common Lisp. It seems that even
Java will have it, using a library ( https://bean-properties.dev.java.net/
).

I know about 'property' function

Actually, it's a class.

in Python, but it's normal usage
isn't declarative, because I have to code imperatively getters and
setters:

Indeed. What would be the use of a property (AKA computed attribute) if
you don't need to do something specific ?

class Person(object):
def __init__(self, name):
self._name = name
def _get_name(self):
return self._name
def _set_name(self, new_name):
self._name = new_name
name = property(_get_name, _set_name)

While it's often seen as demonstration code for a property, it's
actually totally useless. If all you want to do is to get and set an
attribute (I mean, without any computation anywhere), then just use an
attribute - you will always be able to turn it into a property if and
whene the need arises.

I would like to have something like that:

class Person(object):
name = property('_name')

I assume that this causes "generation" of instance field '_name' and
default getters and setters.

So far, it's a waste of time. That's exactly what you'd get with a plain
attribute, with far less complexity and overhead.

And if I would like to customize (g|
s)etters, I would write them by hand,

Which bring us back to how properties actually work !-)

or, better, use more declarative
approach (like @NotNull annotations in Java version).

I could probably code a solution to my problem with help of wonderful
introspection functionalities. But I'm looking for a standard and/or
proven way of doing this. Maybe some PEP is being prepared for this?

I guess you want to have a look at the descriptor protocol - it's what
makes properties work, and it allow you to write your own custom 'smart
attributes'. If you want automatic validation/conversion, you could
write custom descriptors working with the FormEncode package (I started
such a thing for Elixir, but had no time to finish it so far). It would
then looks like:

class Person(object):
name = StringField(empty=False)

// etc

HTH

 

[Bruno Desthuilliers]

Artur Siekielski a écrit :

I see some problems with your approach:

1. If I use instance field 'name' which is accessed directly by other
classes,
and later I decide to implement nonstandard getter, I must refactor
'Person' class
and in some places change 'name' to '_name' (assuming this is now the
field's name).

Why so ?

class Toto(object):
def __iinit__(self, name):
self.name = name
@apply
def name():
def fget(self):
print "getting %s.name" % self
return self._name
def fset(self, val):
print "setting %s.name to %s" % (self, val)
self._name = name
def say_hello(self):
print "Hello, my name is %s" % self.name

t = Toto('bruno')
t.say_hello()

The problem is that I cannot automatically change 'name' to '_name'
everywhere, because
in some places I want public property value (eg. validated and
formatted), and in other
places raw property value.

But then, you do know where you want the raw value, don't you ? And this
would be for special cases that didn't exist before this refactoring,
anyway ?

2. Properties define (a part of) public interface of a class.

So do attributes.

When
using fields for public
access, you must tell this explicitly in documentation, or use name
convention. And public
fields definition is mixed with private fields, which doesn't happen
when using properties.

I just don't understand what you're saying here. As long as you did not
prefix a name with an underscore, it's part of the API. And anyway,
nothing (except common sens) will prevent anyone to mess with
implementation attributes.

3. Using properties as first-class objects

Which they are already - I don't know of anything that isn't a
"first-class" object in Python.

 

[Dan Stromberg]

So what? Otherwise you carry *always* the baggage of a public property
and a private attribute whether you need this or not. At least for me it
would be unnecessary in most cases.

That "baggage" of carrying around "unneeded" methods is something the
computer carries for you - IE, no big deal in 99.99% of all cases.

The "baggage" of possibly fixing (AKA "generalizing") how your attributes
are accessed is something you lug around while your deadline looms.

Here's some code that defines such methods for you:

#!/usr/bin/env python

def gimme_set_get(foo, attribute):
lst = [ \
'def set_%s(self, value):' % attribute, \
' self._%s = value' % attribute, \
'def get_%s(self):' % attribute, \
' return self._%s' % attribute, \
'foo.set_%s = set_%s' % (attribute, attribute), \
'foo.get_%s = get_%s' % (attribute, attribute) \
]
s = '\n'.join(lst)
code = compile(s, '<string>', 'exec')
eval(code)

class foo:
def __init__(self, value):
self.public_value = value
gimme_set_get(foo, 'via_accessor_method_only')

f = foo(1)
f.set_via_accessor_method_only(1/9.0)
print f.get_via_accessor_method_only()

print dir(f)

 

[Bruno Desthuilliers]

Artur Siekielski a écrit :

Ok, I appreciate your and Marc's argument. So I see that a pythonic
way is
using public attributes, and it works in real life.

Indeed !-)

Automatic
"generation" of
getters/setters is a nice thing when using Java style of properties,
ie. no
public fields.

Java doesn't have computed attributes, so you have no other choice if
you want to keep your design flexible. For languages that have computed
attributes, this is not a problem to use plain attributes until you
*really* need to do otherwise.

Nice thing about properites is that they are defined in more
declarative way -
in class body. Attributes are "defined" in a piece of code (which can
be long
and can contain loops etc.) by assigning a value to 'self'.

Usually, all there is to know is readable in the initializer. And I've
rarely seen (good) code that defines public attribute elsewehere (i've
seen such a thing on a couple occasion, and that's bad coding IMHO).

Anyway, since attributes (including methods, class etc...) can be
added/replaced/deleted by anyone having access to the object at any
point of the program, using properties won't help that much here IMHO.
Not that I don't see your point - I also have a taste for declarative
APIs and readability - but this is enough to justify the use of
properties IMHO.

 

[Marc 'BlackJack' Rintsch]

That "baggage" of carrying around "unneeded" methods is something the
computer carries for you - IE, no big deal in 99.99% of all cases.

It shows twice as much attributes if I inspect the objects and I don't know
which are merely useless default getters and setters. And it is more and
more complex code. Code can be cut down a bit by some metaclass magic but
this brings in another complexity.

The "baggage" of possibly fixing (AKA "generalizing") how your attributes
are accessed is something you lug around while your deadline looms.

Sorry I don't get it. If I want to customize the access to a "normal"
attribute I simply turn it into a property.

Here's some code that defines such methods for you:

#!/usr/bin/env python

def gimme_set_get(foo, attribute):
lst = [ \
'def set_%s(self, value):' % attribute, \
' self._%s = value' % attribute, \
'def get_%s(self):' % attribute, \
' return self._%s' % attribute, \
'foo.set_%s = set_%s' % (attribute, attribute), \
'foo.get_%s = get_%s' % (attribute, attribute) \
]
s = '\n'.join(lst)
code = compile(s, '<string>', 'exec')
eval(code)

class foo:
def __init__(self, value):
self.public_value = value
gimme_set_get(foo, 'via_accessor_method_only')

f = foo(1)
f.set_via_accessor_method_only(1/9.0)
print f.get_via_accessor_method_only()

print dir(f)

And the benefit of this evil ``eval`` dance is exactly what!?

Ciao,
Marc 'BlackJack' Rintsch

 

[Paul Hankin]

Hi.

I would like to have declarative properties in Python, ie. something
like slots definitions in defclass in Common Lisp. It seems that even
Java will have it, using a library (https://bean-properties.dev.java.net/
).

I know about 'property' function in Python, but it's normal usage
isn't declarative, because I have to code imperatively getters and
setters:

class Person(object):
def __init__(self, name):
self._name = name
def _get_name(self):
return self._name
def _set_name(self, new_name):
self._name = new_name
name = property(_get_name, _set_name)

I would like to have something like that:

class Person(object):
name = property('_name')

Here's something that does what I think you want. I think your use
case is quite unusual in that you expect the public accessors to
change (keeping private use the same), and that you know you're going
to have a costly rewrite of your class when that happens.

Attributes that are declared with 'magic_attribute' access the private
attribute '_<name>', unless there's a '_get_<name>' or '_set_<name>'
method in which case those are called instead.

def magic_attribute(_name):
def set(self, value):
try:
getattr(self, '_set' + _name)(value)
except AttributeError:
setattr(self, _name, value)
def get(self):
try:
return getattr(self, '_get' + _name)()
except AttributeError:
return getattr(self, _name)
return property(get, set)

# An example use...
class C(object):
x = magic_attribute('_x')
y = magic_attribute('_y')

# Override the default set method for x
def _set_x(self, value):
self._x = value + ' world'

# Override the default get method for y
def _get_y(self):
return self._y.upper()

# Test it works...
c = C()
c.x = c.y = 'hello'
print c.x
print c.y

# hello world
# HELLO

 

[Paul Hankin]

Sorry I don't get it. If I want to customize the access to a "normal"
attribute I simply turn it into a property.

I *think* I understand Artur's problem: he wants to be able to add
(for example) clean-up and validation code to public accesses of these
attributes, but he doesn't want this code run internally to the class.
Or another way of saying the same thing: he wants two views of a
variable; one internal to the class, another to the public.

If I understand correctly, his plan is to use 'X._name' internally to
code in his class, but the public uses 'X.name'. Initially, one is an
alias for the other, but later he writes getters and setters to
customise access through the public version.

[I could be totally wrong of course]

 

[George Sakkis]

I know about 'property' function in Python, but it's normal usage
isn't declarative, because I have to code imperatively getters and
setters:

class Person(object):
def __init__(self, name):
self._name = name
def _get_name(self):
return self._name
def _set_name(self, new_name):
self._name = new_name
name = property(_get_name, _set_name)

I would like to have something like that:

class Person(object):
name = property('_name')

By now you must have been convinced that default getters/setters is
not a very useful idea in Python but this does not mean you can't do
it; it's actually straightforward:

def make_property(attr):
return property(lambda self: getattr(self,attr),
lambda self, value: setattr(self, attr, value))

class Person(object):
name = make_property('_name')
def __init__(self, name):
self.name = name

You could take it even further by removing the need to repeat the
attribute's name twice. Currently this can done only through
metaclasses but in the future a class decorator would be even better:

def PropertyMaker(*names, **kwds):
format = kwds.get('format', '_%s')
def meta(cls,bases,attrdict):
for name in names:
attrdict[name] = make_property(format % name)
return type(cls,bases,attrdict)
return meta

class Person(object):
__metaclass__ = PropertyMaker('name', format='__%s__')

def __init__(self, name):
self.name = name
print self.__name__


George

 

[George Sakkis]

You could take it even further by removing the need to repeat the
attribute's name twice. Currently this can done only through
metaclasses but in the future a class decorator would be even
better:

Replying to myself here, but actually metaclasses is not the only way;
another solution involves a descriptor class:


class Property(object):

# cache the mapping of types to 'private' attribute names
_type2attrname = {}

def __init__(self, format='_%s'):
self._format = format

def __get__(self, obj, type=None):
try: name = self._type2attrname[type(obj)]
except KeyError:
self._type2attrname[type(obj)] = name =
self._get_propname(obj)
return getattr(obj, name)

def __set__(self, obj, value):
try: name = self._type2attrname[type(obj)]
except KeyError:
self._type2attrname[type(obj)] = name =
self._get_propname(obj)
setattr(obj, name, value)

def _get_propname(self, obj):
for cls in type(obj).mro():
for name,value in cls.__dict__.iteritems():
if value is self:
return self._format % name
assert False # unreachable


#---- example ------------------------------------

class Person(object):
name = Property()

def __init__(self, name):
self.name = name

p = Person('John')
q = Person('Mike')
print p.name, q.name
print p.__dict__


George

 

[Artur Siekielski]

By now you must have been convinced that default getters/setters is
not a very useful idea in Python but this does not mean you can't do
it;

It's a perfect summary of my thoughts after reading this thread. I
will use public attributes (with access customizable with properties)
and remember that in Python I can do everything .
Thanks everybody.

 

[Bruno Desthuilliers]

Dan Stromberg a écrit :

That "baggage" of carrying around "unneeded" methods is something the
computer carries for you - IE, no big deal in 99.99% of all cases.

1/ Accessing the value of a property is not free. Accessing a plain
attribute is much cheaper.

2/ cluttering the class's namespace with useless names and the source
code with useless code is definitively not a good thing.

The "baggage" of possibly fixing (AKA "generalizing") how your attributes
are accessed is something you lug around while your deadline looms.

Here's some code that defines such methods for you:

#!/usr/bin/env python

def gimme_set_get(foo, attribute):
lst = [ \
'def set_%s(self, value):' % attribute, \
' self._%s = value' % attribute, \
'def get_%s(self):' % attribute, \
' return self._%s' % attribute, \
'foo.set_%s = set_%s' % (attribute, attribute), \
'foo.get_%s = get_%s' % (attribute, attribute) \
]
s = '\n'.join(lst)
code = compile(s, '<string>', 'exec')
eval(code)

Yuck. This is not only a waste of time, but also badly coded. have mercy
and at least learn how to properly use lexical scoping and closures.

class foo:
def __init__(self, value):
self.public_value = value
gimme_set_get(foo, 'via_accessor_method_only')

f = foo(1)
f.set_via_accessor_method_only(1/9.0)
print f.get_via_accessor_method_only()

print dir(f)

Correction: this is not only a waste of time and badly coded, this is
also misleading and braindead.