Python descriptor protocol (for more or less structured data)

[CWr]

Hi together,

Some years ago I started a small WSGI project at my university. Since then the project was grown up every year. Some classes have more than 600 lines of code with (incl. boiler-plates mostly in descriptors/properties).

Many of these properties are similar or have depencies among themselves.
The idea is to grouping similar properties like:

new style:
----------{'complex''data','string')}

I wrote this descriptor:

class Descr:

def __init__(self, value):
self.attribute = self.__class__.__name__
self.__set__(None, value)

def __get__(self, obj, Type=None):
return getattr(obj, self.attribute, self)

def __set__(self, obj, value):
if obj is None: # descripting yourself
# do something here ...
self.value = value
else:
if hasattr(obj, self.attribute):
self.__get__(obj).__set__(None, value)
else:
setattr(obj, self.attribute, type(self)(value))

This works fine as long as the value attribute of Descr is read-only and the
user have to use the descriptor interface e.g. __get__/__set__. Because
it's not guaranteed that the user gets a seperated instance of Descr which
will be living in obj.__dict__. If obj is None the descriptor will be returned
themselves.

But I would like that the user can use the following statement:
'some other'

But this usage will be problematic. If the descriptor returned themselves (default case) and the user modified the value, he modified the default
value without to create a seperated instance attribute.
'default value'
'default value'

The following is the main Problem:
'other'

The user could think that a new instance based value will be created. But it
isn't.

It will works fine only if I assign a value explicitly.

Has anyone had a similar problem in the past? Or I am on the wrong way.

Kind Regards,
Chris

Sorry for my terrible english ...

 

[Peter Otten]

Some years ago I started a small WSGI project at my university. Since then
the project was grown up every year. Some classes have more than 600 lines
of code with (incl. boiler-plates mostly in descriptors/properties).

Many of these properties are similar or have depencies among themselves.
The idea is to grouping similar properties like:

new style:
----------
{'complex''data','string')}

I wrote this descriptor:

class Descr:

def __init__(self, value):
self.attribute = self.__class__.__name__
self.__set__(None, value)

def __get__(self, obj, Type=None):
return getattr(obj, self.attribute, self)

def __set__(self, obj, value):
if obj is None: # descripting yourself
# do something here ...
self.value = value
else:
if hasattr(obj, self.attribute):
self.__get__(obj).__set__(None, value)
else:
setattr(obj, self.attribute, type(self)(value))

You must not store per-object data in the descriptor. I suggest a naming
convention (the internal data for obj.attr is stored in obj._attr) together
with a value class that handles breaking of the string into attributes of an
instance of itself:

class StructuredAttribute:
def __init__(self, name, make_default):
self.name = name
self.make_default = make_default

def __get__(self, obj, type=None):
if obj is None:
return self
_name = "_" + self.name
try:
return getattr(obj, _name)
except AttributeError:
setattr(obj, _name, self.make_default())
return getattr(obj, _name)

def __set__(self, obj, value):
self.__get__(obj).update(value)


class Value:
def __init__(self, value):
self.update(value)
def update(self, value):
if isinstance(value, str):
self.value, sep, rest = value.partition(";")
self.extras = dict(item.partition("#")[::2] for item in
rest.split())
else:
self.value = value.value
self.extras = value.extras
def __repr__(self):
return repr("{}; {}".format(self.value, " ".join("{}:
{}".format(*item) for item in self.extras.items())))

def make_default_value():
return Value("some; complex:data#string")

class A:
attr = StructuredAttribute("alpha", make_default_value)

def show(obj):
print("attr:", obj.attr)
print("attr.value:", obj.attr.value)
print("attr.extras:", obj.attr.extras)

a = A()
show(a)
newvalue = "whatever"
print("updating value to", newvalue)
a.attr.value = newvalue
show(a)

That's the general idea if you want "setattr polymorphism". Personally I
would go with simpler standard attributes:

class A:
def __init__(self):
self.attr = Value(...)

a = A()
a.value = Value(...)
a.value.extras = ...

 

[CWr]

Peter, thanks for your response.
Sure, you are right when you say that's easier to use standard attribute assigning via __init__.

But my intention was:
- reducing the complexiticity of __init__
- avoiding boiler-plates (mostly property descriptors inside of the main class)
- creating instances (for complex data strings) only if they will be needed, otherwise use default instances (descriptors)
- make it prossible that the data structure can be used in static context -like MyClass.attr - to get default values

Standard procedure:

vs:

Probably it will be necessary to set the attribute at first access. Alternatively it may be possible to observe the descriptor til an attribute will be setted e.g. instance.attr.value = 'whatever'. At this point a new instance (like Value) should be created on obj.__dict__.

It's the procedure what I'm looking for.

Kind Regards,
Chris

 

[Terry Reedy]

Peter, thanks for your response.
Sure, you are right when you say that's easier to use standard attribute assigning via __init__.

But my intention was:
- reducing the complexiticity of __init__
- avoiding boiler-plates (mostly property descriptors inside of the main class)
- creating instances (for complex data strings) only if they will be needed, otherwise use default instances (descriptors)
- make it prossible that the data structure can be used in static context - like MyClass.attr - to get default values

Standard procedure:

"if two is not None:" reads better and is the preferred form.
'is not' is a single comparison operator, just like '!=' and 'not in'.
The current CPython AST or peephole optimizer happens to notice that the
'is' operator followed by the 'not' operator can be replaced by the 'is
not' operator, but this is not guaranteed for all implementations.

self.two = Value(two if two is not None else self.DEFAULT_4_TWO)

There is no need to introduce the new name DEFAULT_4_TWO. It is a
symptom of using the wrong namespace to get the default.

class C:
two = <default>
def __init__(self, two=None):
self.two = Value(two if two is not None else C.two)