Public attributes with really private data

[Mark Summerfield]

Hi,

I had a quick search & didn't find anything _nice_ that produced
attributes with really private data, so I came up with a possible
solution---for Python 3.
(For Python 2 there does seem to be an approach although I'm not
keen on it:
http://www.builderau.com.au/blogs/byteclub/viewblogpost.htm?p=339270875
)

Here's a standard class with one read-only and one writable
property that has a tiny bit of validation.

class P:
def __init__(self, w):
self.w = w
@property
def r(self): return 5
@property
def w(self): return self.__w
@w.setter
def w(self, value):
if value > 0: # Only +ve values allowed
self.__w = value
else:
raise ValueError("'{0}' is not valid for w".format(value))

The read-only property is completely private because it isn't
actually stored as such.

But if we do dir() on an instance, in addition to 'r' and 'w', we
also have '_P__w'. So the writable property's data is easily
accessible, and the validation can be got around:
Traceback (most recent call last):
....
ValueError: '-7' is not valid for w(5, -7)

Here's a class where I can't think of a way to access the private
data and set invalid values.

class A:
r = Attribute("r", 5)
w = Attribute("w", None, lambda self, value: value > 0)
def __init__(self, w):
self.w = w

The Attribute class is a descriptor that takes three arguments:
name of attribute, initial value (essential for read-only
attributes!), and a validator function (which could be "lambda
*a: True" if any value is accepatble).
Traceback (most recent call last):
....
ValueError: '-7' is not valid for w

If we do a dir(a) the only attributes we get (beyond those from
object) are 'r' and 'w', so it shouldn't be possible to get
around the validation---at least not easily.

Here's a rough & ready implementation of the Attribute class:

class Attribute:
def __init__(self, name, first_value=None, validator=None):
self.__name__ = name
hidden_value = first_value
self.__getter = lambda self: hidden_value
if validator is not None:
def set(self, value):
if validator(self, value):
nonlocal hidden_value
hidden_value = value
else:
raise ValueError("'{0}' is not valid for
{1}".format(value, name))
self.__setter = set
else:
self.__setter = None
def __get__(self, instance, owner=None):
if instance is None:
return self
return self.__getter(instance)
def __set__(self, instance, value):
if self.__setter is None:
raise AttributeError("'{0}' is read-only".format(
self.__name__))
return self.__setter(instance, value)

The key to making the attribute data private is that it is held
as part of a closure's state. Notice that nonlocal is needed,
so you need Python 3.

 

[Peter Otten]

I had a quick search & didn't find anything _nice_ that produced
attributes with really private data, so I came up with a possible
solution---for Python 3.

Do really you think what you suggest below is "nice"?

By the way, your Attribute descriptor stores the value for all instances of
A in the same variable...

Peter

 

[Carl Banks]

Hi,

I had a quick search & didn't find anything _nice_ that produced
attributes with really private data, so I came up with a possible
solution---for Python 3.
(For Python 2 there does seem to be an approach although I'm not
keen on it:http://www.builderau.com.au/blogs/byteclub/viewblogpost.htm?p=339270875
)

Here's a standard class with one read-only and one writable
property that has a tiny bit of validation.

class P:
    def __init__(self, w):
        self.w = w
    @property
    def r(self): return 5
    @property
    def w(self): return self.__w
    @w.setter
    def w(self, value):
        if value > 0: # Only +ve values allowed
            self.__w = value
        else:
            raise ValueError("'{0}' is not valid for w".format(value))

The read-only property is completely private because it isn't
actually stored as such.

But if we do dir() on an instance, in addition to 'r' and 'w', we
also have '_P__w'. So the writable property's data is easily
accessible, and the validation can be got around:


Traceback (most recent call last):
...
ValueError: '-7' is not valid for w>>> p._P__w = -7

(5, -7)

Here's a class where I can't think of a way to access the private
data and set invalid values.

class A:
    r = Attribute("r", 5)
    w = Attribute("w", None, lambda self, value: value > 0)
    def __init__(self, w):
        self.w = w

The Attribute class is a descriptor that takes three arguments:
name of attribute, initial value (essential for read-only
attributes!), and a validator function (which could be "lambda
*a: True" if any value is accepatble).


Traceback (most recent call last):
...
ValueError: '-7' is not valid for w

If we do a dir(a) the only attributes we get (beyond those from
object) are 'r' and 'w', so it shouldn't be possible to get
around the validation---at least not easily.

Ok, I'll bite.

A.w = -7

A.__class__ = A_without_validation
a.w = -7

Attribute.__set__ = function_that_ignores_validation
a.w = -7


I am not opposed to adding a little well-considered protection to some
attributes where mistakes are prone to happen and/or dangerous, but it
is futile to try to stop access entirely. There's just too many back
doors.

I would suggest that there really isn't much point in anything more
complex than your first solution, which was to validate with
properties and to store the value in a separate attribute.
Respectable programmers won't lightly bypass your validation if they
see that you've taken steps to enforce it. OTOH, once a programmer,
respectable or not, decides to override your protection, they are not
likely to be stopped by something more complex. So there is no point
in making it more complex.


Carl Banks

 

[Mark Summerfield]

MarkSummerfieldwrote:

Do really you think what you suggest below is "nice"?

Well the code isn't ugly and doesn't mess with the call stack etc.

By the way, your Attribute descriptor stores the value for all instances of
A in the same variable...

It seems like it does, but it doesn't. The hidden_value is an instance
variable that is created every time an Attribute object is created.
a = Attribute("a", 5, lambda *a: True)
b = Attribute("b", 5, lambda *a: True) a = Attribute("a", 5, lambda *a: True)
b = Attribute("b", 5, lambda *a: True)(1, 2, 3, 4)

 

[Mark Summerfield]

Ok, I'll bite.

A.w = -7

A.__class__ = A_without_validation
a.w = -7

Attribute.__set__ = function_that_ignores_validation
a.w = -7

I don't see how that can work? Sure you could replace the __set__
method, or the Attribute.__setter method. But neither can actually
change the hidden_value because AFAIK that only exists within the
scope of a closure and so isn't accesssible. (Well, maybe if you
started using the inspect module you'd find it.)

I am not opposed to adding a little well-considered protection to some
attributes where mistakes are prone to happen and/or dangerous, but it
is futile to try to stop access entirely.  There's just too many back
doors.

Sure, but I like the fact that there is no "shadow" attribute, so just
"w" not "w" _and_ "_A__w".

I would suggest that there really isn't much point in anything more
complex than your first solution, which was to validate with
properties and to store the value in a separate attribute.
Respectable programmers won't lightly bypass your validation if they
see that you've taken steps to enforce it.  OTOH, once a programmer,
respectable or not, decides to override your protection, they are not
likely to be stopped by something more complex.  So there is no point
in making it more complex.

Carl Banks

I think that's a fair viewpoint, but I still like the idea of there
being no _A__w available for property w.

 

[Mark Summerfield]

[snip]

By the way, your Attribute descriptor stores the value for all instances of
A in the same variable...

Peter

You're right.

 

[Peter Otten]

Well the code isn't ugly and doesn't mess with the call stack etc.


It seems like it does, but it doesn't. The hidden_value is an instance
variable that is created every time an Attribute object is created.

a = Attribute("a", 5, lambda *a: True)
b = Attribute("b", 5, lambda *a: True)
a = Attribute("a", 5, lambda *a: True)
b = Attribute("b", 5, lambda *a: True)
(1, 2, 3, 4)

But attribute values are shared between all instances of the same class:
.... x = Attribute("x", 42, lambda *a: True)
....('yadda', 'yadda')

Peter

 

[Mark Summerfield]

MarkSummerfieldwrote:



But attribute values are shared between all instances of the same class:


...     x = Attribute("x", 42, lambda *a: True)
...>>> a = A()

('yadda', 'yadda')

Peter

Yes. I did think of trying to create the closures dynamically the
first time the getter or setter was called---but that _is_ getting
ugly, so I'll give it up

Thanks!

 

[Mark Summerfield]

But attribute values are shared between all instances of the same class:


...     x = Attribute("x", 42, lambda *a: True)
...>>> a = A()

('yadda', 'yadda')

Peter

OK, I couldn't quite give it up. But the solution isn't nice or good.
It does work, but at the cost of an extra dictionary lookup on every
get or set, plus a dictionary to hold all the getters & setters. I
_don't recommend it_, but here it is anyway. I've done with it now

class Attribute:

__accessors = {}

def __init__(self, name, first_value=None, validator=None):
self.name = name
self.first_value = first_value
self.validator = validator

def __get__(self, instance, owner=None):
if instance is None:
return self
if (id(instance), self.name) not in self.__accessors:
self.__makeAccessors(instance)
return self.__accessors[id(instance), self.name][0](instance)


def __set__(self, instance, value):
if (id(instance), self.name) not in self.__accessors:
self.__makeAccessors(instance)
setter = self.__accessors[id(instance), self.name][1]
if setter is None:
raise AttributeError("'{0}' is read-only".format(
self.__name__))
return setter(instance, value)


def __makeAccessors(self, instance):
hidden_value = self.first_value
getter = lambda self: hidden_value
if self.validator is not None:
def set(instance, value):
if self.validator(instance, value):
nonlocal hidden_value
hidden_value = value
else:
raise ValueError("'{0}' is not valid for {1}"
.format(value, name))
setter = set
else:
setter = None
self.__accessors[id(instance), self.name] = (getter, setter)