design question: no new attributes

[Michele Simionato]

I have a class whose instances should only receive attribute
assignments for attributes that were created at inititialization.
If slots are not appropriate, what is the Pythonic design for this?

Thanks,
Alan Isaac

There is a Cookbook recipe for that:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/252158

Michele Simionato

 

[Alan Isaac]

I don't share your definition of "reasonable". But you should have
guessed by now

My view may be shaped by a different experience.
I have found dynamic attribute creation convenient
when doing something "quick and dirty", but I have
never wanted it as a design feature in a more serious
project.

Can you give me an example where design considerations
make dynamic attribute creation particularly desirable?

Thank you,
Alan Isaac

 

[Alan Isaac]

There's a problem with that when you want to subclass:

Agreed. The following addresses that and, I think, some
of the other objections that have been raised.
Alan

class Lockable:
a = 0
def __init__(self, lock=False):
self.b = 1
self.lock = lock
def __setattr__(self, attr, val):
if not hasattr(self,attr) and hasattr(self,'lock') and self.lock:
raise ValueError("This object accepts no new attributes.")
self.__dict__[attr] = val

 

[Paul Boddie]

Yup, and you are free to use one of them. And as an additional benefit, they
will be more performant because you then can optimize the code further.

I think that's something many people miss: it can be desirable to
declare the range of attributes on instances "up front" for
optimisation purposes (in various languages other than Python), but
the advantages of not risking the occasional AttributeError shouldn't
be too readily discarded either. I'm not sure where the different
tools stand on this front, but second-guessing invalid accesses to
attributes is a completely different level to just identifying dodgy
accesses to globals or locals, along with dubious imports and "bad
practice" programming styles consisting of one or more of those more
detectable things.

Paul

 

[Bruno Desthuilliers]

Alan Isaac a écrit :

My view may be shaped by a different experience.
I have found dynamic attribute creation convenient
when doing something "quick and dirty", but I have
never wanted it as a design feature in a more serious
project.

Can you give me an example where design considerations
make dynamic attribute creation particularly desirable?

Let's consider an imaginary ORM (or it could be an "object-ldap mapper",
or just any other stuff where it may be useful to have 'smart
attributes', like Plone's Archetype or FormEncode or...):

class Person(Schema):
firstname = TextField(empty=False, format=str.capitalize)
lastname = TextField(empty=False, format=str.upper)
nickname = TextField(empty=False, format=str.lower, dbfield="login")

p = Person(firstname="joe", lastname="Hacker")

p.firstname
=> "Joe"
p.nickname
=> "joe"

p.fields['firstname'].dbfield
=> 'firstname'

I let you try to write the SchemaType metaclass without dynamically
adding attributes to either the Person class (remember that classes are
objects too) and/or the TextField instances...


Let's consider the controller part of a web MVC application. You want to
manage security in a declarative way.

class PageController(Controller):
@requires_perm('ViewContent')
def view(self):
# code here

@requires_perm('EditContent'):
def edit(self):
# code here

etc...

What do you think this decorator will do ? yes, that's right: annotate
the action methods with needed perms, so the relevant part of the
framework can take appropriate action...



As a side note : in Python, *all* attributes are "dynamically created".
Class attributes are dynamically added to the class object by it's
metaclass, instance attributes are dynamically added to the instance,
usually - but not necessarily - by the __init__ method. Now the __init__
method is nothing more than a wrapper around the __init__ function,
which itself takes the (bare) instance as argument. Writing methods
within the class statement body makes things clearer, but is by no mean
mandatory:

class Foo(object):
pass

def init_foo(foo, name):
foo.name = name

Foo.__init__ = init_foo


Like a lot of other tools (HOFs, metaclasses, descriptors, generators
etc), dynamic attribute creation looks mostly like a useless gadget when
you never used it before - because it's not part of your "mind map", you
don't naturally use it in your design.

My 2 cents

 

[Alan Isaac]

The security application seems to call for roles.
I'll have to think about the schema example.
But in any case, my question was poorly stated.
I started out wanting to trap was the dynamic
addition of attributes to class instances after
initialization. While you responded to my later question
as asked, the response does not I think take this context
into account.

As a side note : in Python, *all* attributes are "dynamically created".
... Writing methods
within the class statement body makes things clearer, but is by no mean
mandatory:

Point taken. And I understand it means that my talk of
"after initialization" above contains some ambiguity.

One last point. While I remain interested in examples of how
"late" addition of attributes to class instances is useful,
I must note that everyone who responded agreed that it
has been a source of bugs. This seems to argue against a
general ban on "locking" objects in some way, in some
circumstances.

Thanks!
Alan

 

[Diez B. Roggisch]

One last point. While I remain interested in examples of how
"late" addition of attributes to class instances is useful,
I must note that everyone who responded agreed that it
has been a source of bugs. This seems to argue against a
general ban on "locking" objects in some way, in some
circumstances.

I've got some examples. I quite often used dynamic attributes in
situations where I wasn't the creator of certain objects, but had to use
them and wanted to link them with my own code.

For example, in various GUI-environments, you've got some sort of tree
view, usually containing item-objects that form the tree.

Now in other languages like C++ or Java, I have to subclass those items
(if possible at all) if I want to associate my own data (for example the
"real" object behind such an item) with them. Or I need to have an
awkward global mapping between tree-items and my data.

But in python, I can just easily add a new .data_object property, and in
whatever event-handler spits a e.g. selected item at me, I'm easily
accessing the data behind it.

Other examples are processing XML-trees that also have associated data
and the like.

Diez

 

[Bruno Desthuilliers]

Alan Isaac a écrit :

The security application seems to call for roles.

Considering that a role is a set of permissions in a context. The
application doesn't want to say "you need this role", but "you need to
have a role that has this permission" (nb : this is based on Zope's
security model).

I'll have to think about the schema example.
But in any case, my question was poorly stated.
I started out wanting to trap was the dynamic
addition of attributes to class instances after
initialization.

From withing methods of the class, or from the outside world ? In the
first case, pylint is what you're looking for.

While you responded to my later question
as asked, the response does not I think take this context
into account.

Of course - this was not what I was answering to !-)

Now, in the above context, I still don't think your solution is the best
option.

Point taken. And I understand it means that my talk of
"after initialization" above contains some ambiguity.

Indeed !-)

One last point. While I remain interested in examples of how
"late" addition of attributes to class instances is useful,

Then read the sources of packages / frameworks like SQLAlchemy,
SQLObject, FormEncode, CherryPy, TurboGears, Django, etc - and probably
a lot of other.

Heck, FWIW, modules are objects too - instances of class 'module', and
all the top-level names of a module are de facto examples of "late
addition of attributes to a class instance" !-)

I must note that everyone who responded agreed that it
has been a source of bugs.

Yes, of course - typos happens. But I've seen much more bugs introduced
by all the boilerplate and complexities needed to work around arbitrary
restrictions imposed by some well-known mainstream languages.

My experience with Python is that typos, type errors and all this class
of programmer error are usually *very* quickly noticed (ie : within
minutes).

This seems to argue against a
general ban on "locking" objects in some way, in some
circumstances.

I'd rather go for better (well... pylint is already a pretty nice tool)
code checkers.

My 2 cents...

 

[Arnaud Delobelle]

I have a class whose instances should only receive attribute
assignments for attributes that were created at inititialization.
If slots are not appropriate, what is the Pythonic design for this?

Hi !

Even though a lot of people have argued against such a thing, I have
been thinking about this last night and I have the following hack.
Classes descending from SuspiciousObject below won't allow new
attributes to be added to their instances apart from within trusted
methods (i.e. methods decorated with @trustedmethod)

Note: this is probably not of great interest but I've decided to share
it since I did this as a result of reading this thread

class LockableDict(dict):
"A dict where addition of new keys can be prevented by setting the
locked attribute"
__slots__ = ('locked',)
def __init__(self, locked=False):
self.locked = locked
def force_setitem(self, key, value):
super(LockableDict, self).__setitem__(key, value)
def __setitem__(self, key, value):
if self.has_key(key) or not self.locked:
self.force_setitem(key, value)
else:
raise KeyError, key

def trustedmethod(f):
def pf(self, *args, **kwargs):
was_locked = self.__dict__.locked
self.__dict__.locked = False
try:
return f(self, *args, **kwargs)
finally:
self.__dict__.locked = was_locked
return pf

class SuspiciousObject(object):
def __new__(cls, *args, **kwargs):
self = object.__new__(cls)
super(SuspiciousObject, self).__setattr__('__dict__',
LockableDict(locked=True))
return self
def __setattr__(self, attr, value):
try:
self.__dict__[attr] = value
except KeyError:
raise AttributeError, "'%s' object has no attribute '%s" %
(type(self).__name__, attr)

# Instances of SuspiciousObject refuse anyone the right to create a
new attribute apart from methods marked with the decorator
@trustedmethod.
# Example:

class Foo(SuspiciousObject):
@trustedmethod
def __init__(self):
self.bar = 2
self.baz = 'Hello'
def foobar(self, v):
self.fubar = v
@trustedmethod
def force_setattr(self, attr, val=None):
setattr(self, attr, val)

This would give something like:
....
AttributeError: 'Foo' object has no attribute 'fubar....
AttributeError: 'Foo' object has no attribute 'fubar

foo.__dict__['fubar']=4 # Neither will this

....
KeyError: 'fubar'....
AttributeError: 'Foo' object has no attribute 'fubar
By creating a custom metaclass for SuspiciousObject, it would be easy
to make SuspiciousObjects trust their own methods by default instead
of having to declare which method are trusted.

 

[Harold Fellermann]

Hi Alan,

One last point. While I remain interested in examples of how
"late" addition ofattributesto class instances is useful,
I must note that everyone who responded agreed that it
has been a source of bugs. This seems to argue against a
general ban on "locking" objects in some way, in some
circumstances.

If you want to restrict "late" addition of attributes, no-one will
prevent you to do so.
Arnaud has already given you an example implementation. Here is mine:
....
.... _locked = False
....
.... def __setattr__(self,attr,var) :
.... if self._locked and not attr in dir(self):
.... raise RuntimeError
.... else :
.... object.__setattr__(self,attr,var)
....
.... def lock(self) :
.... self._locked = True
....Traceback (most recent call last):
File "<stdin>", line 1, in ?


See how it works? The lock method *dynamically* adds the attribute
foo._locked *after* initialization to the instance. Before the call
of foo.lock() foo._locked is a class attribute. Now you might argue
that one should better set foo._locked = False in the __init__ method
rather than as a class attribute. Something like:

class Foo(object) :
def __init__(self) :
self._locked = False

But no! The initialization would trigger
Foo.__setattr__(foo,'_locked',False)
which naturally runs into an attribute error since __setattr__ looks
up this
attribute. So this very same implementation is one of the pro examples
you
asked for


cheers,

- harold -