stackoverflow question

[Ethan Furman]

Hey all!

I posted a question/answer on SO earlier, but there seems to be some
confusion around either the question or the answer (judging from the
comments).

http://stackoverflow.com/q/9638921/208880

If anyone here is willing to take a look at it and let me know if I did
not write it well, I would appreciate the feedback.


Here's the question text:
------------------------
I'm writing a metaclass to do some cool stuff, and part of its
processing is to check that certain attributes exist when the class is
created. Some of these are mutable, and would normally be set in
`__init__`, but since `__init__` isn't run until the instance is created
the metaclass won't know that the attribute *will* be created, and
raises an error. I could do something like:

class Test(meta=Meta):
mutable = None
def __init__(self):
self.mutable = list()

But that isn't very elegant, and also violates DRY.

What I need is some way to have:

class Test(metaclass=Meta):
mutable = list()

t1 = Test()
t2 = Test()
t1.mutable.append('one')
t2.mutable.append('two')
t1.mutable # prints ['one']
t2.mutable # prints ['two']

Any ideas on how this can be accomplished?

Also, the metaclass doing the checking doesn't care what type of object
the attribute is, only that it is there.
---------------------------

and the answer text:
-------------------
There are a couple ways to do this:

1. Have the metaclass check all the attributes, and if they are one
of the mutables (`list`, `dict`, `set`, etc.) replace the attribute with
a descriptor that will activate on first access and update the instance
with a fresh copy of the mutable.

2. Provide the descriptor from (1) as a decorator to be used when
writing the class.

I prefer (2) is it gives complete control to the class author, and
simplifies those cases where the class-level mutable attribute *should*
be shared amongst all the instances.

Here's the decorator-descriptor:

class ReplaceMutable:
def __init__(self, func):
self.func = func
def __call__(self):
return self
def __get__(self, instance, owner):
if instance is None:
return self
result = self.func()
setattr(instance, self.func.__name__, result)
return result

and the test class:

class Test:
@ReplaceMutable
def mutable():
return list()

t1 = Test()
t2 = Test()
t1.mutable.append('one')
t2.mutable.append('two')
print(t1.mutable)
print(t2.mutable)

How it works:

Just like `property`, `ReplaceMutable` is a descriptor object with the
same name as the attribute it is replacing. Unlike `property`, it does
not define `__set__` nor `__delete__`, so when code tries to rebind the
name (`mutable` in the test above) in the instance Python will allow it
to do so. This is the same idea behind caching descriptors.

`ReplaceMutable` is decorating a function (with the name of the desired
attribute) that simply returns whatever the instance level attribute
should be initialized with (an empty `list` in the example above). So
the first time the attribute is looked up on an instance it will not be
found in the instance dictionary and Python will activate the
descriptor; the descriptor then calls the function to retrieve the
initial object/data/whatever, stores it in the instance, and then
returns it. The next time that attribute is accessed *on that instance*
it will be in the instance dictionary, and that is what will be used.

 

[Owen Jacobson]

Hey all!

I posted a question/answer on SO earlier, but there seems to be some
confusion around either the question or the answer (judging from the
comments).

http://stackoverflow.com/q/9638921/208880

If anyone here is willing to take a look at it and let me know if I did
not write it well, I would appreciate the feedback.


Here's the question text:
------------------------
I'm writing a metaclass to do some cool stuff, and part of its
processing is to check that certain attributes exist when the class is
created. Some of these are mutable, and would normally be set in
`__init__`, but since `__init__` isn't run until the instance is
created the metaclass won't know that the attribute *will* be created,
and raises an error. I could do something like:

class Test(meta=Meta):
mutable = None
def __init__(self):
self.mutable = list()

But that isn't very elegant, and also violates DRY.

What I need is some way to have:

class Test(metaclass=Meta):
mutable = list()

t1 = Test()
t2 = Test()
t1.mutable.append('one')
t2.mutable.append('two')
t1.mutable # prints ['one']
t2.mutable # prints ['two']

Any ideas on how this can be accomplished?

Also, the metaclass doing the checking doesn't care what type of object
the attribute is, only that it is there.
---------------------------

Why check what you can ensure? The __init__ function your metaclass
passes to type() doesn't have to be the __init__ method your metaclass
received. Consider the following:

import functools as f

def make_init(real_init):
"""Define an __init__ method that ensures ``self.mutable`` is set. If the
passed ``real_init`` function later replaces ``self.mutable``, that value
is preserved; otherwise, ``self.mutable`` is set to a new, empty list.

Arguments to the generated ``__init__`` method are passed to the original
``real_init`` unchanged.
"""
def __init__(self, *args, **kwargs):
self.mutable = list()
if real_init is not None:
return real_init(self, *args, **kwargs)
if real_init is not None:
f.update_wrapper(__init__, real_init)
return __init__

class Meta(type):
def __new__(meta, name, parents, attributes):
attributes['__init__'] = make_init(attributes.get('__init__', None))
return type.__new__(Meta, name, parents, attributes)

class C(object):
__metaclass__ = Meta

a, b = C(), C()

a.mutable.append(3)
b.mutable.append(5)

a.mutable [3]
b.mutable

[5]

All instances of classes whose metaclass is Meta will, guaranteed, have
an instance field named 'mutable'. Its value is a list created at
instance creation time, unless the instance's __init__ provides a
different value.

What've I missed?

-o

 

[Ethan Furman]

Hey all!

I posted a question/answer on SO earlier, but there seems to be some
confusion around either the question or the answer (judging from the
comments).

http://stackoverflow.com/q/9638921/208880

If anyone here is willing to take a look at it and let me know if I
did not write it well, I would appreciate the feedback.


Here's the question text:
------------------------
I'm writing a metaclass to do some cool stuff, and part of its
processing is to check that certain attributes exist when the class is
created. Some of these are mutable, and would normally be set in
`__init__`, but since `__init__` isn't run until the instance is
created the metaclass won't know that the attribute *will* be created,
and raises an error. I could do something like:

class Test(meta=Meta):
mutable = None
def __init__(self):
self.mutable = list()

But that isn't very elegant, and also violates DRY.

What I need is some way to have:

class Test(metaclass=Meta):
mutable = list()

t1 = Test()
t2 = Test()
t1.mutable.append('one')
t2.mutable.append('two')
t1.mutable # prints ['one']
t2.mutable # prints ['two']

Any ideas on how this can be accomplished?

Also, the metaclass doing the checking doesn't care what type of
object the attribute is, only that it is there.
---------------------------

Why check what you can ensure? The __init__ function your metaclass
passes to type() doesn't have to be the __init__ method your metaclass
received. Consider the following:

import functools as f

def make_init(real_init):
"""Define an __init__ method that ensures ``self.mutable`` is
set. If the
passed ``real_init`` function later replaces ``self.mutable``,
that value
is preserved; otherwise, ``self.mutable`` is set to a new, empty
list.

Arguments to the generated ``__init__`` method are passed to the
original
``real_init`` unchanged.
"""
def __init__(self, *args, **kwargs):
self.mutable = list()
if real_init is not None:
return real_init(self, *args, **kwargs)
if real_init is not None:
f.update_wrapper(__init__, real_init)
return __init__

class Meta(type):
def __new__(meta, name, parents, attributes):
attributes['__init__'] =
make_init(attributes.get('__init__', None))
return type.__new__(Meta, name, parents, attributes)

class C(object):
__metaclass__ = Meta

a, b = C(), C()

a.mutable.append(3)
b.mutable.append(5)

a.mutable [3]
b.mutable

[5]

All instances of classes whose metaclass is Meta will, guaranteed, have
an instance field named 'mutable'. Its value is a list created at
instance creation time, unless the instance's __init__ provides a
different value.

The idea is good. The devil is in the details, as usual. How is the
metaclass going to know:

1) which attributes to replace
2) what to replace them with?

~Ethan~

 

[Owen Jacobson]

Hey all!

I posted a question/answer on SO earlier, but there seems to be some
confusion around either the question or the answer (judging from the
comments).

http://stackoverflow.com/q/9638921/208880

If anyone here is willing to take a look at it and let me know if I did
not write it well, I would appreciate the feedback.


Here's the question text:
------------------------
I'm writing a metaclass to do some cool stuff, and part of its
processing is to check that certain attributes exist when the class is
created. Some of these are mutable, and would normally be set in
`__init__`, but since `__init__` isn't run until the instance is
created the metaclass won't know that the attribute *will* be created,
and raises an error. I could do something like:

class Test(meta=Meta):
mutable = None
def __init__(self):
self.mutable = list()

But that isn't very elegant, and also violates DRY.

What I need is some way to have:

class Test(metaclass=Meta):
mutable = list()

t1 = Test()
t2 = Test()
t1.mutable.append('one')
t2.mutable.append('two')
t1.mutable # prints ['one']
t2.mutable # prints ['two']

Any ideas on how this can be accomplished?

Also, the metaclass doing the checking doesn't care what type of object
the attribute is, only that it is there.
---------------------------

Why check what you can ensure? The __init__ function your metaclass
passes to type() doesn't have to be the __init__ method your metaclass
received.

[… __init__-generation technique elided …]

The idea is good. The devil is in the details, as usual. How is the
metaclass going to know:

1) which attributes to replace
2) what to replace them with?

I can think of at least three techniques; others are certainly possible:

1. As with the example code, the list is hard-coded in the metaclass's
source code.
2. The list (or, rather, a dictionary) is drawn from a class attribute
of the class being created:

class Foo(object):
mandatory_fields = {'mutable': list, 'more_stuff': str}
__metaclass__ = IntrospectingMetaclass

3. A metaclass-returning factory produces new metaclasses on demand,
each of which has a dict of mandatory fields baked into it. (This is a
hybrid of the two approaches, and can easily have some messy side
effects on your app's type ecology if used carelessly.)

Of course, you can also treat this the other way around: instead of
enforcing that instances have specific fields, you could have users of
those instances be aware that the field might not exist, and wrap
access to the field in a try/except NameError block or use getattr to
read the attribute.

What's appropriate really depends on how you plan to use this
metaclass, and on the nature of the higher-level problem to which "I
know, I'll use metaclasses" is your answer. How about telling us a
slightly broader story about your problem?

-o