Inheritable Slots Metaclass

[Rebel Lion]

I made a metaclass to inherit __slots__ automatically.

I think this feature should be included in builtin object's metaclass.

You can now write:

class Foo(object):
__metaclass__ = SlotMetaclass

@slot
def foo():
pass


class Bar(Foo):

@slot
def bar():
pass

foo = Foo()
foo.foo = 1
bar = Bar()
bar.bar = 1
bar.foo = 1
try:
bar.baz = 1 # this should fall
except AttributeError, e:
print 'yeah', e


Instead of

class Foo(object):
__slots__ = ['foo']
class Bar(Foo)
__slots__ = ['foo', 'bar']


Please discuss the pros & cons for this feature.


Here is the metaclass:

class slot(object):
"""Slot Decorator"""

def __init__(self, func):
pass


class SlotMetaclass(type):
"""Inheritable Slots Metaclass"""

def __new__(cls, name, bases, attrs):
# make a normal class, and get its attributes to decide which
ones are slots
tmpcls = type.__new__(cls, name, bases, attrs)
slots = []
for k in dir(tmpcls):
v = getattr(tmpcls, k)
if isinstance(v, slot):
slots.append(k)
# clean up
del tmpcls
for x in slots:
del attrs[x]
# create the real class with __slots__
attrs['__slots__'] = slots
return type.__new__(cls, name, bases, attrs)

 

[Carl Banks]

I made a metaclass to inherit __slots__ automatically.

I think this feature should be included in builtin object's metaclass.

I'd be ok with a metatype in the standard library that makes slots
more transparent, but since slots are intended as an optimization, it
doesn't really need (and might be detrimental to have) transparency
for ordinary objects.

However, Aahz will be by shortly to tell you never to use slots.

You can now write:

    class Foo(object):
        __metaclass__ = SlotMetaclass

        @slot
        def foo():
            pass

    class Bar(Foo):

        @slot
        def bar():
            pass

    foo = Foo()
    foo.foo = 1
    bar = Bar()
    bar.bar = 1
    bar.foo = 1
    try:
        bar.baz = 1 # this should fall
    except AttributeError, e:
        print 'yeah', e

Instead of

    class Foo(object):
          __slots__ = ['foo']
    class Bar(Foo)
          __slots__ = ['foo', 'bar']

Please discuss the pros & cons for this feature.

It seems like a good approach, but the use of decorators to define
slots is hideous. I'd recommend creating slots with simple assignment
instead:

slot = object()

class Foo():
__metaclass__ = SlotsMetaclass
foo = slot

class Bar():
bar = slot

Then replace

"isinstance(v,slot)" with "v is slot"

in the metaclass.

Here is the metaclass:

class slot(object):
    """Slot Decorator"""

    def __init__(self, func):
        pass

class SlotMetaclass(type):
    """Inheritable Slots Metaclass"""

    def __new__(cls, name, bases, attrs):
        # make a normal class, and get its attributes to decide which
ones are slots
        tmpcls = type.__new__(cls, name, bases, attrs)
        slots = []
        for k in dir(tmpcls):
            v = getattr(tmpcls, k)
            if isinstance(v, slot):
                slots.append(k)
        # clean up
        del tmpcls
        for x in slots:
            del attrs[x]
        # create the real class with __slots__
        attrs['__slots__'] = slots
        return type.__new__(cls, name, bases, attrs)

You don't need to create a temporary class here; you should loop
through the base classes and inspect their slots. Also, you don't
need to recreate slots that exist in base classes, so you could just
get all the new slots from the class dict.

You should consider corner cases if you think it should be standard.
For instance, what if a derived class defined the same slot as a a
base class--it'll create a new slot in yours but it shouldn't. What
if you want to create a subclass that does have a dict? Can't do it
in your version. You need to consider stuff like that.


Carl Banks

 

[geremy condra]

I'd be ok with a metatype in the standard library that makes slots
more transparent, but since slots are intended as an optimization, it
doesn't really need (and might be detrimental to have) transparency
for ordinary objects.

However, Aahz will be by shortly to tell you never to use slots.

Ok, I'll bite: why?

Geremy Condra

 

[Carl Banks]

Ok, I'll bite: why?

I was just razzing him a bit.


Carl Banks

 

[Rebel Lion]

I'd be ok with a metatype in the standard library that makes slots
more transparent, but since slots are intended as an optimization, it
doesn't really need (and might be detrimental to have) transparency
for ordinary objects.

But why there is __slots__ if it's not indeed needed. we should make
it more usable instead of a hack, which it now looks like.

However, Aahz will be by shortly to tell you never to use slots.

Obviously he didn't work on graphs, at least huge graphs.

It seems like a good approach, but the use of decorators to define
slots is hideous.  I'd recommend creating slots with simple assignment
instead:

slot = object()

class Foo():
    __metaclass__ = SlotsMetaclass
    foo = slot

class Bar():
    bar = slot

Then replace

"isinstance(v,slot)" with "v is slot"

in the metaclass.

This is a style not a problem. My way intents to keep the consistency
with @property

You don't need to create a temporary class here; you should loop
through the base classes and inspect their slots.  

As you can see, I'm very lazy. If we got some decision here, we can
surely make a better implementation that this hack.

Also, you don't
need to recreate slots that exist in base classes, so you could just
get all the new slots from the class dict.

You should consider corner cases if you think it should be standard.
For instance, what if a derived class defined the same slot as a a
base class--it'll create a new slot in yours but it shouldn't.  What
if you want to create a subclass that does have a dict?  Can't do it
in your version.  You need to consider stuff like that.

The most arguable point here. I raise this post inorder to make slot
inheritable.
In most cases, if a class uses slots, it is designed to be heavily
used.
Ans subclass of it should be heavily used too, but in this case you
want to create __slots__,
you have to manually lookup __slots__ of bases.

But your corner is considerable. We can offer a __no_slots__ to
disable slots.

In conclusion, my point is:

If you don't want to use slots in common days, you can subclass a
class which doesn't use slots(in most cases)
If you want to use slots, you need inherit parent classes' slots(in a
few cases)
If you don't want to use slots when you are subclassing a class which
uses slots(in few cases):
use a __noslots__ magic



Regards,

 

[John Nagle]

But why there is __slots__ if it's not indeed needed. we should make
it more usable instead of a hack, which it now looks like.

I'd argue that "slotting" of classes should be automatic, unless a class
1) defines "__setattr__" or 2) performs an operation which visibly
creates a class attribute, like "self[foo] = ...". This would handle
the common use cases. Only classes to which attributes are added from
outside the class would be affected. And really, if you're going to do
that, you should subclass.

John Nagle

 

[Carl Banks]

But why there is __slots__ if it's not indeed needed. we should make
it more usable instead of a hack, which it now looks like.

It is needed--for optimization. The thinking is that for most uses it
is not a good idea to defeat Python's dynamicsm, therefore making it
too easy to add slots would be detrimental to the community because it
would encourage people to use slots even when not needed.

I more or less agree with this, and would be very -1 on any attempt to
mske slots the "default", so to speak; however I don't think it would
make slots dangerously accessible to have a user import a module and
explicitly set the metaclass, for the benefit of getting rid of their
weirdness.

Obviously he didn't work on graphs, at least huge graphs.

There's no need to be presumptuous. You can state your use case with
accusing him of anything.

This is a style not a problem. My way intents to keep the consistency
with @property

It's a problem if you think this metaclass ought to be in the standard
library. This "style" will never, ever make it to the standard
library, ever. I guarantee you.

Frankly, you're observing a pattern where there is none. You see
decorator syntax and you think "Ah, that's the standard way to define
attributes that affect the behavior of a class". It's not.

Descriptor syntax is syntax sugar to make it less unwieldy to apply a
function to a function or class. That's it, that's all it is. If you
don't have a function, there is no reason for a decorator.

You're entitled to your own preferred style, of course, but I'm just
telling you it'll never fly with most people.

As you can see, I'm very lazy. If we got some decision here, we can
surely make a better implementation that this hack.

No, sorry, it doesn't work that way. Unless you're a known genius and/
or a long time contributor, you better have some pretty worked out
code and some persuasive use cases if you want any consideration.
Ideas are cheap.

(If you want some inspiration, go read PEP 389.)

The most arguable point here. I raise this post inorder to make slot
inheritable.

Slots already are inheritable, as I pointed out. It's just weird and
complicated how they affect things.

class A(object): __slots__ = ['a']
class B(A): __slots__ = ['b']
B().a = 1 # this will work

In most cases, if a class uses slots, it is designed to be heavily
used.
Ans subclass of it should be heavily used too, but in this case you
want to create __slots__,
you have to manually lookup __slots__ of bases.

You don't have to, unless you want to work around some of those other
corner cases I spoke of.

But your corner is considerable. We can offer a __no_slots__ to
disable slots.

In conclusion, my point is:

If you don't want to use slots in common days, you can subclass a
class which doesn't use slots(in most cases)
If you want to use slots, you need inherit parent classes' slots(in a
few cases)
If you don't want to use slots when you are subclassing a class which
uses slots(in few cases):
  use a __noslots__ magic

That seems like a reasonable way to handle it.

I don't want to sound to pessimistic about it, I really wouldn't mind
a metaclass that makes slots more normal; but you have work to do.


Carl Banks

 

[Carl Banks]

I don't want to sound to pessimistic about it, I really wouldn't mind
a metaclass that makes slots more normal; but you have work to do.

Just as a minor followup, I'll mention that slots and inheritance have
some issues that aren't going to be resolvable to everyone's
satisfaction, and that's another reason why I don't think slots should
be accessible by default.

For instance, some might assume that deriving from a class with slots
automatically grants the derived class the same optimization that the
base class has. Others will not even consider that the base class has
slots and will assume that their derived class has dynamic attributes
like most other Python classes. Who's right?

I'd go with the former, for the same reason you gave, but the latter
is not unreasonable. type does the latter.

The best answer might be to always insist on an explicit declaration
one way or another, but that makes it prone to other problems like
forward compatibility issues. Point is, it's not a simple thing.


Carl Banks

 

[Aahz]

However, Aahz will be by shortly to tell you never to use slots.

Please note that there is an important distinction between "don't use
slots" and "never use slots" -- if you can locate any instances where I
wrote the latter, I will be surprised and issue a retraction. Obviously,
slots are an important element of Python, but they should be used only
when other approaches have proven impractical.

To quote Guido:

__slots__ is a terrible hack with nasty, hard-to-fathom side
effects that should only be used by programmers at grandmaster and
wizard levels. Unfortunately it has gained an enormous undeserved
popularity amongst the novices and apprentices, who should know
better than to use this magic incantation casually.

See also

http://www.dalkescientific.com/writ...06/03/19/class_instantiation_performance.html