Python Mixins

[Matt]

I'm curious about what people's opinions are about using mixins in
Python. I really like, for example, the way that class based views
were implemented in Django 1.3 using mixins. It makes everything
extremely customizable and reusable. I think this is a very good
practice to follow, however, in Python mixins are achieved by using
(or perhaps misusing) inheritance and often multiple inheritance.

Inheritance is a very powerful tool, and multiple inheritance is an
even more powerful tool. These tools have their uses, but I feel like
"mixing in" functionality is not one of them. There are much different
reasons and uses for inheriting functionality from a parent and mixing
in functionality from elsewhere.

As a person, you learn certain things from your parents, you learn
other things from your peers all of those things put together becomes
you. Your peers are not your parents, that would not be possible. You
have completely different DNA and come from a completely different
place.

In terms of code, lets say we have the following classes:

class Animal
class Yamlafiable
class Cat(Animal, Yamlafiable)
class Dog(Animal, Yamlafiable)

I've got an Animal that does animal things, a Cat that does cat things
and a Dog that does dog things. I've also got a Yamlafiable class that
does something clever to generically convert an object into Yaml in
some way. Looking at these classes I can see that a Cat is an Animal,
a Dog is an Animal, a Dog is not a Cat, a Cat is not a Dog, a Dog is a
Yamlafiable? and a Cat is a Yamlafiable? Is that really true? If my
objects are categorized correctly, in the correct inheritance
hierarchy shouldn't that make more sense? Cats and Dogs aren't
Yamlafiable, that doesn't define what they are, rather it defines
something that they can do because of things that they picked up from
their friend the Yamlafile.

This is just a ridiculous example, but I think it is valid to say that
these things shouldn't be limited to inherit functionality only from
their parents, that they can pick other things up along the way as
well. Which is easy to do, right?

Dog.something_new = something_new

(I wish my stupid dog would learn that easily)

Ideally, what I would like to see is something like Ruby's mixins. It
seems to me like Ruby developed this out of necessity due to the fact
that it does not support multiple inheritance, however I think the
implementation is much more pure than inheriting from things that
aren't your parents. (although having only a single parent doesn't
make much sense either, I believe there are very few actual documented
cases of that happening). Here is a Ruby snippet:

module ReusableStuff
def one_thing
"something cool"
end
end
class MyClass < MyParent
include ReusableStuff
end

x = MyClass.new
x.one_thing == "something cool"
MyClass.superclass == Object

So I'm inheriting from MyParent and mixing in additional functionality
from ReusableStuff without affecting who my Parents are. This, to me,
makes much more sense than using inheritance to just grab a piece of
functionality that I want to reuse. I wrote a class decorator for
Python that does something similar (https://gist.github.com/1233738)
here is a snippet from that:

class MyMixin(object):
def one_thing(self):
return "something cool"

@mixin(MyMixin)
class MyClass(object):
pass

x = MyClass()
x.one_thing() == 'something cool'
x.__class__.__bases__ == (object,)

To me, this is much more concise. By looking at this I can tell what
MyClass IS, who it's parents are and what else it can do. I'm very
interested to know if there are others who feel as dirty as I do when
using inheritance for mixins or if there are other things that Python
developers are doing to mix in functionality without using inheritance
or if the general populous of the Python community disagrees with me
and thinks that this is a perfectly valid use of inheritance.

I look forward to hearing back.

Thanks,
Matthew J Morrison
www.mattjmorrison.com


P.S. - This is a good article about not using inheritance as a code
reuse tool: http://littletutorials.com/2008/06/23/inheritance-not-for-code-reuse/

 

[rantingrick]

(although having only a single parent doesn't
make much sense either, I believe there are very few actual documented
cases of that happening).

There is nothing wrong with an object having only one parent. Most
times the reasons are for maintainability. I might have a TextEditor
that exposes all the generic functionality that are ubiquitous to text
editors and then a FancyTextEditor(TextEditor) that exposes
functionality that is unique to a confined set of text editing uses. A
silly example, but proves the point. Do not judge an object by the
number of prodigy.

 

[Thomas Jollans]

I'm curious about what people's opinions are about using mixins in
Python. I really like, for example, the way that class based views
were implemented in Django 1.3 using mixins. It makes everything
extremely customizable and reusable. I think this is a very good
practice to follow, however, in Python mixins are achieved by using
(or perhaps misusing) inheritance and often multiple inheritance.

Inheritance is a very powerful tool, and multiple inheritance is an
even more powerful tool. These tools have their uses, but I feel like
"mixing in" functionality is not one of them. There are much different
reasons and uses for inheriting functionality from a parent and mixing
in functionality from elsewhere.

As a person, you learn certain things from your parents, you learn
other things from your peers all of those things put together becomes
you. Your peers are not your parents, that would not be possible. You
have completely different DNA and come from a completely different
place.

In terms of code, lets say we have the following classes:

class Animal
class Yamlafiable
class Cat(Animal, Yamlafiable)
class Dog(Animal, Yamlafiable)

I think this is an excellent use of multiple inheritance. One could also
have a construction like this:

class Dog (Animal)
class YamlafialbleDog (Dog, Yamlafiable)

.... which you may be more comfortable with. In you above example, yes, a
Dog object is a Yamlafiable object. If you need a Yamlafiable object,
you can use a Cat or Dog. That's what inheritance is about.

In Python or Ruby, this way of doing things is not all that different
from the one you present below. Here, it doesn't really matter. In
strictly typed languages, it makes a world of difference. What if you
don't care what kind of object you're dealing with, as long as it
supports the interface a certain mixin provides? In Python, true, duck
typing will do the trick. In C++, for example, where you could use the C
preprocessor to do something like Ruby mixins, multiple inheritance is a
lot more useful for mixing in something that has a public interface.

The Vala language, and, I suppose the GObject type system, actually
allows interfaces to act as mixins. This is really a more formalised way
of doing just this: using multiple inheritance (which, beyond
interfaces, Vala does not support) to mix in functionality.

Oh and your thing looks kind of neat.

Thomas

 

[Steven D'Aprano]

I'm curious about what people's opinions are about using mixins in
Python. I really like, for example, the way that class based views
were implemented in Django 1.3 using mixins.

Mixins considered harmful:
http://www.artima.com/weblogs/viewpost.jsp?thread=246341
http://www.artima.com/weblogs/viewpost.jsp?thread=246483

Mixins are much too easy to abuse, but there are uses for them.

I have found mixins useful for testing. If I have a bunch of functions that
take mostly similar unit tests, I create a mixin for the common unit tests,
then apply them to the tests.

Suppose I have two functions, ham and spam, which are unrelated but do share
some common behaviours. Since they are unrelated, I don't want the spam
test suite to inherit from the ham test suite, or vice versa, but I don't
want to write the same tests multiple times. (Especially when I add cheese,
egg and tomato functions.) So I put the common behaviours in a mixin class:

class StandardTestMixin:
def test_requires_one_argument(self):
self.assertRaises(TypeError, self.func)
def test_has_docstring(self):
self.assertTrue(self.func.__doc__)

class TestHam(unittest.TestCase, StandardTestMixin):
...

class TestSpam(unittest.TestCase, StandardTestMixin):
...


In this case, I simply don't care that the use of a mixin implies that
TestHam is a StandardTestMixin. I treat that as an accident of
implementation: it makes no practical difference, since I write no code
that depends on isinstance(instance, StandardTestMixin).



[...]

In terms of code, lets say we have the following classes:

class Animal
class Yamlafiable
class Cat(Animal, Yamlafiable)
class Dog(Animal, Yamlafiable)

I've got an Animal that does animal things, a Cat that does cat things
and a Dog that does dog things. I've also got a Yamlafiable class that
does something clever to generically convert an object into Yaml in
some way. Looking at these classes I can see that a Cat is an Animal,
a Dog is an Animal, a Dog is not a Cat, a Cat is not a Dog, a Dog is a
Yamlafiable? and a Cat is a Yamlafiable? Is that really true?

That depends on what you want.

Python says is that issubclass(Cat, Yamlafiable) will return True. The
*interpretation* of that fact is entirely up to you. If you want to
interpret it as meaning that cats are Yamlafiables, go right ahead. If you
want to interpret it as a technical result with no semantic meaning, that's
fine too. After all, just because "ram" in "programming" returns True
doesn't actually mean anything about male sheep and computing.

The map is not the territory, and object-oriented ancestor/descendant
relationships are not the same as real life relationships. It's a tool,
nothing more, and if the metaphor starts to creak at the edges, oh well, so
much for the metaphor.

In fact, even in real life, the ancestor/descendant metaphor creaks. What
should we make of bacteria which exchange DNA with other species of
bacteria? Which is the ancestor and which is the descendant? How about
symbionts like lichen? What about when we splice genes from one species
into another? So even in real life, there are multiple inheritance and
mixins.

If my
objects are categorized correctly, in the correct inheritance
hierarchy shouldn't that make more sense? Cats and Dogs aren't
Yamlafiable, that doesn't define what they are, rather it defines
something that they can do because of things that they picked up from
their friend the Yamlafile.

The standard metaphor for inheritance in OOP doesn't include "learning
things from a friend". It is a very simple metaphor: everything is
either "is-a" or "has-a". If you inherit from a class, the is-a
relationship holds. If you don't want that, you can use composition
instead:

class Cat(Animal):
def __init__(self, yamlifier):
self.yamlifier = yamlifier
def make_yaml(self):
self.yamlifier.make_yaml(self, self.spam)

Now we say that a Cat has-a Yamlafiable.

This is just a ridiculous example, but I think it is valid to say that
these things shouldn't be limited to inherit functionality only from
their parents, that they can pick other things up along the way as
well. Which is easy to do, right?

Dog.something_new = something_new

And that works in Python. But the thing is, which would you rather write?

#1
class Dog(Animal, Yamlafiable):
pass

or this?

#2
class Dog(Animal):
pass

Dog.__dict__.update(Yamlafiable.__dict__)


But wait! The two aren't the same. There are quite big differences in
behaviour between inheritance and attribute injection:

* In the case of inheritance, attributes defined by Animal take
precedence over those defined by Yamlafiable. In the case of
injection, the opposite is the case.

* In the case of inheritance, attributes defined by Yamlafiable's
superclasses are reachable. In the case of injection, they
aren't (at least not without more work).

* Inheritance is late-binding, injection is early-binding: if
you dynamically add a new method to Yamlafiable, Dog will
see the change in the first case but not the second.


That's not to say that one is right and the other is wrong. They're just
different. Use whichever tool you prefer.

Another option is straits:

http://www.artima.com/weblogs/viewpost.jsp?thread=246488

 

[Carl Banks]

On Thursday, September 22, 2011 2:14:39 PM UTC-7, Matt wrote:
[snip]

class MyMixin(object):
def one_thing(self):
return "something cool"

@mixin(MyMixin)
class MyClass(object):
pass

x = MyClass()
x.one_thing() == 'something cool'
x.__class__.__bases__ == (object,)

To me, this is much more concise. By looking at this I can tell what
MyClass IS, who it's parents are and what else it can do. I'm very
interested to know if there are others who feel as dirty as I do when
using inheritance for mixins

Not me. Inheritance perfectly encompasses the mixin relationship, and because inheritance is so thoroughly ingrained in Python, it makes sense not tocreate a completely different mechanism to share behavior just for the case of mixins.

I know that, as someone who reads code, I would rather coders stick to well-known mechanisms than to create their own ad hoc mechanisms that don't actually add any new capability. Take your MyClass example above. If you hadstuck to inheritance I could have seen what classes all the behavior was implemented by listing the __bases__. But since you used an ad hoc mechanism, now I have to track down where the hell that one_thing() method is coming from.

No mechanism is ever perfect, and Python's MI is very far from perfect, butsticking to well-known and understood methods is usually more important than whatever little improvement you can make. (And it is little; best as I can tell, your main objection is that mixins make it harder to see what the"main" parent is. I'd say that's a dubious justification to spring a new behavior-sharing mechanism on a reader.)

or if there are other things that Python
developers are doing to mix in functionality without using inheritance
or if the general populous of the Python community disagrees with me
and thinks that this is a perfectly valid use of inheritance.

I'd guess the majority just use inheritance, although I can't say I've seenenough code out there to gauge it.

But there is something else a lot of Pythonistas will do in many cases: just define regular functions. In your example, instead of defining one_thing() as a method of a mixin, define it as a function. Personally, I find that I almost never use mixins, though I have absolutely nothing against them and I use MI and metaclasses all the time. It's just that for most things I'd use a mixin for, I find that one or two regular functions work perfectly well.


Carl Banks

 

[Carl Banks]

In terms of code, lets say we have the following classes:

class Animal
class Yamlafiable
class Cat(Animal, Yamlafiable)
class Dog(Animal, Yamlafiable)

I've got an Animal that does animal things, a Cat that does cat things
and a Dog that does dog things. I've also got a Yamlafiable class that
does something clever to generically convert an object into Yaml in
some way. Looking at these classes I can see that a Cat is an Animal,
a Dog is an Animal, a Dog is not a Cat, a Cat is not a Dog, a Dog is a
Yamlafiable? and a Cat is a Yamlafiable? Is that really true?

Yes.

I hope you are not confusing Cats with cats.

If my
objects are categorized correctly, in the correct inheritance
hierarchy shouldn't that make more sense? Cats and Dogs aren't
Yamlafiable, that doesn't define what they are, rather it defines
something that they can do because of things that they picked up from
their friend the Yamlafile.

The whole point of OOP is that objects are defined by their behavior. A Cat is whatever it can do. A Dog is whatever it can do. If a Cat is yamlafiable, then it's coorect to say that a Cat is a Yamlafible (even if a cat isn't).


Carl Banks

 

[rantingrick]

I'm curious about what people's opinions are about using mixins in
Python. I really like, for example, the way that class based views
were implemented in Django 1.3 using mixins. It makes everything
extremely customizable and reusable. I think this is a very good
practice to follow, however, in Python mixins are achieved by using
(or perhaps misusing) inheritance and often multiple inheritance.

Have a look at this article also. Even though he uses java source code
anyone can follow along:

http://berniesumption.com/software/inheritance-is-evil-and-must-be-destroyed/

 

[rantingrick]

class StandardTestMixin:
    def test_requires_one_argument(self):
        self.assertRaises(TypeError, self.func)
    def test_has_docstring(self):
        self.assertTrue(self.func.__doc__)

And this is another example of why we need doc-string reforms. Here we
have a well know "pythonista" (for lack of better word) who needs to
write a test for doc-string inclusion because he refuses to get in the
habit of writing them. People, EVERY method and function MUST have a
doc-string! What is the purpose of having doc-strings if we are too
lazy to use them!

 

[Tim Chase]

And this is another example of why we need doc-string reforms. Here we
have a well know "pythonista" (for lack of better word) who needs to
write a test for doc-string inclusion because he refuses to get in the
habit of writing them. People, EVERY method and function MUST have a
doc-string! What is the purpose of having doc-strings if we are too
lazy to use them!

http://www.python.org/dev/peps/pep-0257/

"""
The aim of this PEP is to standardize the high-level structure of
docstrings: what they should contain, and how to say it (without
touching on any markup syntax within docstrings). The PEP
contains conventions, not laws or syntax.
"""

You keep saying the word "MUST". I do not think it means what
you think it means. Just for the record [1], [2], [3], [4] all
in my first page of search Google search results. Pot, meet
kettle.

-tkc

[1]
http://mail.python.org/pipermail/python-list/2009-September/1220068.html

[2]
http://mail.python.org/pipermail//python-list/2011-July/1276021.html

[3]
http://mail.python.org/pipermail/python-list/2010-March/1237898.html

[4]
http://mail.python.org/pipermail/python-list/2010-June/1246305.html

 

[Terry Reedy]

Python says is that issubclass(Cat, Yamlafiable) will return True. The
*interpretation* of that fact is entirely up to you. If you want to
interpret it as meaning that cats are Yamlafiables, go right ahead. If you
want to interpret it as a technical result with no semantic meaning, that's
fine too. After all, just because "ram" in "programming" returns True
doesn't actually mean anything about male sheep and computing.

Today's comics had a strip in which girl A suggests to girl B that they
go to a disco to disco-ver someone interesting.

In fact, even in real life, the ancestor/descendant metaphor creaks. What
should we make of bacteria which exchange DNA with other species of
bacteria?

That the human concept of 'species' does not really apply to bacteria.
And similarly, that the CompSci concept of 'class' may also not always
fit what we want to model.

Which is the ancestor and which is the descendant? How about
symbionts like lichen? What about when we splice genes from one species
into another? So even in real life, there are multiple inheritance and
mixins.

Good point.