What is the semantics meaning of 'object'?

A

Adam

class FooBar(object):
def __init__(self):
...

Inheritance usually takes a class name to indicate which class is the 'parent' class. However, in the previous example, from a django book, the class actually takes an 'object' like parameter, doesn't it? What is the semantics meaning of such kind of notation?

Thanks,
/Adam
 
S

Steven D'Aprano

class FooBar(object):
def __init__(self):
...

Inheritance usually takes a class name to indicate which class is the
'parent' class. However, in the previous example, from a django book,
the class actually takes an 'object' like parameter, doesn't it? What is
the semantics meaning of such kind of notation?

It's not merely notation, "object" is the name of a class. If you type it
(without quotes) at the interactive interpreter, you will see it is a
built-in class:

py> object
<class 'object'>


In Python 3, the use of object as base class is optional, but in Python 2
there is a subtle difference between classes that inherit from object and
those that don't. The reason for this difference is buried in the mists
of time, going back to Python 2.2. If you are interested, google on
"Python unifying types and classes":

https://duckduckgo.com/html/?q=Python+unifying+types+and+classes


As a general rule, unless you actually want "old-style class" behaviour,
you should always inherit from object (or some other built-in type) in
Python 2. In Python 3, it doesn't matter.

The differences include:

* property only works in "new-style" classes that inherit from object;

* likewise for super;

* multiple inheritance with old-style classes can be buggy;

* new-style classes may be slightly faster in general;

* on the down side, automatic delegation of special double-underscore
methods like __getitem__ and __str__ doesn't work with new-style classes.


If none of this means anything to you, be glad, and just inherit from
object or some other built-in type in all your classes, and all will be
good.
 
I

Ian Kelly

* on the down side, automatic delegation of special double-underscore
methods like __getitem__ and __str__ doesn't work with new-style classes.

I actually consider that an up side. Sure it's inconvenient that you
can't delegate all such methods at once just by overriding
__getattribute__, but it would be more troublesome to *accidentally*
implement such methods because you implemented __getattribute__. And
then there are methods that really should not be delegated in the
first place, like __del__.
 
S

Steven D'Aprano

I actually consider that an up side. Sure it's inconvenient that you
can't delegate all such methods at once just by overriding
__getattribute__, but it would be more troublesome to *accidentally*
implement such methods because you implemented __getattribute__.

It's hard to see a case where that would be a bad thing.

1) If the proxied object doesn't include __dunder__ methods, then the
proxy will just end up up calling the default object dunder methods,
exactly as if they weren't proxied.

2) If the proxied object does include dunders, then you generally want
the proxy to call them, with perhaps one or two exceptions, which need to
be overridden regardless of whether they are dunders or not.

And
then there are methods that really should not be delegated in the first
place, like __del__.

If you're using __del__, you're probably doing something wrong :)

I suppose that __del__ is a good counter-example, but (1) hardly any
classes use __del__, and (2) for those that do, it's *way* simpler to
manually override __del__ in the proxy than to manually delegate every
dunder method you care about. There are typically a lot of dunder methods
you care about.

It is not the case that dunder methods cannot be automatically proxied
because somebody deliberately designed Python to work that way. It's an
accidental side-effect of the way new-style classes resolve method calls,
due to decisions made for other reasons having nothing to do with
delegation. That this accident just happens to have one tiny silver
lining doesn't change the fact that, overall, it's still a dark cloud.

In classic classes, automatic delegation was a first-class design
pattern. With new-style classes, it's second-class, and that's a pity.
 
I

Ian Kelly

It's hard to see a case where that would be a bad thing.

1) If the proxied object doesn't include __dunder__ methods, then the
proxy will just end up up calling the default object dunder methods,
exactly as if they weren't proxied.

2) If the proxied object does include dunders, then you generally want
the proxy to call them, with perhaps one or two exceptions, which need to
be overridden regardless of whether they are dunders or not.

Proxying objects is not the only use of __getattribute__.
If you're using __del__, you're probably doing something wrong :)

I suppose that __del__ is a good counter-example, but (1) hardly any
classes use __del__, and (2) for those that do, it's *way* simpler to
manually override __del__ in the proxy than to manually delegate every
dunder method you care about. There are typically a lot of dunder methods
you care about.

If you manually override __del__ in the proxy, then as far as the
garbage collector is concerned, your proxy object has a __del__ method
(even if it does nothing), and so it will be treated differently from
an object without a __del__ method.
It is not the case that dunder methods cannot be automatically proxied
because somebody deliberately designed Python to work that way. It's an
accidental side-effect of the way new-style classes resolve method calls,
due to decisions made for other reasons having nothing to do with
delegation.

Can you elaborate or provide a link? I'm curious to know what other
reason there could be for magic methods to behave differently from
normal methods in this regard.
 
A

Adam Jiang

* property only works in "new-style" classes that inherit from object;
* likewise for super;

Another question raised here is that what is the proper way to refer
to parent class? For example,

class A(object):
def __init__(self, arg):
print "A"

class B(A):
def __init__(self, arg):
super(B, self).__init__(arg)

Is this correct? As the result, whenever you wanted to refer to a
method in parent class, super() functions has to be called. This seems
inefficient.

How to refer to a field defined in parent class?

Thanks,
/Adam
 
I

Ian Kelly

Another question raised here is that what is the proper way to refer
to parent class? For example,

class A(object):
def __init__(self, arg):
print "A"

class B(A):
def __init__(self, arg):
super(B, self).__init__(arg)

Is this correct? As the result, whenever you wanted to refer to a
method in parent class, super() functions has to be called. This seems
inefficient.

Generally, use super() any time you want to refer to a class attribute
(such as a method) in a parent class that is overridden in the child
class. Also note that in Python 3, the call "super(B, self)" can be
condensed to just "super()".

If you're worried about efficiency, you can also explicitly name the
superclass in order to call the method directly, like:

A.__init__(self, arg)

However, super() is generally considered the right way to do this, in
order to avoid repeating the parent class name, for brevity in Python
3, and because it is needed to correctly handle some cases of multiple
inheritance.
How to refer to a field defined in parent class?

For instance attributes or attributes that haven't been overridden,
just write "self.attribute_name".
 
S

Steven D'Aprano

If you're worried about efficiency, you can also explicitly name the
superclass in order to call the method directly, like:

A.__init__(self, arg)

Please don't. This is false economy. The time you save will be trivial,
the overhead of inheritance is not going to be the bottleneck in your
code, and by ignoring super, you only accomplish one thing:

- if you use your class in multiple inheritance, it will be buggy.
 
R

Roy Smith

Steven D'Aprano said:
Please don't. This is false economy. The time you save will be trivial,
the overhead of inheritance is not going to be the bottleneck in your
code, and by ignoring super, you only accomplish one thing:

- if you use your class in multiple inheritance, it will be buggy.

One thing I've never understood about Python 2.x's multiple inheritance
(mostly because I almost never use it) is how you do something like this:

class Base1(object):
def __init__(self, foo):
self.foo = foo

class Base2(object):
def __init__(self, bar):
self.bar = bar

class Derived(Base1, Base2):
def __init__(self, foo, bar):
# now what???

I need to call __init__() in both of my base classes. I don't see how
super() can do that for me. I assume I would just do:

def __init__(self, foo, bar):
Base1.__init__(self, foo)
Base2.__init__(self, bar)

am I missing something here?

For what it's worth, I never bother to inherit from object unless I know
there's something I need from new style classes. Undoubtedly, this
creates a disturbance in The Force, but such is life.
 
I

Ian Kelly

One thing I've never understood about Python 2.x's multiple inheritance
(mostly because I almost never use it) is how you do something like this:

class Base1(object):
def __init__(self, foo):
self.foo = foo

class Base2(object):
def __init__(self, bar):
self.bar = bar

class Derived(Base1, Base2):
def __init__(self, foo, bar):
# now what???

I need to call __init__() in both of my base classes. I don't see how
super() can do that for me. I assume I would just do:

def __init__(self, foo, bar):
Base1.__init__(self, foo)
Base2.__init__(self, bar)

am I missing something here?

Yes, you're missing that super() does not simply call the base class,
but rather the next class in the MRO for whatever the type of the
"self" argument is. If you write the above as:

class Base1(object):
def __init__(self, foo, **kwargs):
super(Base1, self).__init__(**kwargs)

class Base2(object):
def __init__(self, bar, **kwargs):
super(Base2, self).__init__(**kwargs)

class Derived(Base1, Base2):
def __init__(self, **kwargs):
super(Derived, self).__init__(**kwargs)

And then you create an instance of Derived by calling
Derived(foo='foo', bar='bar') and trace the call chain, you find that
Derived.__init__ will call Base1.__init__(foo='foo', bar='bar'), which
extracts its argument and then calls (surprise!)
Base2.__init__(bar='bar'), which again extracts its argument and then
calls object.__init__(), ending the chain.

Of course if you create an instance of Base1, then the Base1 super
call will next call object.__init__ directly, instead of
Base2.__init__. This happens because Base2 occurs after Base1 in the
MRO for the class Derived, but Base2 does not appear at all in the MRO
for the class Base1.
 
I

Ian Kelly

Yes, you're missing that super() does not simply call the base class,
but rather the next class in the MRO for whatever the type of the
"self" argument is. If you write the above as:

Incidentally, although super() is useful, it's not perfect, and this
is one of my grievances with it: that a user can, based upon the name,
draw an inaccurate assumption about what it does without reading or
fully understanding the documentation on it, which might then result
in misusing it. There might still be some code I wrote out there from
when I first started using Python that looks something like:

def __init__(self):
super(Base, self).__init__()
Mixin.__init__(self)

Which is simply wrong, wrong, wrong.
 
S

Steven D'Aprano

Proxying objects is not the only use of __getattribute__.

Okay, fair point.

Actually, I made a mistake... automatic delegation uses __getattr__, not
__getattribute__.

If you manually override __del__ in the proxy, then as far as the
garbage collector is concerned, your proxy object has a __del__ method
(even if it does nothing), and so it will be treated differently from an
object without a __del__ method.

Ack, I get that.

A thought comes to mind... perhaps Python ought to treat a class with
__del__ set to None as if there was no __del__ at all? At the moment the
garbage collector blindly calls __del__ and gets a TypeError.

Can you elaborate or provide a link? I'm curious to know what other
reason there could be for magic methods to behave differently from
normal methods in this regard.

It's an efficiency optimization. I don't quite get the details, but when
you run something like "a + b", Python doesn't search for __add__ using
the normal method lookup procedure. That allows it to skip checking the
instance __dict__, as well as __getattribute__ and __getattr__. End
result is that it's quite a bit faster than ordinary lookup, but the side-
effect is that the semantics are a little different:

- you can't proxy magic methods automatically;

- __getattribute__ and __getattr__ are only called for explicit attribute
access, not for implied calls to magic methods;

- you can't override a magic method on a per instance basis.

Note that explicit calls like "obj.__add__(thing)" take the regular
method lookup, it's just "obj + thing" that takes the shortcut.
 
S

Steven D'Aprano

Incidentally, although super() is useful, it's not perfect, and this is
one of my grievances with it: that a user can, based upon the name, draw
an inaccurate assumption about what it does without reading or fully
understanding the documentation on it, which might then result in
misusing it.

Wait a second... are you saying that the Python developers created an
advanced language feature relating to multiple inheritance, one of the
most complex OOP concepts around, so difficult that most other languages
simply prohibit it completely, and it wasn't instantly and correctly
intuited by every single programmer based only on the name? Oh my stars,
somebody call Ranting Rick, he needs to write a PyWart post to expose
this scandal!!!

*wink*
 
I

Ian Kelly

Wait a second... are you saying that the Python developers created an
advanced language feature relating to multiple inheritance, one of the
most complex OOP concepts around, so difficult that most other languages
simply prohibit it completely, and it wasn't instantly and correctly
intuited by every single programmer based only on the name? Oh my stars,
somebody call Ranting Rick, he needs to write a PyWart post to expose
this scandal!!!

Mostly I'm saying that super() is badly named.
 
R

Roy Smith

Ian Kelly said:
Yes, you're missing that super() does not simply call the base class,
but rather the next class in the MRO for whatever the type of the
"self" argument is. If you write the above as:

class Base1(object):
def __init__(self, foo, **kwargs):
super(Base1, self).__init__(**kwargs)

class Base2(object):
def __init__(self, bar, **kwargs):
super(Base2, self).__init__(**kwargs)

class Derived(Base1, Base2):
def __init__(self, **kwargs):
super(Derived, self).__init__(**kwargs)

And then you create an instance of Derived by calling
Derived(foo='foo', bar='bar') and trace the call chain, you find that
Derived.__init__ will call Base1.__init__(foo='foo', bar='bar'), which
extracts its argument and then calls (surprise!)
Base2.__init__(bar='bar'), which again extracts its argument and then
calls object.__init__(), ending the chain.

Mind. Blown.

I'm tempted to go all Ranting Rick about this being non-obvious, but I
see you already covered that in another post :)

The other thing I see here is that to make this work, the base classes
need to accept **kwargs. That's kind of annoying, since it means a
class has to explicitly designed to be multiply-inheritable.
 
S

Steven D'Aprano

One thing I've never understood about Python 2.x's multiple inheritance
(mostly because I almost never use it) is how you do something like
this:

class Base1(object):
def __init__(self, foo):
self.foo = foo

class Base2(object):
def __init__(self, bar):
self.bar = bar

class Derived(Base1, Base2):
def __init__(self, foo, bar):
# now what???

I need to call __init__() in both of my base classes. I don't see how
super() can do that for me.

It doesn't, and it can't, because your classes are not cooperative. The
problem is, Derived.__init__ needs to call two super methods, but both of
them require different arguments. If this was an ordinary method, you'd
probably see the flaw straight away:

class Artist:
def draw(self, painting): ...

class Gambler:
def draw(self, cards): ...

class GamblingArtist(Gambler, Artist):
def draw(self, painting, cards):
...


Here you've got two completely different actions that merely share the
same name, is it really sensible to have *one* method try to do both? And
if so, you certainly can't expect an automated call to work out which
argument goes to which superclass. So here's a place where cooperative
multiple inheritance doesn't work, and you're reduced to either manually
calling the methods (and hoping that they don't, in turn, end up in some
sort of diamond inheritance diagram, which would be bad), or else you
have to redesign your classes to be more cooperative.

This is one of the reasons why many languages simply prohibit multiple
inheritance outright, except perhaps for mixins, or at least require that
all methods have compatible signatures.

All is not lost, there are ways to make your classes cooperative. The
trick is to have your classes' __init__ methods ignore keyword arguments
they don't know what to do with. object used to do the same thing, but it
no longer does, so you need to add an extra class just before object to
swallow any args before they read object.


class Blocker(object):
def __init__(self, **kwargs):
# Block kwargs from reaching object
super(Blocker, self).__init__()

class Base1(Blocker):
def __init__(self, foo, **kwargs):
self.foo = foo
super(Base1, self).__init__(**kwargs)

class Base2(Blocker):
def __init__(self, bar, **kwargs):
self.bar = bar
super(Base2, self).__init__(**kwargs)

class Derived(Base1, Base2):
def __init__(self, foo, bar):
super(Derived, self).__init__(foo=foo, bar=bar)


This may seem like a lot of work, but nobody said that cooperative
multiple inheritance with different method signatures was easy!

The reality is, multiple inheritance is *hard*. Mixins and traits remove
most of the problems, so if all you've ever used MI for is mixing in new
behaviour, you won't get just how hard it really is. The best way to do
MI is not to do it at all. But if you have to use it, then super is the
only way to do it *right*. Anything else, and you're almost certainly
either duplicating what super would do, or you've got bugs in your code.

Anyway, here are some links about super:

Super considered Super:
http://rhettinger.wordpress.com/2011/05/26/super-considered-super/

Here's a set of three posts, written quite a long time ago but still
relevant, dissecting super in gory detail:

http://www.artima.com/weblogs/viewpost.jsp?thread=236275
http://www.artima.com/weblogs/viewpost.jsp?thread=236278
http://www.artima.com/weblogs/viewpost.jsp?thread=237121

and a more recent update:

http://www.artima.com/weblogs/viewpost.jsp?thread=281127

and a counter-argument, provocatively titled "Super considered Harmful",
although the author backs away from this claim, since it turns out that
the problems are not *super* but multiple inheritance itself.

https://fuhm.net/super-harmful/

Despite telling people not to use super, James Knight doesn't actually
give them a better alternative. I believe this is because he can't --
super is the worst solution to multiple inheritance except for all the
others.
 
I

Ian Kelly

All is not lost, there are ways to make your classes cooperative. The
trick is to have your classes' __init__ methods ignore keyword arguments
they don't know what to do with. object used to do the same thing, but it
no longer does, so you need to add an extra class just before object to
swallow any args before they read object.


class Blocker(object):
def __init__(self, **kwargs):
# Block kwargs from reaching object
super(Blocker, self).__init__()

I don't like the idea of doing this with a cooperative __init__
method. If any keyword arguments were passed that weren't consumed,
that is probably a bug, and this just swallows the exception instead
of reporting it.

Of course, if you're doing cooperative inheritance with some other
method that doesn't exist on object, then this technique is necessary
to prevent the topmost class from trying to call that method on object
and erroring out.
 
I

Ian Kelly

What else would you call a function that does lookups on the current
object's superclasses?

Well, as James Knight points out in the "Super Considered Harmful"
article, the equivalent in Dylan is called "next-method", which isn't
a valid identifier in Python but seems like a reasonable starting
point.
 
R

Roy Smith

Steven D'Aprano said:
What else would you call a function that does lookups on the current
object's superclasses?

Well, mro_lookup() would have been a better choice. Super() has an
obvious meaning, which just happens to be wrong.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,048
Latest member
verona

Latest Threads

Top