issubclass(dict, Mapping)

[kj]

In a message (<[email protected]>)

I suspect you're trying to make this more complicated than it actually
is. You keep finding little corner cases that expose implementation
details (such as the heap-types issue above) and leaping to the erroneous
conclusion that because you didn't understand this tiny little corner of
Python's class model, you didn't understand any of it. Python's object
model is relatively simple, but it does occasionally expose a few messy
corners.

I disagree with your assessment. What you call "little corner
cases" I call "fundamental", as in "you can't really call yourself
competent with Python if you're ignorant about them".

To use a term I first saw in an article by Joel Spolsky
(http://is.gd/je42O), Python's object model is a rather "leaky
abstraction". This refers to the situation in which a user is not
shielded from the "implementation details". When an abstraction
leaks, implementation details are no longer negligible, they cease
to be "little corner cases".

Here's another example, fresh from today's crop of wonders:

(v. 2.7.0)

from collections import Mapping
issubclass(dict, Mapping) True
dict.__bases__

[issubclass(b, Mapping) for b in dict.__bases__]

[False]


So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.

I suspect this is another abstraction leak ("dict is *supposed* to
be a Python class like all others, but in fact it's not *really*.
You see, once upon a time...").

I conclude that, for me to understand Python's (rather leaky) object
model abstraction, I have to understand its underlying implementation.
Unfortunately, as far as I know, there's no other choice but to
study the source code, since there's no other more readable
description of this implementation.

Maybe there are fewer "abstraction leaks" in 3.0...

~kj

 

[Adam Tauno Williams]

In a message (<[email protected]>)

I suspect you're trying to make this more complicated than it actually
is. You keep finding little corner cases that expose implementation
details (such as the heap-types issue above) and leaping to the erroneous
conclusion that because you didn't understand this tiny little corner of
Python's class model, you didn't understand any of it. Python's object
model is relatively simple, but it does occasionally expose a few messy
corners.

I disagree with your assessment. What you call "little corner
cases" I call "fundamental", as in "you can't really call yourself
competent with Python if you're ignorant about them".
To use a term I first saw in an article by Joel Spolsky
(http://is.gd/je42O), Python's object model is a rather "leaky
abstraction". This refers to the situation in which a user is not
shielded from the "implementation details". When an abstraction
leaks, implementation details are no longer negligible, they cease
to be "little corner cases".
Here's another example, fresh from today's crop of wonders:
(v. 2.7.0)

from collections import Mapping
issubclass(dict, Mapping) True
dict.__bases__

[issubclass(b, Mapping) for b in dict.__bases__]

[False]
So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.
I suspect this is another abstraction leak ("dict is *supposed* to
be a Python class like all others, but in fact it's not *really*.
You see, once upon a time...").
I conclude that, for me to understand Python's (rather leaky) object
model abstraction, I have to understand its underlying implementation.
Unfortunately, as far as I know, there's no other choice but to
study the source code, since there's no other more readable
description of this implementation.
Maybe there are fewer "abstraction leaks" in 3.0...

Boy howdy are you going to incite the ire of the Pythonistas!

IMO, the "object model" isn't "leaky", it is simply "adhoc" and not
really a "model" at all [write as many 800 page books as you want: if it
walks like a zombie duck, smells like a zombie duck - it is still a
zombie duck]. Performing introspection in Python is awful and a
veritable land-mine of "implementation details". The short and honest
answer is: avoid doing it whenever possible, try to figure out how to
accomplish the task some other way.

 

[Antoine Pitrou]

So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.

I suspect this is another abstraction leak ("dict is *supposed* to
be a Python class like all others, but in fact it's not *really*.

It is. You just haven't read about Python's ABCs (abstract base
classes):

http://docs.python.org/library/abc.html#abc.ABCMeta

« You can also register unrelated concrete classes (even built-in
classes) and unrelated ABCs as “virtual subclasses†– these and their
descendants will be considered subclasses of the registering ABC by the
built-in issubclass() function, but the registering ABC won’t show up
in their MRO (Method Resolution Order) nor will method implementations
defined by the registering ABC be callable (not even via super()). »

With a very simple example in the register() doc:

http://docs.python.org/library/abc.html#abc.ABCMeta.register

Regards

Antoine.

 

[Antoine Pitrou]

IMO, the "object model" isn't "leaky", it is simply "adhoc" and not
really a "model" at all [write as many 800 page books as you want: if it
walks like a zombie duck, smells like a zombie duck - it is still a
zombie duck]. Performing introspection in Python is awful and a
veritable land-mine of "implementation details".

Introspection is fine as long as you stick to officially promoted tools
such as isinstance(), issubclass(), dir() or the inspect module.
If you start looking inside the pants of the object model, you can have
surprises

Regards

Antoine.

 

[Steve Holden]

On 12/22/2010 9:20 AM, kj wrote:
[...]

I suspect this is another abstraction leak ("dict is *supposed* to
be a Python class like all others, but in fact it's not *really*.
You see, once upon a time...").

So your suspicions are to be placed above the knowledge of those who
really do understand Python's object model? That seems like a recipe for
cargo cult programming ...

I conclude that, for me to understand Python's (rather leaky) object
model abstraction, I have to understand its underlying implementation.
Unfortunately, as far as I know, there's no other choice but to
study the source code, since there's no other more readable
description of this implementation.

You don't have to understand "the implementation" (there are at least
five different implementations, which one will you choose as your standard?)

Maybe there are fewer "abstraction leaks" in 3.0...

Python deliberately exposes introspection interfaces, which you may use
if you wish. As with all introspectable languages (including Java) if
you push the envelope you are likely to hit corner cases. As Steven
d'Aprano has already said, these *are* corner cases and not the whole of
the language.

Don't worry about having a complete knowledge of the language before you
start to use it. That can induce paralysis ...

regards
Steve

 

[Ethan Furman]

In a message (<[email protected]>)


I disagree with your assessment. What you call "little corner
cases" I call "fundamental", as in "you can't really call yourself
competent with Python if you're ignorant about them".

So where on the sliding scale do you place 'competent'? It sounds to me
like you are looking at 'master'.

Here's another example, fresh from today's crop of wonders:

(v. 2.7.0)

from collections import Mapping
issubclass(dict, Mapping) True
dict.__bases__

[issubclass(b, Mapping) for b in dict.__bases__]

[False]

Firstly, as I'm sure you know, if you don't import Mapping from
collections the issubclass test fails with a NameError.

Secondly, why do you care? Did you get bitten by something? Some
error, or worse, silently got wrong results? (Sincere question.)

So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.

I suspect this is another abstraction leak

My take on abstraction leaks is when the underlying actuality shows
through in a non-ignorable way -- so I ask again, how is this
discrepancy making it so you can't ignore it?

~Ethan~

 

[Terry Reedy]

Documented as an *ABSTRACT* base class. ABCs were added in 3.0 and
backparted to 2.7. One can be quite competant in Python completely
ignoring ABCs.

True

Yes, dict is a concrete Mapping class. I suppose we could have instead
added a new builtin function 'isconcretetizationof' but is seemed easier
to reuse issubclass for the test. Most people have no problem with that.

(<type 'object'>,)

The one and only *CONCRETE* base class. In 3.x, all classes are
subclasses of object, which simplifies the class model a bit.

[issubclass(b, Mapping) for b in dict.__bases__]

[False]

So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.

Right. dict is direct concrete Mapping implementation and not subclassed
from one.

The main reason for introducing ABCs was to make it easier to test
whether an object passed to a function is an instance of a possibly
unknown or yet-to-be-written class in an abstract category, which has a
certain api or interface. The actual usage of an ABC would be more like
this:

from collections import Mapping
def f(m):
if not isinstance(m, Mapping):
raise ValueError('input is not a mapping')
else: return True

f(dict())
# True
f([])
# produces
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
f([])
File "<pyshell#6>", line 3, in f
raise ValueError('input is not a mapping')
ValueError: input is not a mapping

 

[Steven D'Aprano]

Here's another example, fresh from today's crop of wonders:

(v. 2.7.0)

from collections import Mapping
issubclass(dict, Mapping) True
dict.__bases__

[issubclass(b, Mapping) for b in dict.__bases__]

[False]


So dict is a subclass of Mapping, even though none of the bases of dict
is either Mapping or a subclass of Mapping. Great.

Yes. So what?

(1) What *actual* problem does this cause you?

(2) Do you have an example of code that breaks because of this?

(3) Do you understand that since the introduction of ABC (abstract base
classes) in Python 2.6 (I think), isinstance and issubclass checks are
performed cooperatively? The instance or class are asked if they wish to
be known as an instance/subclass of the second argument. Classes can
register themselves as subclasses of (say) Mapping without sharing any
actual code with Mapping.

This is a good thing, and the problem isn't that the abstraction leaks,
as you believe, but the opposite: you're *ignoring* the abstraction and
looking for concrete details that may or may not exist.

I fear that you have fundamentally misunderstood the concept of "leaky
abstraction". It does not mean, as you seem to think, that some concrete
implementation detail differs between two classes (or functions). It
means that some difference in behaviour is exposed, that difference being
irrelevant to the abstraction but nevertheless important in some other
sense. A contrived example:

class MyList(list):
def __len__(self):
import time
time.sleep(3600000)
return list.__len__(self)

MyList can be used anywhere a regular list can be used. Functionally the
two are identical. The abstraction is that MyList is the same as list.
But the leak is that len(MyList()) is *incredibly* slow.


Coming back to Mapping:

Abstraction: issubclass(dict, Mapping)

One possible concrete implementation detail of how issubclass is
implemented:
any(base is Mapping for base in dict.__bases__)


The statement "dict is a subclass of Mapping" is about an abstract
relationship. It's not necessarily a statement about __bases__.

To give an analogy, if you insist on doing DNA testing to determine
whether a boy is a son of a man, you're going to be confused and
distressed every time you find fathers whose sons are genetically
unrelated to them.

 

[kj]

Here's another example, fresh from today's crop of wonders:

(v. 2.7.0)

from collections import Mapping
issubclass(dict, Mapping) True
dict.__bases__

[issubclass(b, Mapping) for b in dict.__bases__]

[False]

So dict is a subclass of Mapping, even though none of the bases of dict
is either Mapping or a subclass of Mapping. Great.

Yes. So what?

That's being deliberately obtuse. The situation described goes
smack against standard OOP semantics, which would be fine if all
this stuff was documented clearly and reasonably, i.e. in one
(preferably "official") place rather than scattered over a bazillion
separate documents, PEP this, module that, GvR musing #42, etc.

Let's just say that I'm looking forward to the end to these surprises.

~kj

 

[Steven D'Aprano]

In <[email protected]> Steven D'Aprano

Here's another example, fresh from today's crop of wonders:

(v. 2.7.0)
from collections import Mapping
issubclass(dict, Mapping)
True
dict.__bases__
(<type 'object'>,)
[issubclass(b, Mapping) for b in dict.__bases__]
[False]

So dict is a subclass of Mapping, even though none of the bases of
dict is either Mapping or a subclass of Mapping. Great.

Yes. So what?

That's being deliberately obtuse. The situation described goes smack
against standard OOP semantics,

What are these "standard OOP semantics" you're referring to, and who made
them "standard"? If people can't even decide whether multiple inheritance
should be allowed or not, what makes you think that there is any such
thing as "standard OOP"?

I think you are confusing concrete implementation details with the
interface. The interface for subclass testing in Python is as follows:

A class K is a subclass of class C if, and only if,
issubclass(C, K) returns a true result.


That's it. Everything else is implementation. __bases__ is
implementation. __subclasscheck__ is implementation. If Python adds a
third mechanism for implementing subclass checks in version 3.3 or 3.4,
the interface will just continue to work correctly.

which would be fine if all this stuff
was documented clearly and reasonably, i.e. in one (preferably
"official") place rather than scattered over a bazillion separate
documents, PEP this, module that, GvR musing #42, etc.

This is documented, in the docs.

http://docs.python.org/reference/datamodel.html#customizing-instance-and-subclass-checks

Nobody says the Python docs are perfect, but most things you ask are in
there. Patches for the docs to improve them are welcome.

 

[Aahz]

standard OOP semantics

"...some experts might say a C++ program is not object-oriented without
inheritance and virtual functions. As one of the early Smalltalk
implementors myself, I can say they are full of themselves." --zconcept