issubclass(dict, Mapping)

Discussion in 'Python' started by kj, Dec 22, 2010.

  1. kj

    kj Guest

    In a message (<4cf97c94$0$30003$c3e8da3$>)
    on a different thread, Steven D'Aprano tells me:

    >I suspect you're trying to make this more complicated than it actually
    >is. You keep finding little corner cases that expose implementation
    >details (such as the heap-types issue above) and leaping to the erroneous
    >conclusion that because you didn't understand this tiny little corner of
    >Python's class model, you didn't understand any of it. Python's object
    >model is relatively simple, but it does occasionally expose a few messy
    >corners.


    I disagree with your assessment. What you call "little corner
    cases" I call "fundamental", as in "you can't really call yourself
    competent with Python if you're ignorant about them".

    To use a term I first saw in an article by Joel Spolsky
    (http://is.gd/je42O), Python's object model is a rather "leaky
    abstraction". This refers to the situation in which a user is not
    shielded from the "implementation details". When an abstraction
    leaks, implementation details are no longer negligible, they cease
    to be "little corner cases".

    Here's another example, fresh from today's crop of wonders:

    (v. 2.7.0)
    >>> from collections import Mapping
    >>> issubclass(dict, Mapping)

    True
    >>> dict.__bases__

    (<type 'object'>,)
    >>> [issubclass(b, Mapping) for b in dict.__bases__]

    [False]


    So dict is a subclass of Mapping, even though none of the bases of
    dict is either Mapping or a subclass of Mapping. Great.

    I suspect this is another abstraction leak ("dict is *supposed* to
    be a Python class like all others, but in fact it's not *really*.
    You see, once upon a time...").

    I conclude that, for me to understand Python's (rather leaky) object
    model abstraction, I have to understand its underlying implementation.
    Unfortunately, as far as I know, there's no other choice but to
    study the source code, since there's no other more readable
    description of this implementation.

    Maybe there are fewer "abstraction leaks" in 3.0...

    ~kj
    kj, Dec 22, 2010
    #1
    1. Advertising

  2. On Wed, 2010-12-22 at 14:20 +0000, kj wrote:
    > In a message (<4cf97c94$0$30003$c3e8da3$>)
    > on a different thread, Steven D'Aprano tells me:
    > >I suspect you're trying to make this more complicated than it actually
    > >is. You keep finding little corner cases that expose implementation
    > >details (such as the heap-types issue above) and leaping to the erroneous
    > >conclusion that because you didn't understand this tiny little corner of
    > >Python's class model, you didn't understand any of it. Python's object
    > >model is relatively simple, but it does occasionally expose a few messy
    > >corners.

    > I disagree with your assessment. What you call "little corner
    > cases" I call "fundamental", as in "you can't really call yourself
    > competent with Python if you're ignorant about them".
    > To use a term I first saw in an article by Joel Spolsky
    > (http://is.gd/je42O), Python's object model is a rather "leaky
    > abstraction". This refers to the situation in which a user is not
    > shielded from the "implementation details". When an abstraction
    > leaks, implementation details are no longer negligible, they cease
    > to be "little corner cases".
    > Here's another example, fresh from today's crop of wonders:
    > (v. 2.7.0)
    > >>> from collections import Mapping
    > >>> issubclass(dict, Mapping)

    > True
    > >>> dict.__bases__

    > (<type 'object'>,)
    > >>> [issubclass(b, Mapping) for b in dict.__bases__]

    > [False]
    > So dict is a subclass of Mapping, even though none of the bases of
    > dict is either Mapping or a subclass of Mapping. Great.
    > I suspect this is another abstraction leak ("dict is *supposed* to
    > be a Python class like all others, but in fact it's not *really*.
    > You see, once upon a time...").
    > I conclude that, for me to understand Python's (rather leaky) object
    > model abstraction, I have to understand its underlying implementation.
    > Unfortunately, as far as I know, there's no other choice but to
    > study the source code, since there's no other more readable
    > description of this implementation.
    > Maybe there are fewer "abstraction leaks" in 3.0...


    Boy howdy are you going to incite the ire of the Pythonistas!

    IMO, the "object model" isn't "leaky", it is simply "adhoc" and not
    really a "model" at all [write as many 800 page books as you want: if it
    walks like a zombie duck, smells like a zombie duck - it is still a
    zombie duck]. Performing introspection in Python is awful and a
    veritable land-mine of "implementation details". The short and honest
    answer is: avoid doing it whenever possible, try to figure out how to
    accomplish the task some other way.
    Adam Tauno Williams, Dec 22, 2010
    #2
    1. Advertising

  3. On Wed, 22 Dec 2010 14:20:51 +0000 (UTC)
    kj <> wrote:
    >
    > So dict is a subclass of Mapping, even though none of the bases of
    > dict is either Mapping or a subclass of Mapping. Great.
    >
    > I suspect this is another abstraction leak ("dict is *supposed* to
    > be a Python class like all others, but in fact it's not *really*.


    It is. You just haven't read about Python's ABCs (abstract base
    classes):

    http://docs.python.org/library/abc.html#abc.ABCMeta

    « You can also register unrelated concrete classes (even built-in
    classes) and unrelated ABCs as “virtual subclasses†– these and their
    descendants will be considered subclasses of the registering ABC by the
    built-in issubclass() function, but the registering ABC won’t show up
    in their MRO (Method Resolution Order) nor will method implementations
    defined by the registering ABC be callable (not even via super()). »

    With a very simple example in the register() doc:

    http://docs.python.org/library/abc.html#abc.ABCMeta.register

    Regards

    Antoine.
    Antoine Pitrou, Dec 22, 2010
    #3
  4. On Wed, 22 Dec 2010 09:35:48 -0500
    Adam Tauno Williams <> wrote:
    >
    > IMO, the "object model" isn't "leaky", it is simply "adhoc" and not
    > really a "model" at all [write as many 800 page books as you want: if it
    > walks like a zombie duck, smells like a zombie duck - it is still a
    > zombie duck]. Performing introspection in Python is awful and a
    > veritable land-mine of "implementation details".


    Introspection is fine as long as you stick to officially promoted tools
    such as isinstance(), issubclass(), dir() or the inspect module.
    If you start looking inside the pants of the object model, you can have
    surprises :)

    Regards

    Antoine.
    Antoine Pitrou, Dec 22, 2010
    #4
  5. kj

    Steve Holden Guest

    On 12/22/2010 9:20 AM, kj wrote:
    [...]
    > I suspect this is another abstraction leak ("dict is *supposed* to
    > be a Python class like all others, but in fact it's not *really*.
    > You see, once upon a time...").
    >

    So your suspicions are to be placed above the knowledge of those who
    really do understand Python's object model? That seems like a recipe for
    cargo cult programming ...

    > I conclude that, for me to understand Python's (rather leaky) object
    > model abstraction, I have to understand its underlying implementation.
    > Unfortunately, as far as I know, there's no other choice but to
    > study the source code, since there's no other more readable
    > description of this implementation.
    >

    You don't have to understand "the implementation" (there are at least
    five different implementations, which one will you choose as your standard?)

    > Maybe there are fewer "abstraction leaks" in 3.0...
    >

    Python deliberately exposes introspection interfaces, which you may use
    if you wish. As with all introspectable languages (including Java) if
    you push the envelope you are likely to hit corner cases. As Steven
    d'Aprano has already said, these *are* corner cases and not the whole of
    the language.

    Don't worry about having a complete knowledge of the language before you
    start to use it. That can induce paralysis ...

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
    See Python Video! http://python.mirocommunity.org/
    Holden Web LLC http://www.holdenweb.com/
    Steve Holden, Dec 22, 2010
    #5
  6. kj

    Ethan Furman Guest

    kj wrote:
    > In a message (<4cf97c94$0$30003$c3e8da3$>)
    > on a different thread, Steven D'Aprano tells me:
    >
    >> I suspect you're trying to make this more complicated than it actually
    >> is. You keep finding little corner cases that expose implementation
    >> details (such as the heap-types issue above) and leaping to the erroneous
    >> conclusion that because you didn't understand this tiny little corner of
    >> Python's class model, you didn't understand any of it. Python's object
    >> model is relatively simple, but it does occasionally expose a few messy
    >> corners.

    >
    > I disagree with your assessment. What you call "little corner
    > cases" I call "fundamental", as in "you can't really call yourself
    > competent with Python if you're ignorant about them".


    So where on the sliding scale do you place 'competent'? It sounds to me
    like you are looking at 'master'.


    > Here's another example, fresh from today's crop of wonders:
    >
    > (v. 2.7.0)
    >>>> from collections import Mapping
    >>>> issubclass(dict, Mapping)

    > True
    >>>> dict.__bases__

    > (<type 'object'>,)
    >>>> [issubclass(b, Mapping) for b in dict.__bases__]

    > [False]


    Firstly, as I'm sure you know, if you don't import Mapping from
    collections the issubclass test fails with a NameError.

    Secondly, why do you care? Did you get bitten by something? Some
    error, or worse, silently got wrong results? (Sincere question.)


    > So dict is a subclass of Mapping, even though none of the bases of
    > dict is either Mapping or a subclass of Mapping. Great.
    >
    > I suspect this is another abstraction leak


    My take on abstraction leaks is when the underlying actuality shows
    through in a non-ignorable way -- so I ask again, how is this
    discrepancy making it so you can't ignore it?

    ~Ethan~
    Ethan Furman, Dec 22, 2010
    #6
  7. kj

    Terry Reedy Guest

    On 12/22/2010 9:20 AM, kj wrote:

    >>>> from collections import Mapping


    Documented as an *ABSTRACT* base class. ABCs were added in 3.0 and
    backparted to 2.7. One can be quite competant in Python completely
    ignoring ABCs.

    >>>> issubclass(dict, Mapping)

    > True


    Yes, dict is a concrete Mapping class. I suppose we could have instead
    added a new builtin function 'isconcretetizationof' but is seemed easier
    to reuse issubclass for the test. Most people have no problem with that.

    >>>> dict.__bases__

    > (<type 'object'>,)


    The one and only *CONCRETE* base class. In 3.x, all classes are
    subclasses of object, which simplifies the class model a bit.

    >>>> [issubclass(b, Mapping) for b in dict.__bases__]

    > [False]


    > So dict is a subclass of Mapping, even though none of the bases of
    > dict is either Mapping or a subclass of Mapping. Great.


    Right. dict is direct concrete Mapping implementation and not subclassed
    from one.

    The main reason for introducing ABCs was to make it easier to test
    whether an object passed to a function is an instance of a possibly
    unknown or yet-to-be-written class in an abstract category, which has a
    certain api or interface. The actual usage of an ABC would be more like
    this:

    from collections import Mapping
    def f(m):
    if not isinstance(m, Mapping):
    raise ValueError('input is not a mapping')
    else: return True

    f(dict())
    # True
    f([])
    # produces
    Traceback (most recent call last):
    File "<pyshell#8>", line 1, in <module>
    f([])
    File "<pyshell#6>", line 3, in f
    raise ValueError('input is not a mapping')
    ValueError: input is not a mapping

    --
    Terry Jan Reedy
    Terry Reedy, Dec 22, 2010
    #7
  8. On Wed, 22 Dec 2010 14:20:51 +0000, kj wrote:

    > Here's another example, fresh from today's crop of wonders:
    >
    > (v. 2.7.0)
    >>>> from collections import Mapping
    >>>> issubclass(dict, Mapping)

    > True
    >>>> dict.__bases__

    > (<type 'object'>,)
    >>>> [issubclass(b, Mapping) for b in dict.__bases__]

    > [False]
    >
    >
    > So dict is a subclass of Mapping, even though none of the bases of dict
    > is either Mapping or a subclass of Mapping. Great.


    Yes. So what?

    (1) What *actual* problem does this cause you?

    (2) Do you have an example of code that breaks because of this?

    (3) Do you understand that since the introduction of ABC (abstract base
    classes) in Python 2.6 (I think), isinstance and issubclass checks are
    performed cooperatively? The instance or class are asked if they wish to
    be known as an instance/subclass of the second argument. Classes can
    register themselves as subclasses of (say) Mapping without sharing any
    actual code with Mapping.

    This is a good thing, and the problem isn't that the abstraction leaks,
    as you believe, but the opposite: you're *ignoring* the abstraction and
    looking for concrete details that may or may not exist.

    I fear that you have fundamentally misunderstood the concept of "leaky
    abstraction". It does not mean, as you seem to think, that some concrete
    implementation detail differs between two classes (or functions). It
    means that some difference in behaviour is exposed, that difference being
    irrelevant to the abstraction but nevertheless important in some other
    sense. A contrived example:

    class MyList(list):
    def __len__(self):
    import time
    time.sleep(3600000)
    return list.__len__(self)

    MyList can be used anywhere a regular list can be used. Functionally the
    two are identical. The abstraction is that MyList is the same as list.
    But the leak is that len(MyList()) is *incredibly* slow.


    Coming back to Mapping:

    Abstraction: issubclass(dict, Mapping)

    One possible concrete implementation detail of how issubclass is
    implemented:
    any(base is Mapping for base in dict.__bases__)


    The statement "dict is a subclass of Mapping" is about an abstract
    relationship. It's not necessarily a statement about __bases__.

    To give an analogy, if you insist on doing DNA testing to determine
    whether a boy is a son of a man, you're going to be confused and
    distressed every time you find fathers whose sons are genetically
    unrelated to them.


    --
    Steven
    Steven D'Aprano, Dec 22, 2010
    #8
  9. kj

    kj Guest

    In <4d127d5e$0$29997$c3e8da3$> Steven D'Aprano <> writes:

    >On Wed, 22 Dec 2010 14:20:51 +0000, kj wrote:


    >> Here's another example, fresh from today's crop of wonders:
    >>
    >> (v. 2.7.0)
    >>>>> from collections import Mapping
    >>>>> issubclass(dict, Mapping)

    >> True
    >>>>> dict.__bases__

    >> (<type 'object'>,)
    >>>>> [issubclass(b, Mapping) for b in dict.__bases__]

    >> [False]
    >>
    >> So dict is a subclass of Mapping, even though none of the bases of dict
    >> is either Mapping or a subclass of Mapping. Great.



    >Yes. So what?


    That's being deliberately obtuse. The situation described goes
    smack against standard OOP semantics, which would be fine if all
    this stuff was documented clearly and reasonably, i.e. in one
    (preferably "official") place rather than scattered over a bazillion
    separate documents, PEP this, module that, GvR musing #42, etc.

    Let's just say that I'm looking forward to the end to these surprises.

    ~kj
    kj, Dec 23, 2010
    #9
  10. On Thu, 23 Dec 2010 01:41:08 +0000, kj wrote:

    > In <4d127d5e$0$29997$c3e8da3$> Steven D'Aprano
    > <> writes:
    >
    >>On Wed, 22 Dec 2010 14:20:51 +0000, kj wrote:

    >
    >>> Here's another example, fresh from today's crop of wonders:
    >>>
    >>> (v. 2.7.0)
    >>>>>> from collections import Mapping
    >>>>>> issubclass(dict, Mapping)
    >>> True
    >>>>>> dict.__bases__
    >>> (<type 'object'>,)
    >>>>>> [issubclass(b, Mapping) for b in dict.__bases__]
    >>> [False]
    >>>
    >>> So dict is a subclass of Mapping, even though none of the bases of
    >>> dict is either Mapping or a subclass of Mapping. Great.

    >
    >
    >>Yes. So what?

    >
    > That's being deliberately obtuse. The situation described goes smack
    > against standard OOP semantics,



    What are these "standard OOP semantics" you're referring to, and who made
    them "standard"? If people can't even decide whether multiple inheritance
    should be allowed or not, what makes you think that there is any such
    thing as "standard OOP"?

    I think you are confusing concrete implementation details with the
    interface. The interface for subclass testing in Python is as follows:

    A class K is a subclass of class C if, and only if,
    issubclass(C, K) returns a true result.


    That's it. Everything else is implementation. __bases__ is
    implementation. __subclasscheck__ is implementation. If Python adds a
    third mechanism for implementing subclass checks in version 3.3 or 3.4,
    the interface will just continue to work correctly.



    > which would be fine if all this stuff
    > was documented clearly and reasonably, i.e. in one (preferably
    > "official") place rather than scattered over a bazillion separate
    > documents, PEP this, module that, GvR musing #42, etc.


    This is documented, in the docs.

    http://docs.python.org/reference/datamodel.html#customizing-instance-and-subclass-checks

    Nobody says the Python docs are perfect, but most things you ask are in
    there. Patches for the docs to improve them are welcome.



    --
    Steven
    Steven D'Aprano, Dec 23, 2010
    #10
  11. kj

    Aahz Guest

    In article <ieu9bk$mt0$>, kj <> wrote:
    >
    >standard OOP semantics


    "...some experts might say a C++ program is not object-oriented without
    inheritance and virtual functions. As one of the early Smalltalk
    implementors myself, I can say they are full of themselves." --zconcept
    --
    Aahz () <*> http://www.pythoncraft.com/

    "The volume of a pizza of thickness 'a' and radius 'z' is
    given by pi*z*z*a"
    Aahz, Jan 18, 2011
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris Green

    when does issubclass fail?

    Chris Green, Apr 16, 2004, in forum: Python
    Replies:
    1
    Views:
    290
    =?ISO-8859-1?Q?Walter_D=F6rwald?=
    Apr 20, 2004
  2. Nathan Bullock

    How do you make issubclass work

    Nathan Bullock, Sep 11, 2004, in forum: Python
    Replies:
    4
    Views:
    606
    Alex Martelli
    Sep 11, 2004
  3. Osmo Maatta
    Replies:
    3
    Views:
    466
    Osmo Maatta
    Aug 29, 2010
  4. kj
    Replies:
    5
    Views:
    398
  5. Replies:
    1
    Views:
    136
    Steven D'Aprano
    May 31, 2012
Loading...

Share This Page