Need a better understanding on how MRO works?

[Steven W. Orr]

Given the following code: (I hope it's as simple as possible)
#! /usr/bin/python
import new
class BASE:
def __init__( self ):
print 'Hello from BASE init'
def m1( self ):
print 'M1 Base: Self = ', self

def m1replace( self ):
print 'm1replace:Self = ', self

class D1(BASE):
def __init__(self):
BASE.__init__(self)

def __InitDS101Classes():
name = 'C1'
nclass = new.classobj(name,(D1,),globals())
globals()[name] = nclass
name = 'C2'
nclass = new.classobj(name,(D1,),globals())
globals()[name] = nclass
globals()[name].m1 = m1replace

__InitDS101Classes()

s = C1()
s.m1()
t = C2()
t.m1()

I get the following output:

1100 > ./foo1.py
Hello from BASE init
m1replace:Self = <__main__.C1 instance at 0xb7e637cc>
Hello from BASE init
m1replace:Self = <__main__.C2 instance at 0xb7e6388c>

But if I make BASE inherit from object
class BASE(object):
then I get this:

1100 > ./foo1.py
Hello from BASE init
m1replace:Self = <__main__.NewClass instance at 0xb7f5070c>
Hello from BASE init
M1 Base: Self = <__main__.D1 instance at 0xb7f5088c>

Can someone please explain why the assignment to C2.m1 would overwrite
BASE.m1?

TIA

--
Time flies like the wind. Fruit flies like a banana. Stranger things have .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

 

[Alex Martelli]

name = 'C1'
nclass = new.classobj(name,(D1,),globals())
globals()[name] = nclass

Here, you're creating a VERY anomalous class C1 whose __dict__ is
globals(), i.e. the dict of this module object;

name = 'C2'
nclass = new.classobj(name,(D1,),globals())
globals()[name] = nclass

and here you're creating another class with the SAME __dict__;

globals()[name].m1 = m1replace

So of course this assignment affects the 'm1' entries in the dict of
both classes, since they have the SAME dict object (a la Borg) -- that
is, IF they're old-style classes (i.e. if D1 is old-style), since in
that case a class's __dict__ is in fact a dict object, plain and simple.

However, if D1 is new-style, then C1.__dict__ and C2.__dict__ are in
fact instances of <dictproxy> -- each with a copy of the entries that
were in globals() when you called new.classobj, but DISTINCT from each
other and from globals(), so that further changes in one (or globals)
don't affect globals (nor the other).

I guess this might be a decent interview question if somebody claims to
be a "Python guru": if they can make head or tails out of this mess, boy
the *ARE* a Python guru indeed (in fact I'd accord minor guruhood even
to somebody who can get a glimmer of understanding of this with ten
minutes at a Python interactive prompt or the like, as opposed to
needing to understand it "on paper" without the ability to explore.

Among the several "don't"s to learn from this: don't use old-style
classes, don't try to make two classes share the same dictionary, and
don't ask about MRO in a question that has nothing to do with MRO
(though I admit that was a decent attempt at misdirection, it wouldn't
slow down even the minor-guru in any appreciable way.


Alex

 

[Steven W. Orr]

On Saturday, Aug 25th 2007 at 17:19 -0700, quoth Alex Martelli:

=> ...
=>> name = 'C1'
=>> nclass = new.classobj(name,(D1,),globals())
=>> globals()[name] = nclass
=>
=>Here, you're creating a VERY anomalous class C1 whose __dict__ is
=>globals(), i.e. the dict of this module object;
=>
=>> name = 'C2'
=>> nclass = new.classobj(name,(D1,),globals())
=>> globals()[name] = nclass
=>
=>and here you're creating another class with the SAME __dict__;
=>
=>> globals()[name].m1 = m1replace
=>
=>So of course this assignment affects the 'm1' entries in the dict of
=>both classes, since they have the SAME dict object (a la Borg) -- that
=>is, IF they're old-style classes (i.e. if D1 is old-style), since in
=>that case a class's __dict__ is in fact a dict object, plain and simple.
=>
=>However, if D1 is new-style, then C1.__dict__ and C2.__dict__ are in
=>fact instances of <dictproxy> -- each with a copy of the entries that
=>were in globals() when you called new.classobj, but DISTINCT from each
=>other and from globals(), so that further changes in one (or globals)
=>don't affect globals (nor the other).
=>
=>I guess this might be a decent interview question if somebody claims to
=>be a "Python guru": if they can make head or tails out of this mess, boy
=>the *ARE* a Python guru indeed (in fact I'd accord minor guruhood even
=>to somebody who can get a glimmer of understanding of this with ten
=>minutes at a Python interactive prompt or the like, as opposed to
=>needing to understand it "on paper" without the ability to explore.
=>
=>Among the several "don't"s to learn from this: don't use old-style
=>classes, don't try to make two classes share the same dictionary, and
=>don't ask about MRO in a question that has nothing to do with MRO
=>(though I admit that was a decent attempt at misdirection, it wouldn't
=>slow down even the minor-guru in any appreciable way.
=>
=>
=>Alex
=>--
=>http://mail.python.org/mailman/listinfo/python-list
=>

Thanks Alex. I am humbled, though I was before I started.
I really don't have a lot of understanding of what you're saying so I'll
probably have to study this for about a year or so.

* (I need to look up what dictproxy is.) I don't have any idea what the
ramifications are of your use of the word DISTINCT. Are you somehow
suggesting that new.classobj does a deep copy of the globals copy that's
passed to it?

* Also, I'd like to understand what the difference is between
nclass = new.classobj(name,(D1,),globals())
vs.
def classfactory():
class somename(object):
def somestuff():
pass
return somename
G1 = classfactory()
globals()[name] = G1

Does new.classobj do anything special?

I appreciate your time.

--
Time flies like the wind. Fruit flies like a banana. Stranger things have .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

 

[Alex Martelli]

Thanks Alex. I am humbled, though I was before I started.
I really don't have a lot of understanding of what you're saying so I'll
probably have to study this for about a year or so.

* (I need to look up what dictproxy is.) I don't have any idea what the
ramifications are of your use of the word DISTINCT. Are you somehow
suggesting that new.classobj does a deep copy of the globals copy that's
passed to it?

No, most definitely NOT deep!!!, but type.__new__ does "a little" of
what you've said (a shallow copy, which is not quite "a copy" because it
embeds [some of] the entries in slots). new.classobj determines the
metaclass (from the bases, or a __metaclass__ entry in the dictionary)
and calls it to generate the new class. For modern style classes, the
class is type; for old-style legacy classes, it's types.ClassType, and
they're not exactly identical in behavior (of course not, or there would
no point in having both.

* Also, I'd like to understand what the difference is between
nclass = new.classobj(name,(D1,),globals())
vs.
def classfactory():
class somename(object):
def somestuff():
pass
return somename
G1 = classfactory()
globals()[name] = G1

Does new.classobj do anything special?

No, new.classobj does essentially the same thing that Python does after
evaluating a class statement to prepare the class's name, bases and
dictionary: finds the metaclass and calls it with these arguments.

A key difference of course is that a class statement prepares the class
dictionary as a new, ordinary, distinct dictionary, while new.classobj
accepts whatever dictionary you give it (so you can, though shouldn't,
do strange things such as pass globals()....


Alex

 

[Steven W. Orr]

On Saturday, Aug 25th 2007 at 22:14 -0700, quoth Alex Martelli:


=>> * Also, I'd like to understand what the difference is between
=>> nclass = new.classobj(name,(D1,),globals())
=>> vs.
=>> def classfactory():
=>> class somename(object):
=>> def somestuff():
=>> pass
=>> return somename
=>> G1 = classfactory()
=>> globals()[name] = G1
=>>
=>> Does new.classobj do anything special?
=>
=>No, new.classobj does essentially the same thing that Python does after
=>evaluating a class statement to prepare the class's name, bases and
=>dictionary: finds the metaclass and calls it with these arguments.
=>
=>A key difference of course is that a class statement prepares the class
=>dictionary as a new, ordinary, distinct dictionary, while new.classobj
=>accepts whatever dictionary you give it (so you can, though shouldn't,
=>do strange things such as pass globals()....

In fact, I wanted to make a common routine that could be called from
multiple modules. I have classes that need to be created from those
multiple modules. I did run into trouble when I created a common routine
even though I passed globals() as one of the args. The """though
shouldn't""" is prompting me to ask why, and where I might be able to read
more.

--
Time flies like the wind. Fruit flies like a banana. Stranger things have .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

 

[Alex Martelli]

=>accepts whatever dictionary you give it (so you can, though shouldn't,
=>do strange things such as pass globals()....

In fact, I wanted to make a common routine that could be called from
multiple modules. I have classes that need to be created from those
multiple modules. I did run into trouble when I created a common routine
even though I passed globals() as one of the args. The """though
shouldn't""" is prompting me to ask why, and where I might be able to read
more.

The dictionary you pass to new.classobj should be specifically
constructed for the purpose -- globals() will contains all sort of odds
and ends that have nothing much to do with the case.

You appear to be trying to embody lot of black magic in your "common
routine", making it communicate with its callers by covert channels; the
way you use globals() to give that routine subtle "side effects" (making
the routine stick entries there) as well as pass it an opaque,
amorphous, unknown blobs of input information, strongly suggests that
the magic is running away with you (a good general reference about that
is <http://video.google.com/videoplay?docid=4611491525028588899>).

"Explicit is better than implicit", "simple is better than complex",
etc, can be read by typing ``import this'' at an interactive Python
prompt.

The best book I know about the do's and don't's of large-scale software
architecture is Lakos' "Large-Scale C++ Software Design",
<http://www.amazon.com/Large-Scale-Software-Design-John-Lakos/dp/0201633
620> -- very C++ specific, but even though some of the issues only apply
to C++ itself, many of its crucial lessons will help with large scale SW
architecture in just about any language, Python included.

What I had to say about the lures and pitfalls of black magic in Python
specifically is spread through the Python Cookbook 2nd edition (and, to
a lesser extent, Python in a Nutshell).


Alex

 

[Steven D'Aprano]

In fact, I wanted to make a common routine that could be called from
multiple modules. I have classes that need to be created from those
multiple modules. I did run into trouble when I created a common routine
even though I passed globals() as one of the args.

I'm thinking that you should cross out "even though" and insert
"because"

Perhaps I'm misunderstanding exactly what you're trying to accomplish,
but if you want to call a routine (a function? class?) from multiple
modules, the simplest way is the best: create the routine in one module,
then import it into all the others.