metaclass and customization with parameters

[zipher]

Thanks everyone who responded with such care to my message.

Mike Fletcher's suggestion hit my original intent and source of my
confusion right on the nose. The suggestion seems so obvious now.
But I am carefully reviewing the effects of Alex's hammer on my head
and may be reconsidering...

zipher

 

[Gustavo Niemeyer]

Hello Carlos,

I'm not a metaclass expert, but I think I can share a few useful tips
with you. I've been studying metaclasses lately, and I've fallen on
some traps that seem to be common. I apologize if my tone sound
pretentious at times -- my intention is probably better than my words.

Your description is really interesting and informative,
as usual.

[...]

... for i in range(10):
... def newmethod(self,i=i):
... print "I'm method #%d" % i
... newname = "method%d" % i
... locals()[newname] = newmethod
... I'm method #1

Although it doesn't involve metaclasses at all, it shows how dynamic
the class creation process is, and how much control one can have with
it. There are two worthy comments to make:

1) the locals() dict is the key at this step of class creation. The
local dict is one of the parameters passed to the metaclass. If you
fill the locals() with values -- either manually, as in the example
above, or by assigning values to local vars -- all the values will be
part of the class definition.

As an interesting side effect, the class above will have
'newname', 'newmethod', and 'i' attributes, besides 'method0-9'.

2) the signature of "def newmethod(self,i=i)" shows another
interesting side effect. The value if <i> (defined in the for loop) is
passed as a keyword paramenter, and is evaluated whenever the 'def'
statement is found. Because it changes, the def is re-run at every
loop. In the lack of this hack, the def would run only at the first
time, and all methods would in fact be the same.

That's not true. The def statement is rerun every loop, no matter
what parameters are used. One can check that by issuing:

print MyClass.method0.im_func
print MyClass.method1.im_func

Of course, in your example, not using the 'i=i' trick would be a
problem, since the 'i' variable is lost, and inside the method a
global is looked for, creating an error.

 

[Carlos Ribeiro]

That's not true. The def statement is rerun every loop, no matter
what parameters are used. One can check that by issuing:

print MyClass.method0.im_func
print MyClass.method1.im_func

You *right*. For some reason I was under the impression that the
code was not re-executed everytime -- a kind of optimization, indeed,
if the code does not have any external dependency. But you're right.

Anyway, this is a horrible hack I just wrote it to point out that
classes are much more dynamic in their workings than most people
realize, *specially* if you come from a C++/Java/Delphi background.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Andrew Durdin]

That's not true. The def statement is rerun every loop, no matter
what parameters are used. One can check that by issuing:

print MyClass.method0.im_func
print MyClass.method1.im_func

Hmmm... If I create MyClass as above, I get the following results
which I don't understand:
9355504

Why do mc.method0 and mc.method1 appear to be the same object?

Even more confusing to me is:
.... def method0(self):
.... pass
.... def method1(self):
.... pass
.... def method2(self):
.... pass
....9918464

Here f.method1 and f.method2 share the same id, which is different to
that of f.method1...

Can anyone enlighten me as to why this happens?

 

[Jack Diederich]

Hmmm... If I create MyClass as above, I get the following results
which I don't understand:

<bound method MyClass.newmethod of <__main__.MyClass instance at 0x008E93A0>>

Why do mc.method0 and mc.method1 appear to be the same object?

Read it more closely, they are just saying they are bound methods of the same
object instance, not that they are the same object themselves.

Even more confusing to me is:

... def method0(self):
... pass
... def method1(self):
... pass
... def method2(self):
... pass
...
9918464

Here f.method1 and f.method2 share the same id, which is different to
that of f.method1...

Can anyone enlighten me as to why this happens?

A temporary object is being made when you lookup 'method2' in 'f'
Try assigning the value to a local variable and then running id() on those
local variables. The id()s should remain stable. What you are seeing is
just a fluke, the same id gets used and then scrapped and then used again.

-Jack

 

[Carlos Ribeiro]

....


A temporary object is being made when you lookup 'method2' in 'f'
Try assigning the value to a local variable and then running id() on those
local variables. The id()s should remain stable. What you are seeing is
just a fluke, the same id gets used and then scrapped and then used again.

Check this (from the same code snippet you've written), running on my
PC with 2.3:
<function method2 at 0x01733330>

I think this is because methods are now implemented as descriptors,
and as such, are not required to return stable id values as one would
expect. The im_func entry is clearly different, even if first part
(the id() listings) seems to corroborate your findings.


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Alex Martelli]

class MyClass:

... for i in range(10):
... def newmethod(self,i=i):
... print "I'm method #%d" % i
... newname = "method%d" % i
... locals()[newname] = newmethod

...
As an interesting side effect, the class above will have
'newname', 'newmethod', and 'i' attributes, besides 'method0-9'.

....which incidentally shows that i is a +local+ variable...

Of course, in your example, not using the 'i=i' trick would be a
problem, since the 'i' variable is lost, and inside the method a
global is looked for, creating an error.

True, the opcode for i is indeed a LOAD_GLOBAL. Why _that_ should be
the case is anything but an 'of course' to _me_, though, as we just
showed that i is a +local+. newmethod is not seen as a nested function
of the anonymous function built from the codeobject's that's the
classbody. I _think_ there's some adhoccery in the Python compiler (not
the easiest part of Python to read and understand, so I apologize for
any potential confusion here) to specifically ensure that -- I guess
it's for backwards compatibility with old Python versions which didn't
have nested scopes, otherwise any barename in a method would refer first
to the classbody (acting as its 'outer function') which could shadow
globals. A bit confusing for an "of course", nevertheless;-).


Alex

 

[Gustavo Niemeyer]

Hello Alex!

True, the opcode for i is indeed a LOAD_GLOBAL. Why _that_ should be
the case is anything but an 'of course' to _me_, though, as we just

Sorry, my intention was to say that removing "i=i" would be
clearly a problem, not that the whole issue was clear.

showed that i is a +local+. newmethod is not seen as a nested function
of the anonymous function built from the codeobject's that's the
classbody. I _think_ there's some adhoccery in the Python compiler (not
the easiest part of Python to read and understand, so I apologize for
any potential confusion here) to specifically ensure that -- I guess
it's for backwards compatibility with old Python versions which didn't
have nested scopes, otherwise any barename in a method would refer first

In this case, the external closure must be the global scope, otherwise,
code like this:

class C:
var = 1
def method(self):
print var

Would show self.__class__.var, when it shouldn't.

to the classbody (acting as its 'outer function') which could shadow
globals. A bit confusing for an "of course", nevertheless;-).

Again, I'm sorry. My "of course" should be more closely related to the
fact that removing "i=i" would be a problem, as was my original
intention. Even if the internal 'i' was bound to a closure, as you
seem to belive would be the expected behavior, the OP's code would end
up doing the wrong thing.

Here is an example:
.... l = []
.... for i in range(10):
.... def subfunc():
.... print i
.... l.append(subfunc)
.... return l
........ subfunc()
....
9
9
9
9
[...]

 

[Alex Martelli]

Hello Alex!


Sorry, my intention was to say that removing "i=i" would be
clearly a problem, not that the whole issue was clear.

Sorry, I should have inserted smilies -- I did get your intended meaning
and was just playing around with the words!

In this case, the external closure must be the global scope, otherwise,
code like this:

class C:
var = 1
def method(self):
print var

Would show self.__class__.var, when it shouldn't.

IMHO the only reason it shouldn't is backwards compatibility with old
Python versions, as I mentioned above. It's a good reason, mind you,
but maybe in Python 3000 we can rethink this issue, too.

Again, I'm sorry. My "of course" should be more closely related to the
fact that removing "i=i" would be a problem, as was my original
intention. Even if the internal 'i' was bound to a closure, as you
seem to belive would be the expected behavior, the OP's code would end
up doing the wrong thing.

Exactly, and _that_ would be the normal way to fail -- omitting to
"snapshot" some variables, of which we want the _current_ value, and
which may still be changing... (incidentally a strong argument against
certain... hmmmm... "beings", who claim it's "undisputably wrong" that
default values get evaluated ONCE, at def-time...).


Alex