Re-executing the code object from a class 'declaration'

C

Carlos Ribeiro

As part of a pet project of mine, I needed to find a way to re-execute
a class declaration[1]. The reason behind it was simple: the class
declaration in question had some external dependencies that could
potentially change during the lifetime of the program. I needed to be
able to execute the class 'declaration' block again to produce new
classes (in opposition to new instances, which would not solve my
problem)[2].

I solved the problem using inspect to find the current code object
*inside* the class declaration, and stored it an attribute of the
class itself. Then, a method of the class can be used to re-create it.
A metaclass-enabled version is shown below:

class Meta(type):
def __new__(klass, name, bases, dct):
dct['my_code'] = inspect.currentframe(1).f_code
return type(name, bases, dct)

class MyClass:
__metaclass__ = Meta
creation_time = time.ctime()
print "I'm inside my class definition!"
def re_execute(klass):
exec klass.my_code
return MyClass
re_execute = classmethod(re_execute)

Testing:
...
I'm inside my class definition!'Tue Oct 05 21:51:20 2004'

Implementation notes:

1) The exec statement actually binds the name to the same original
name, but in the current local scope. That's why it does a 'return
MyClass' in the re_execute() method. The MyClass symbol refers to the
local symbol with this name that was just re-created by the exec
method.

2) If you call the exec from the same scope where the class was
originally created, then it is automatically bound to the same name:
'Tue Oct 05 21:56:53 2004'

3) I thought that the same technique could be made to work without a
metaclass, but I couldn't make it work. I haven't invested much time
to understand why -- it just works as implemented above. (It seems
that it does not bind correctly to the original name as it does if you
use the metaclass)

----
[1] After a long and interesting discussion for which I have to thank
Alex Martelli and several other members of c.l.py, I've come to
realize that there are no real class declarations in Python. But for
all effects, it's just easier to call them this way, even if the
underlying mechanism is different from the one used on other
traditional languages.

[2] Think about it as a templating mechanism, similar to the C++ one,
but for Python; this mechanism generates new classes that can be
instantiated as many times as needed. It's not the same thing as to
customize a instance during its initialization.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
A

Alex Martelli

Carlos Ribeiro said:
As part of a pet project of mine, I needed to find a way to re-execute
a class declaration[1]. The reason behind it was simple: the class

If you start calling it 'class statement' I may read and perhaps help
with the rest of your post, but since I consider calling 'declaration'
something that is no more and no less than an executable statement an
excellent way to try and confuse the issues (by leading many readers to
expect something completely different from the reality), I'm going to
personally boycott any post deliberately perpetrating such wilful use of
language to confuse readers, and I urge the same course of action on
anybody who cares about clarity, understanding and communication.


Alex
 
S

Steve Holden

Alex said:
As part of a pet project of mine, I needed to find a way to re-execute
a class declaration[1]. The reason behind it was simple: the class


If you start calling it 'class statement' I may read and perhaps help
with the rest of your post, but since I consider calling 'declaration'
something that is no more and no less than an executable statement an
excellent way to try and confuse the issues (by leading many readers to
expect something completely different from the reality), I'm going to
personally boycott any post deliberately perpetrating such wilful use of
language to confuse readers, and I urge the same course of action on
anybody who cares about clarity, understanding and communication.


Alex

How would you feel about "class definition"?

regards
Steve
 
A

Alex Martelli

Steve Holden said:
As part of a pet project of mine, I needed to find a way to re-execute
a class declaration[1]. The reason behind it was simple: the class
...
If you start calling it 'class statement' I may read and perhaps help
with the rest of your post, but since I consider calling 'declaration'
...
How would you feel about "class definition"?

Since we have a 'def' statement it seems confusing to me to talk of
'definition' about something else than a 'def' (even there I prefer "a
def statement" as clearer and more direct, but I guess I'll have to
accept "a function definition" given ``def''... I do wish that keyword
was 'function' rather than 'def', but, oh well, too late for that;-).


Alex
 
C

Carlos Ribeiro

In respectful consideration to Alex's opposition to the misuse of the
word 'declaration', here is a repost with the suggested changes:
---
As part of a pet project of mine, I needed to find a way to re-execute
a class statement. The reason behind it was simple: the class
statement in question had some external dependencies that could
potentially change during the lifetime of the program. I needed to be
able to execute the class statement block again to produce new
classes (in opposition to new instances, which would not solve my
problem)[1].

I solved the problem using inspect to find the current code object
*inside* the class statement, and stored it in an attribute of the
class itself. Then, a method of the class can be used to re-create it.
A metaclass-enabled version is shown below:

class Meta(type):
def __new__(klass, name, bases, dct):
dct['my_code'] = inspect.currentframe(1).f_code
return type(name, bases, dct)

class MyClass:
__metaclass__ = Meta
creation_time = time.ctime()
print "I'm inside my class definition!"
def re_execute(klass):
exec klass.my_code
return MyClass
re_execute = classmethod(re_execute)

Testing:
...
I'm inside my class definition!'Tue Oct 05 21:51:20 2004'

Implementation notes:

1) The exec statement actually binds the name to the same original
name, but in the current local scope. That's why it does a 'return
MyClass' in the re_execute() method. The MyClass symbol refers to the
local symbol with this name that was just re-created by the exec
method.

2) If you call the exec from the same scope where the class was
originally created, then it is automatically bound to the same name:
'Tue Oct 05 21:56:53 2004'

3) I thought that the same technique could be made to work without a
metaclass, but I couldn't make it work. I haven't invested much time
to understand why -- it just works as implemented above. (It seems
that it does not bind correctly to the original name as it does if you
use the metaclass)

----
[1] Think about it as a templating mechanism, similar to the C++ one,
but for Python; this mechanism generates new classes that can be
instantiated as many times as needed. It's not the same thing as to
customize a instance during its initialization.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
A

Alex Martelli

Carlos Ribeiro said:
As part of a pet project of mine, I needed to find a way to re-execute
a class statement. The reason behind it was simple: the class
statement in question had some external dependencies that could
potentially change during the lifetime of the program. I needed to be
able to execute the class statement block again to produce new
classes (in opposition to new instances, which would not solve my
problem)[1].

Interesting problem.

I solved the problem using inspect to find the current code object
*inside* the class statement, and stored it in an attribute of the
class itself. Then, a method of the class can be used to re-create it.
A metaclass-enabled version is shown below:

class Meta(type):
def __new__(klass, name, bases, dct):
dct['my_code'] = inspect.currentframe(1).f_code
return type(name, bases, dct)

class MyClass:
__metaclass__ = Meta
creation_time = time.ctime()
print "I'm inside my class definition!"
def re_execute(klass):
exec klass.my_code
return MyClass
re_execute = classmethod(re_execute)

Urk. Ahem, I mean, interesting solution. I _think_ it poses certain,
ahem, problems, such as infinite recursion if a call to re_execute is in
the same code object as that where the 'class MyClass' is, for example
-- because basically you ARE re-executing the whole kaboodle. You're
not just re-executing the 'class' statement, but everything that comes
before or after it in the same containing codeobject. Your tests in an
interactive interpreter are tricky that way, because in that special
environment a code object is a tiny little thing. But stick this into a
module and place a dis.dis(dct['my_code']) in the metaclass's __new__
and you'll see that basically MyClass.re_execute isn't very different
from a reload of the module. Isn't that a _bit_ too much in general?

The code object for the body of the class can also be found (with some
work): it will be among the co_consts of what you've stored as
dct['my_code'], it will be an instance of types.CodeType, and its
co_name will be the same as the 'name' parameter to Meta.__new__. If
you have more than one code object with those characteristics, e.g. two
classes with the same name at the same module's toplevel, you do have
some trouble. But otherwise, you can build a function, with
new.function(thatclassbodycodeobject, theglobalsfrominspect), which,
when called (w/o arguments), reexecutes the classbody and returns the
same kind of dictionary that Meta.__new__ is receiving as its 'dct'
argument. Wouldn't this be preferable to re-executing the whole module
body, perhaps?
Implementation notes:

1) The exec statement actually binds the name to the same original
name, but in the current local scope. That's why it does a 'return
MyClass' in the re_execute() method. The MyClass symbol refers to the
local symbol with this name that was just re-created by the exec
method.

This also fails if you have multiple uses of name MyClass at toplevel of
the same module -- it actually fails more often than my suggestions
above, e.g if your module is:

class MyClass:
... etc ...
FakeName = MyClass
MyClass = 23

then your FakeName.re_execute() will return 23, while the approach I
suggest will be able to recover in this case (the codeobject in the
co_consts of the module's code object will still be the only one in that
container that's named 'MyClass').

2) If you call the exec from the same scope where the class was
originally created, then it is automatically bound to the same name:

Same issue as above -- you're getting closer and closer to a module's
reload (save in the anomalous case where you type the whole class
statement at an interactive prompt;-). Note that it IS quite possible
that what you want to do IS in fact to reload the module, period, but,
if so, then the reload statement might be handier.
3) I thought that the same technique could be made to work without a
metaclass, but I couldn't make it work. I haven't invested much time
to understand why -- it just works as implemented above. (It seems
that it does not bind correctly to the original name as it does if you
use the metaclass)

It appears to me, but I haven't tested this, that putting the inspect
introspection in the classbody itself should be a sounder technique, as
it should be able to find itself and stash itself away more reliably
than just based on name (which is the best the metaclass can do). The
name binding issue is separate, however.
[1] Think about it as a templating mechanism, similar to the C++ one,
but for Python; this mechanism generates new classes that can be
instantiated as many times as needed. It's not the same thing as to
customize a instance during its initialization.

I agree, there's one metalevel of difference between instantiating a
class to make an instance, and instantiating a metaclass to make a
class.


Alex
 
C

Carlos Ribeiro

Urk. Ahem, I mean, interesting solution. I _think_ it poses certain,
ahem, problems, such as infinite recursion if a call to re_execute is in
the same code object as that where the 'class MyClass' is, for example
-- because basically you ARE re-executing the whole kaboodle. You're
not just re-executing the 'class' statement, but everything that comes
before or after it in the same containing codeobject. Your tests in an
interactive interpreter are tricky that way, because in that special
environment a code object is a tiny little thing. But stick this into a
module and place a dis.dis(dct['my_code']) in the metaclass's __new__
and you'll see that basically MyClass.re_execute isn't very different
from a reload of the module. Isn't that a _bit_ too much in general?

(repeating to myself: dis is your friend. dis is your friend).

Well, it seems that I'm running out of options. I've checked dis, and
done a few more tests. I found a way to find the correct code object
for the _body_ of the class statement (or at least I think so), but
this does not solve the problem, as it will be shown below:

-------------
import inspect
import dis
import time
import types

class Meta(type):
def __new__(klass, name, bases, dct):
caller = inspect.stack(0)[1]
caller_code = caller[0].f_code
caller_filename = caller[1]
caller_lineno = caller[2]
print "Caller info: ", caller_filename, caller_lineno, caller_code
print dis.dis(caller[0].f_code)
for code in caller_code.co_consts:
if isinstance(code, types.CodeType):
if (code.co_filename == caller_filename) and \
(code.co_firstlineno == caller_lineno):
print "found!", code.co_name
dct['my_code'] = code
return type(name, bases, dct)

class MyClass(object):
__metaclass__ = Meta
creation_time = time.ctime()
print "I'm inside my class definition!"
def re_execute(klass):
exec klass.my_code
return MyClass
re_execute = classmethod(re_execute)
-------------

The code checks the filename and the line number to find the correct
code object. It could also check the class name, just to make sure
that everything matches. It isn't needed, as far as I know. But the
problem is that the code block that is found refers to the class
_body_ only. Checking dis.dis() helps to understand it:

27 64 LOAD_CONST 4 ('MyClass')
67 BUILD_TUPLE 0
70 LOAD_CONST 5 (<code object MyClass at
01209E60, file "c:\work\help-on-c-l-py\re_exec_class.py", line 27>)
73 MAKE_FUNCTION 0
76 CALL_FUNCTION 0
79 BUILD_CLASS
80 STORE_NAME 7 (MyClass)

There is not a single code block that does all the tasks related to
the class statement, and only it. In fact, there are two different
code objects to choose from, and neither really solves the problem.
One is the module object; as you've pointed out, it's too generic and
executes too much stuff, not just the class statement. The other one
is the <code object MyClass at 01209E60> referred to in the dis()
output above, that is the _body_ of the class statement. Calling it
doesn't call the metaclass, and neither does the name binding 'magic':
I'm inside my class definition!
# it does not print any of the debug messages in the metaclass, and
it's obvious why

It also doesn't take into account that the class may have base
classes. For example, changing the statement to 'class
MyClass(object)', the compiled code looks like this:

27 64 LOAD_CONST 4 ('MyClass')
67 LOAD_NAME 7 (object)
70 BUILD_TUPLE 1
From this point, few options are left. What I need to execute is in
fact a part of the module code object -- just one line of it, that's
the code that wraps the entire class statement. Besides giving up
(which is a reasonable option, given the constraints), what is left
are some horrible hacks. (I'm don't feel like discussing it here; it's
like Mordor Black Speech, if shall not be uttered in a public forum
:).

BTW, the two code objects represent exactly the two alternative
solutions that can be done using only pure Python code:

1) reload the entire module (which is nearly equivalent to calling the
module code object, with the difference that it doesn't reload and
recompile the actual source code)

2) enclose the class statement inside a def, and run the def to obtain
the class template:

def BuildTemplate(<template parameters>):
class MyTemplate(...):
...
return MyTemplate

For my application, I'm tending towards (2), because it allows for a
better modularization of the code. But I'm sorry, in a way, because it
breaks the expect semantics in a way -- it's a function that returns a
class. It's valid, but some may regard it as 'ugly' and 'unpythonic'.
It appears to me, but I haven't tested this, that putting the inspect
introspection in the classbody itself should be a sounder technique, as
it should be able to find itself and stash itself away more reliably
than just based on name (which is the best the metaclass can do). The
name binding issue is separate, however.

I've tried it yesterday, and it won't work, for some still unknown
reason. I could find the code object, but when I ran it, it would not
bind to the name in the same way as the metaclass would do. But it
doesn't matter at this point.
[1] Think about it as a templating mechanism, similar to the C++ one,
but for Python; this mechanism generates new classes that can be
instantiated as many times as needed. It's not the same thing as to
customize a instance during its initialization.

I agree, there's one metalevel of difference between instantiating a
class to make an instance, and instantiating a metaclass to make a
class.


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
C

Carlos Ribeiro

which does something close to what you ask for. Of course the trick
there is NOT recommended for real code ;)

That was the best advice I could have :) Really. I'm trying to outwit
Python, and one has to have the good sense to stop doing it at some
point. I'm giving up (unless I found a really great way to make it),
and I'll use this idiom instead:

def BuildTemplate(<template parameters>)
class TemplateClass(...):
...
return TemplateClass

It's simple, and less hackish, but still unusual. I fear I may be
missing something on my design, if I need to resort to such hacks.
--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
A

Alex Martelli

Carlos Ribeiro said:
module and place a dis.dis(dct['my_code']) in the metaclass's __new__
and you'll see that basically MyClass.re_execute isn't very different
from a reload of the module. Isn't that a _bit_ too much in general?

(repeating to myself: dis is your friend. dis is your friend).

If you're hacking at this level, it sure is;-).
done a few more tests. I found a way to find the correct code object
for the _body_ of the class statement (or at least I think so), but

Good -- as I said I think it can be even more reliably found from inside
the classbody itself, but, no big deal.
exec klass.my_code

Ah, this is your problem -- that's not what you want to do now.
problem is that the code block that is found refers to the class
_body_ only. Checking dis.dis() helps to understand it:

Exactly: it's what builds the dict you want to pass to the metaclass
(once you wrap it into a new.function, as I mentioned).
There is not a single code block that does all the tasks related to
the class statement, and only it. In fact, there are two different
code objects to choose from, and neither really solves the problem.

Not by itself, but add some code and you'll be fine.

1. You need to build an f = new.function(klass.my_code, globals()) in
your classmethod that now erroneously does the exec above quoted (you
could in fact do this in the metaclass, save the new function rather
than the code object -- get the right globals via inspect).

2. you need to obtain the needed dict, that's just d = f() in your
method

3. you can now call your metaclass with the appropriate bases and name:
klas = self.__class__ # or cls if this is a classmethod;-)
return type(klas)(klas.__name__, klas.__bases__, d)

voila, you're there. This is all coded to minimize the need to have
this method located in a specific class -- as I said I think you
probably want to have this stuff in the metaclass (if you have a
metaclass at all), but, whatever.


This is basically just mimicking what Python itself does (check the
dis.dis for a class statement as a starter): get the class body's code
object, build a function from it, call the function (it returns its
locals() as the result -- that 'return' is of course in the class body's
code object, and dis.dis shows that...), and that's how you get the
classdict on the stack. Add name and bases, call the metaclass, and
Bob's your uncle...


Alex
 
A

Alex Martelli

Carlos Ribeiro said:
That was the best advice I could have :) Really. I'm trying to outwit
Python, and one has to have the good sense to stop doing it at some
point. I'm giving up (unless I found a really great way to make it),
and I'll use this idiom instead:

def BuildTemplate(<template parameters>)
class TemplateClass(...):
...
return TemplateClass

It's simple, and less hackish, but still unusual. I fear I may be
missing something on my design, if I need to resort to such hacks.

Personally, I _like_ using 'class' statements to make classes and 'def'
statements to make functions; I don't see either as hacks. Messing with
bytecodes, building a function with new.function around a certain code
object, calling the metaclass explicitly... that feels more hackish to
me, personally. I do it when I must, but 99.9% of the time I have no
problem avoiding it in production code, it seems to me.


Alex
 
C

Carlos Ribeiro

Carlos Ribeiro said:
module and place a dis.dis(dct['my_code']) in the metaclass's __new__
and you'll see that basically MyClass.re_execute isn't very different
from a reload of the module. Isn't that a _bit_ too much in general?

(repeating to myself: dis is your friend. dis is your friend).

If you're hacking at this level, it sure is;-).
done a few more tests. I found a way to find the correct code object
for the _body_ of the class statement (or at least I think so), but

Good -- as I said I think it can be even more reliably found from inside
the classbody itself, but, no big deal.
exec klass.my_code

Ah, this is your problem -- that's not what you want to do now.
problem is that the code block that is found refers to the class
_body_ only. Checking dis.dis() helps to understand it:

Exactly: it's what builds the dict you want to pass to the metaclass
(once you wrap it into a new.function, as I mentioned).
There is not a single code block that does all the tasks related to
the class statement, and only it. In fact, there are two different
code objects to choose from, and neither really solves the problem.

Not by itself, but add some code and you'll be fine.

1. You need to build an f = new.function(klass.my_code, globals()) in
your classmethod that now erroneously does the exec above quoted (you
could in fact do this in the metaclass, save the new function rather
than the code object -- get the right globals via inspect).

That's the part that I was missing. It looks less 'hackish' than I had
expect. I was really afraid that I would need to manage bytecode-level
hacks myself. I am not as concerned about getting the right globals,
though -- it's good enough if I'm able to supply whatever globals I
need upon calling, I think (unless I'm _really_ missing something).
2. you need to obtain the needed dict, that's just d = f() in your
method

There is still a question, but I think I already know the anwswer :)
If the class *has* a base class, it has to be passed as a parameter. I
assume that i can simply call it like this: d=f(bases); or instead,
that I can fill the argdefs in the new.function call. I'll try it
later.
3. you can now call your metaclass with the appropriate bases and name:
klas = self.__class__ # or cls if this is a classmethod;-)
return type(klas)(klas.__name__, klas.__bases__, d)

voila, you're there. This is all coded to minimize the need to have
this method located in a specific class -- as I said I think you
probably want to have this stuff in the metaclass (if you have a
metaclass at all), but, whatever.

Tehre are two reasons to do it in a metaclass: first, I tested it and
it works (of sorts), while for some reason it wont work for a
non-metaclasses enabled class. Also, because in this scenario there
are other things that I already need to check, and a metaclass does it
nicely.
This is basically just mimicking what Python itself does (check the
dis.dis for a class statement as a starter): get the class body's code
object, build a function from it, call the function (it returns its
locals() as the result -- that 'return' is of course in the class body's
code object, and dis.dis shows that...), and that's how you get the
classdict on the stack. Add name and bases, call the metaclass, and
Bob's your uncle...

For now, I think I'll just be using the class-inside-a-def idiom. But
I'm still working around some design issues, and if the templating
mechanism gets in the final production code (it's only one of a few
competing designs), I'll surely try it again.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
C

Carlos Ribeiro

Personally, I _like_ using 'class' statements to make classes and 'def'
statements to make functions; I don't see either as hacks. Messing with
bytecodes, building a function with new.function around a certain code
object, calling the metaclass explicitly... that feels more hackish to
me, personally. I do it when I must, but 99.9% of the time I have no
problem avoiding it in production code, it seems to me.

I'm not sure if I understood what you meant. My common sense tells me
that the class-inside-a-def is more acceptable, given all the black
magic that we had to go through to be able to re-execute the class
statement. But the function above really returns a class... which is
'wrong', but many counts (although nothing stops it from happening,
after all, classes are objects pretty much like any other object,
albeit with the special ability to build new objects instances).

On the other hand, the re-execute-class-statement trick is a real hack
-- specially when seen from the inside. However, if everything is
clearly hidden in a metaclass (and I mean, a *real* metaclass, not a
def __metaclass__()); better yet, if the metaclass itself is hidden in
a base class; what do you think is cleaner?

a) class-inside-a-def

def BuildTemplate(<template parameters>)
class TemplateClass(...):
...
return TemplateClass
T = BuildTemplate(<params>)

b) re-exec-class

# GenericTemplate is a metaclass-enabled ancestor
from template import GenericTemplate
class TemplateClass(GenericTemplate):
...

T = TemplateClass.NewTemplate(<params>)

c) neither of the above -- there must be a better way to do it.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
A

Alex Martelli

Carlos Ribeiro said:
...
I'm not sure if I understood what you meant. My common sense tells me
that the class-inside-a-def is more acceptable, given all the black
magic that we had to go through to be able to re-execute the class

Yep, sounds about right.
statement. But the function above really returns a class... which is
'wrong', but many counts (although nothing stops it from happening,

Ah, that's our disagreement, I guess. I consider Factory a perfectly
sensible and respectable design pattern, and thus I see nothing wrong a
function making and returning a new object (class or otherwise).
after all, classes are objects pretty much like any other object,
albeit with the special ability to build new objects instances).

Any callable can build whatever kind of objects it wants -- class
objects have no "special ability" in this regard. Where class objects
are unique is in the automatic chain of delegation -- instance to class
to bases -- when an attribute is being looked up.

On the other hand, the re-execute-class-statement trick is a real hack
-- specially when seen from the inside. However, if everything is
clearly hidden in a metaclass (and I mean, a *real* metaclass, not a
def __metaclass__()); better yet, if the metaclass itself is hidden in
a base class; what do you think is cleaner?

I think that, when feasible, making classes by executing a class
statement (and functions by executing a def statement) is cleanest.
If you can get substantial added value using type(....) calls or
new.function(...) calls or whatever, so be it, of course -- if you NEED
to get to that level, so be it, there's nothing sinful or bad about it,
it may just be a tad harder to understand and maintain... no big deal,
not comparable to (e.g.) building up a string and then doing 'exec' on
it (THAT _is_ sinful and bad and maybe will make Guido cry...;-).

a) class-inside-a-def

def BuildTemplate(<template parameters>)
class TemplateClass(...):
...
return TemplateClass
T = BuildTemplate(<params>)

Quite clean, IMHO, if it does all you need.

b) re-exec-class

# GenericTemplate is a metaclass-enabled ancestor
from template import GenericTemplate
class TemplateClass(GenericTemplate):
...

T = TemplateClass.NewTemplate(<params>)

To be honest I never really understood why you liked that one so much,
apart from the fascinating possibilities it offers for delving into
normally ignored details of 'class' statements' execution;-)

Seriously: classes have one incredible syntax advantage when you expose
them to users... users who write class statements get to write any gd
code they like in a context where every localname created ends up in a
dict, then your metaclass gets to process the dict, etc, etc. In
comparison, functions are fine, with keyword parameters, for the
equivalent of assignment statements, i.e.,

class foo(subletWayToGetMetaclassOmega):
bar = 23

and

foo = factoryFunctionOmega(bar=23)

are roughly equivalent; but as soon as the body of 'class foo' starts
having def statements, class statements, loops, etc, the factory
function has a hard time keeping up. This syntax advantage has little
to do with the *intended* semantics of 'class' vs function calls, and is
the root of most "metaclass abuse" I've seen (and perpetrated -- though
not in production code...).

But it doesn't seem to be what you're after, and a couple days ago I saw
somebody else asking about a metaclass to be called directly (where a
class statement _was_ clearly what they needed, IMHO), so there must be
something more going on here... I think I'll nickname it "class
struggle"... many people DO seem to be struggling with classes, these
days...

c) neither of the above -- there must be a better way to do it.

....must there...? Maybe I'm being short-sighted (wouldn't be the first
time), but I can't see. If you could have your heart's desire, within
the general framework of Python, how WOULD you code what you're after
(which I'm still not entirely clear about)? I mean, how would it look
to the user -- how it looks like _inside_ is a relatively minor issue...


Alex
 
A

Alex Martelli

Carlos Ribeiro said:
That's the part that I was missing. It looks less 'hackish' than I had
expect. I was really afraid that I would need to manage bytecode-level
hacks myself. I am not as concerned about getting the right globals,
though -- it's good enough if I'm able to supply whatever globals I
need upon calling, I think (unless I'm _really_ missing something).

I don't think you're missing anything -- just DO consider that if you
pass your own globals they'll be used to solve every name that's not
local to the class body, e.g. a '__metaclass__ = metaname' assignment in
classbody (of course, you can finesse this by calling the metaclass
explicitly anyway;-).

There is still a question, but I think I already know the anwswer :)
If the class *has* a base class, it has to be passed as a parameter. I
assume that i can simply call it like this: d=f(bases); or instead,
that I can fill the argdefs in the new.function call. I'll try it
later.

Nope, f doesn't get the bases AT ALL; the bases (as a tuple) go in only
when you call the metaclass.

Tehre are two reasons to do it in a metaclass: first, I tested it and
it works (of sorts), while for some reason it wont work for a
non-metaclasses enabled class. Also, because in this scenario there
are other things that I already need to check, and a metaclass does it
nicely.

OK, then, that's where you pass the bases in, not before.
For now, I think I'll just be using the class-inside-a-def idiom. But

If it can give you all you need, that may be best.
I'm still working around some design issues, and if the templating
mechanism gets in the final production code (it's only one of a few
competing designs), I'll surely try it again.

OK, keep us informed of developments, then!


Alex
 
C

Carlos Ribeiro

I don't think you're missing anything -- just DO consider that if you
pass your own globals they'll be used to solve every name that's not
local to the class body, e.g. a '__metaclass__ = metaname' assignment in
classbody (of course, you can finesse this by calling the metaclass
explicitly anyway;-).

I was missing the new.function() call -- I hadn't checked how could I
build a function out of a code block up to that point, and I was
afraid that I would need to dive deeply into Python internals. It's
not that bad.
Nope, f doesn't get the bases AT ALL; the bases (as a tuple) go in only
when you call the metaclass.

Well. I was mistaken (and that's not be the first time). It goes like
this; I first checked the dis.dis() code:

27 64 LOAD_CONST 4 ('MyClass')
67 LOAD_NAME 7 (object)
70 BUILD_TUPLE 1
73 LOAD_CONST 5 (<code object MyClass at
0120A0A0, file "c:\work\help-on-c-l-py\re_exec_class.py", line 27>)
76 MAKE_FUNCTION 0
79 CALL_FUNCTION 0
82 BUILD_CLASS
83 STORE_NAME 8 (MyClass)

.... with that build_tuple there... besides that I knew I *can* access
the bases from within my class statement body:

class MyClass(MyBaseClass):
bar = MyBaseClass.foo

.... so I had suspected that the bases *were* passed to f(). But wait!
Actually, Python is building the stack for the BUILD_CLASS opcode.
Right?
OK, then, that's where you pass the bases in, not before.


If it can give you all you need, that may be best.


OK, keep us informed of developments, then!




Alex


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
C

Carlos Ribeiro

... no big deal,
not comparable to (e.g.) building up a string and then doing 'exec' on
it (THAT _is_ sinful and bad and maybe will make Guido cry...;-).

I swear I'll not go down to this level ;-)
To be honest I never really understood why you liked that one so much,
apart from the fascinating possibilities it offers for delving into
normally ignored details of 'class' statements' execution;-)

That was part of the fun ;-)
Seriously: classes have one incredible syntax advantage when you expose
them to users... users who write class statements get to write any gd
code they like in a context where every localname created ends up in a
dict, then your metaclass gets to process the dict, etc, etc. In
comparison, functions are fine, with keyword parameters, for the
equivalent of assignment statements, i.e.,

class foo(subletWayToGetMetaclassOmega):
bar = 23

and

foo = factoryFunctionOmega(bar=23)

are roughly equivalent; but as soon as the body of 'class foo' starts
having def statements, class statements, loops, etc, the factory
function has a hard time keeping up. This syntax advantage has little
to do with the *intended* semantics of 'class' vs function calls, and is
the root of most "metaclass abuse" I've seen (and perpetrated -- though
not in production code...).

Well, I'm looking into it for a couple of reasons. One of them is
exactly this -- code inside a class is much easier to write and to
structure than similar code inside a def... specialy when it comes to
data declarations[1]. A def is pretty much limited to one level; with
classes, it's easy to 'nest' groups of data declarations, which make
it easier to build complex hierarchical stuff. The other reason is to
be able to build something related to C++ templates -- a factory that
produces customized versions of a class based on parameters, that may
include other classes. I *thought* that a purely class-based
templating would make it easier to understand and manage, specially if
the magic is done "behind the scenes". But I'm not so sure now.

[1] I think that this is the correct term, in this context... but I
may be wrong (again).
But it doesn't seem to be what you're after, and a couple days ago I saw
somebody else asking about a metaclass to be called directly (where a
class statement _was_ clearly what they needed, IMHO), so there must be
something more going on here... I think I'll nickname it "class
struggle"... many people DO seem to be struggling with classes, these
days...



...must there...? Maybe I'm being short-sighted (wouldn't be the first
time), but I can't see. If you could have your heart's desire, within
the general framework of Python, how WOULD you code what you're after
(which I'm still not entirely clear about)? I mean, how would it look
to the user -- how it looks like _inside_ is a relatively minor issue...

Well, I don't think that (c) applies. But I was afraid that other
people, coming with a fresh mental picture of the problem, could point
to other types of solution. That's what (c) really means.


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)
 
A

Alex Martelli

Carlos Ribeiro said:
Well. I was mistaken (and that's not be the first time). It goes like
this; I first checked the dis.dis() code:

27 64 LOAD_CONST 4 ('MyClass')
67 LOAD_NAME 7 (object)
70 BUILD_TUPLE 1
73 LOAD_CONST 5 (<code object MyClass at
0120A0A0, file "c:\work\help-on-c-l-py\re_exec_class.py", line 27>)
76 MAKE_FUNCTION 0
79 CALL_FUNCTION 0
82 BUILD_CLASS
83 STORE_NAME 8 (MyClass)

... with that build_tuple there... besides that I knew I *can* access
the bases from within my class statement body:

class MyClass(MyBaseClass):
bar = MyBaseClass.foo

... so I had suspected that the bases *were* passed to f(). But wait!
Actually, Python is building the stack for the BUILD_CLASS opcode.
Right?

Right. When you mention MyBaseClass in the body, it gets looked up in
the globals, just where it was looked up when you passed it as a base.
The 0 in MAKE_FUNCTION _and_ CALL_FUNCTION opcodes refers to the number
of arguments to the function, btw...


Alex
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top