dynamic class instantiation

[Ognen Duzlevski]

Hi, I have a "language definition" file, something along the lines of:

page ::
name : simple
caption : simple
function : complex

function ::
name : simple
code : simple

component ::
name : simple
type : simple
dataset : complex

etc.

On the other hand as input I have .xml files of the type:

<page>
<name>WebPage</name>
<caption>Browser Visible Caption</caption>
<component>
<name>Countrylist</name>
<type>dropdown</type>
<value>$dropdownlist</value>
<function>
<name>sqlSelect</name>
<code>select countries
from blah into $dropdownlist</code>
</function>
</component>
</page>

I have a parser that will go through the language definition
file and produce the following as a separate .py file:

class page(object):
def __init__():
self.name = None
self.caption = None
self.functions = []

class function(object):
def __init__():
self.name = None
self.code = None

Now I want to use something like xml.dom.minidom to "parse" the
..xml file into a set of classes defined according to the "language
definition" file. The parse() method from the xml.dom.minidom
package will return a document instance and I can get the node
name from it. Say I got "page" as a string. How do I go about
instantiating a class from this piece of information? To make it
more obvious how do I create the page() class based on the "page"
string I have? I want to make this generic so for me the language
definition file will contain pages, functions, datasets etc. but
for someone else mileage will vary.

Thanks,
Ognen

 

[Larry Bates]

Hi, I have a "language definition" file, something along the lines of:

page ::
name : simple
caption : simple
function : complex

function ::
name : simple
code : simple

component ::
name : simple
type : simple
dataset : complex

etc.

On the other hand as input I have .xml files of the type:

<page>
<name>WebPage</name>
<caption>Browser Visible Caption</caption>
<component>
<name>Countrylist</name>
<type>dropdown</type>
<value>$dropdownlist</value>
<function>
<name>sqlSelect</name>
<code>select countries
from blah into $dropdownlist</code>
</function>
</component>
</page>

I have a parser that will go through the language definition
file and produce the following as a separate .py file:

class page(object):
def __init__():
self.name = None
self.caption = None
self.functions = []

class function(object):
def __init__():
self.name = None
self.code = None

Now I want to use something like xml.dom.minidom to "parse" the
.xml file into a set of classes defined according to the "language
definition" file. The parse() method from the xml.dom.minidom
package will return a document instance and I can get the node
name from it. Say I got "page" as a string. How do I go about
instantiating a class from this piece of information? To make it
more obvious how do I create the page() class based on the "page"
string I have? I want to make this generic so for me the language
definition file will contain pages, functions, datasets etc. but
for someone else mileage will vary.

Thanks,
Ognen

I think you should rethink this approach. Why have 3 different
files instead of just defining everything as Python classes?
Get good baseclass definitions and use OOP inheritance to
create your specific class instances. I think mixing XML,
language definition file (LDF), LDF to python parser is a LOT
harder (and slower) than just writing the classes outright.

-Larry Bates

 

[Ognen Duzlevski]

I think you should rethink this approach. Why have 3 different
files instead of just defining everything as Python classes?
Get good baseclass definitions and use OOP inheritance to
create your specific class instances. I think mixing XML,
language definition file (LDF), LDF to python parser is a LOT
harder (and slower) than just writing the classes outright.

Hi Larry,

here is the whole story. I have a database (postgres) xml "transformer"
and substitution engine. It is written entirely in plpgsql and the whole
environment works something like this. Someone writes an xml file that
defines a web page. This xml file is then fed into the database at the
time of development. Now at runtime a mod_python apache webapp makes a
request for a certain page. Its xml form is retrieved from the database
and a lot of data substitution goes on. Also parts of the page are "masked"
out by the database engine based on the "level" of the person requesting
the page. The "transformed and substituted" xml is then spit back to the
webapp which now applies an xml->html transformation and presents the
final result to the user. So, I want people to be able to "define" their
own "language" when it comes to page definitions. And instead of rewriting
the xml file parser I want to be able to generate the parser.

Thanks,
Ognen

 

[Larry Bates]

Hi Larry,

here is the whole story. I have a database (postgres) xml "transformer"
and substitution engine. It is written entirely in plpgsql and the whole
environment works something like this. Someone writes an xml file that
defines a web page. This xml file is then fed into the database at the
time of development. Now at runtime a mod_python apache webapp makes a
request for a certain page. Its xml form is retrieved from the database
and a lot of data substitution goes on. Also parts of the page are "masked"
out by the database engine based on the "level" of the person requesting
the page. The "transformed and substituted" xml is then spit back to the
webapp which now applies an xml->html transformation and presents the
final result to the user. So, I want people to be able to "define" their
own "language" when it comes to page definitions. And instead of rewriting
the xml file parser I want to be able to generate the parser.

Thanks,
Ognen

Well you certainly can do that, but why invent a new language to define
the pages instead of just using Python directly? I did an application
a few years ago that sounds a little like what you describe and now
wish I had not spent so much time doing Text File->Python transforma-
tions. The logic got pretty ugly and extending things was rather
involved. It wasn't until I started using ReportLab that I began to
"see the light" on just keeping everything in Python and inheriting
from base classes and extending.

If you want to define a Python class, the best way to do that is to
use Python from the outset. I probably don't completely understand
what you want to do, but thought I'd at least put in my 2-cents worth.

Regards,
Larry

 

[Kent Johnson]

I have a parser that will go through the language definition
file and produce the following as a separate .py file:

class page(object):
def __init__():
self.name = None
self.caption = None
self.functions = []Say I got "page" as a string. How do I go about
instantiating a class from this piece of information? To make it
more obvious how do I create the page() class based on the "page"
string I have?

Use getattr().

If the definition of page is in my_objects.py, and that file is on the
Python search path, you can use
import my_module
cls = getattr(my_module, 'path')
instance = cls()

to create an instance of my_module.path.

Kent

 

[Ognen Duzlevski]

Use getattr().

Hi Kent, this is exactly what I was looking for. I can't believe I didn't think
of getattr() myself! ;(

Cheers,
Ognen

 

[Volker Grabsch]

Hi Kent, this is exactly what I was looking for. I can't believe I didn't think
of getattr() myself! ;(

However, remember that this solves your problem just temporarily.

Your main problem is still a deep design failure. Code generators are
relicts, hard to maintain, and usually just plain ugly. Your application
isn't that special.

I'm sure you could replace 2/3 of your code with something much simpler
(and shorter!) just by not inventing a new language and using the power
of Python instead.

People often think that a new language simplifies their problem, and that
code generation saves work. However, a the code of the code generator has
to be maintained, too! That's what most people are realizing too late.
Also, simple class inheritance, closures and similar mechanisms save a
lot of work, too, they do their job even better than a code generator,
and they are a charm to maintain.

Even the "magic" (AKA meta-classes ) may be hard, but it's usually a
much simpler and cleaner approach than a code generator.

Don't generate Python code. Python is neither Java nor C, it's dynamic!


Greets,

Volker

 

[Ognen Duzlevski]

However, remember that this solves your problem just temporarily.

Your main problem is still a deep design failure. Code generators are
relicts, hard to maintain, and usually just plain ugly. Your application
isn't that special.

I'm sure you could replace 2/3 of your code with something much simpler
(and shorter!) just by not inventing a new language and using the power
of Python instead.

Hi Volker,

I appreciate your comments. Basically, the reason why this code generator
exists is the fact that I do not want to hard-code the resulting xml in
any way. The users of the web/db framework of which this solution is part of
might like the "web page" definition I offer them, but, some might want to extend it.
While dom allows me to traverse an xml hierarchy - it does not allow me to
give "meaning" to the tags. The only way to do that (for me) is to represent
the tags as classes with containers containing other classes (tags) and so on. Since
I do not know ahead of time what the tags will be - I need to generate the code
to actually handle them.

Can you suggest a better approach or did you already do that and I just missed
it?

Cheers,
Ognen

 

[Kent Johnson]

Hi Volker,

I appreciate your comments. Basically, the reason why this code generator
exists is the fact that I do not want to hard-code the resulting xml in
any way. The users of the web/db framework of which this solution is part of
might like the "web page" definition I offer them, but, some might want to extend it.
While dom allows me to traverse an xml hierarchy - it does not allow me to
give "meaning" to the tags. The only way to do that (for me) is to represent
the tags as classes with containers containing other classes (tags) and so on. Since
I do not know ahead of time what the tags will be - I need to generate the code
to actually handle them.

Can you suggest a better approach or did you already do that and I just missed
it?

Instead of generating Python code and importing it, you could create the
classes directly. You would still have your own language but no code
generator.

For a very simple example, you could have a base class that initializes
attributes from a class dictionary:
class DynamicInits(object):
def __init__(self):
self.__dict__.update(self.inits)

and a simple factory function that creates a subclass with an
appropriate dict of initializers:
def classFactory(name, initializers):
return type(name, (DynamicInits,), {'inits' : initializers})

In your original post you show this class:
class page(object):
def __init__():
self.name = None
self.caption = None
self.functions = []

With the above definitions, an equivalent class is created by calling
page = classFactory( 'page', { 'name' : None, 'caption': None,
'functions' : []} )

Kent

 

[Ognen Duzlevski]

With the above definitions, an equivalent class is created by calling
page = classFactory( 'page', { 'name' : None, 'caption': None,
'functions' : []} )

Beautiful. Thank you very much.

Cheers,
Ognen

 

[Volker Grabsch]

Instead of generating Python code and importing it, you could create the
classes directly. You would still have your own language but no code
generator.

While I agree that this is much better than code generation, why not go
a step further?

Your "description language" could be as well a subset of the Python language.
So you could get rid of the parser, too. Add the "meaning" simply by creating
subclasses in Python.

This approach has the disadvantage of a slightly more complicated syntax of
the definition files. However, the advantage is that this syntax would be
valid Python code, so everyone who's learning it will also learn a small piece
of Python, and thus reusable knowlege. Learning a special newly invented
language (so-called "DSL" - domain specific language) is *not* resuable
knowledge.

The second advantage is a greater flexibility. You can use real Python code
to model some more complex "definition files". Otherwise, you'll have to
enhance your definition language for every new feature or meta-feature
you introduce. In this case, your own language would become more and more
messy over time. Why don't you take a great, well-designed, simple, general
purpose language which already exist? ... such as Python ;-)

However, this is a completely different strategy: You wouldn't parse the
defintion files. Instead, your definition files are Python modules which
your application imports. This can be a security risk if you want these
modules to be edited "online". However, if your definition files are only
put into a directory of the server, i.e. if you handle them just as your
Python sources anyway, this won't be a disadvantage. (Instead, if would be
a great advantage!)

So you have to decide yourself: Do you really need the description files
to be read in on-the-fly? Then stick with your own language. Are your
definition files more like modules/extensions which are installes together
with the source files? Then just *make* them source files, too, i.e.
realize your "description files" simply as Python modules.


Greets,

Volker