class factory question

[Tim]

I am extending a parser and need to create many classes that are all subclassed from the same object (defined in an external library). When my moduleis loaded I need all the classes to be created with a particular name but the behavior is all the same. Currently I have a bunch of lines like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like

newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?
thanks,
--Tim

 

[Peter Otten]

I am extending a parser and need to create many classes that are all
subclassed from the same object (defined in an external library). When my
module is loaded I need all the classes to be created with a particular
name but the behavior is all the same. Currently I have a bunch of lines
like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like

newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?

What is your objection against that approach?
By the way, I don't think you need

 

[Tim]

I am extending a parser and need to create many classes that are all
subclassed from the same object (defined in an external library). When my
module is loaded I need all the classes to be created with a particular
name but the behavior is all the same. Currently I have a bunch of lines
like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like

newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?

What is your objection against that approach?
By the way, I don't think you need

tmp.__name__ = name

I am not completely understanding the type function I guess. Here is an example from the interpreter:

In [1]: class MyClass(object):
...: pass
...:
In [2]: type('Vspace', (MyClass,), {})
Out[2]: __main__.Vspace
In [3]: x = Vspace()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
C:\Python27\Scripts\<ipython-input-3-a82f21420bf3> in <module>()
----> 1 x = Vspace()

NameError: name 'Vspace' is not defined

In [4]: Vspace = type('Vspace', (MyClass,), {})
In [5]: x = Vspace()
In [6]: type(x)
Out[6]: __main__.Vspace

I don't understand how to make `Vspace` usable for creating instances later (line 3) when I just call `type`; that is why I thought adding the `__name__` attribute would work. Hmm, now I know that doesn't work either:

In [8]: del Vspace
In [9]: m = type('Vspace', (MyClass,), {})
In [10]: m.__name__ = 'Vspace'
In [11]: x = Vspace()
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
C:\Python27\Scripts\<ipython-input-11-a82f21420bf3> in <module>()
----> 1 x = Vspace()

NameError: name 'Vspace' is not defined
In [11]: m
Out[12]: __main__.Vspace

Maybe this is too much trouble just to save a few lines (~50) of code.

thanks,
--Tim

 

[Peter Otten]

I am not completely understanding the type function I guess. Here is an
example from the interpreter:

In [1]: class MyClass(object):
...: pass
...:
In [2]: type('Vspace', (MyClass,), {})
Out[2]: __main__.Vspace
In [3]: x = Vspace()
---------------------------------------------------------------------------
NameError Traceback (most recent call
last) C:\Python27\Scripts\<ipython-input-3-a82f21420bf3> in <module>()
----> 1 x = Vspace()

NameError: name 'Vspace' is not defined

No, you are not understanding how Python namespaces work
To get a Vspace in the global namespace you'd have to bind that name

Vspace = type(...)

which defeats your plan of mass creation of such names. The clean way to
cope with the situation is to use a dict:

classnames = ["Vspace", ...]
classes = {name: type(name, ...) for name in classnames}

Then you can access the Vspace class with

classes["Vspace"]

If that is inconvenient for your usecase you can alternatively update the
global (module) namespace:

globals().update((name, type(name, ...) for name in classnames)

For example:

class A(object): pass ....
globals().update((n, type(n, (A,), {})) for n in ["Beta", "Gamma"])
Beta

issubclass(Beta, A)

True

 

[Tim]

I am not completely understanding the type function I guess. Here is an
example from the interpreter:

No, you are not understanding how Python namespaces work
To get a Vspace in the global namespace you'd have to bind that name
Vspace = type(...)

which defeats your plan of mass creation of such names. The clean way to
cope with the situation is to use a dict:

classnames = ["Vspace", ...]
classes = {name: type(name, ...) for name in classnames}

Then you can access the Vspace class with
classes["Vspace"]

If that is inconvenient for your usecase you can alternatively update the
global (module) namespace:
globals().update((name, type(name, ...) for name in classnames)

For example:

class A(object): pass ...
globals().update((n, type(n, (A,), {})) for n in ["Beta", "Gamma"])
Beta

issubclass(Beta, A)

True

Thank you for that great explanation. That is exactly what I needed to know!
What a fantastic group this is.
--Tim

 

[Joshua Landau]

The clean way to
cope with the situation is to use a dict:

classnames = ["Vspace", ...]
classes = {name: type(name, ...) for name in classnames}

Then you can access the Vspace class with

classes["Vspace"]

If that is inconvenient for your usecase you can alternatively update the
global (module) namespace:

globals().update((name, type(name, ...) for name in classnames)

For example:

class A(object): pass ...
globals().update((n, type(n, (A,), {})) for n in ["Beta", "Gamma"])
Beta

issubclass(Beta, A)

True

I would say if a dict isn't good, there are still some cases where you
might not want to use globals.

I _might_ do:

import sys
from types import ModuleType

# As before
newclasses = ['Vspace', 'Boldpath', "and so on", "and yes, these all
become variables"]
little_classes = {name: type(name, (int,), {}) for name in newclasses}

# Make a module
module_for_little_classes = ModuleType("module_for_little_classes",
"All the things")
module_for_little_classes.__dict__.update(little_classes)

# Hack it in there!
sys.modules["module_for_little_classes"] = module_for_little_classes

# Now we can undo all
import module_for_little_classes as mlc
mlc.Vspace
mlc.Boldpath
....

# And undo all our hard work avoiding globals():
from module_for_little_classes import *
Vspace
Boldpath


So, why avoid globals()?
1) Linters don't like globals()
2) Urm... it's ugly?
3) Ur.......

 

[Peter Otten]

I would say if a dict isn't good, there are still some cases where you
might not want to use globals.

I _might_ do:

# Make a module
module_for_little_classes = ModuleType("module_for_little_classes",
"All the things")
module_for_little_classes.__dict__.update(little_classes)

Hm, from within module_for_little_classes that is globals(). To illustrate:
True

Also, I'd spell module.__dict__ vars(module).

That said I agree that it's a good idea to use a dedicated module (not
necessarily created on the fly) for those dynamically generated classes.

 

[Joshua Landau]

Hm, from within module_for_little_classes that is globals(). To illustrate:

True

Yes, that's true - but the point wasn't not to use "globals the
function", but not to use *this* global scope.

Also, I'd spell module.__dict__ vars(module).

Again, good catch. Definitely that.

 

[Fábio Santos]

I am extending a parser and need to create many classes that are all

subclassed from the same object (defined in an external library). When my
module is loaded I need all the classes to be created with a particular
name but the behavior is all the same. Currently I have a bunch of lines
like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like

newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?
thanks,
--Tim

I would say The Most Pythonic Way is to use the class declarations as you
are doing now. Explicit is better than implicit, or so the zen says.

It will be better for tools as well. I'd like to see code completion work
on dynamically created classes like that (unless you use __all__ to list
them.).

And, are you really looking for classes here? If you just want to create
different names with different identities, you could consider using plain
old strings or object() to do that.

 

[Joshua Landau]

I am extending a parser and need to create many classes that are all subclassed from the same object (defined in an external library). When my module is loaded I need all the classes to be created with a particular name but the behavior is all the same. Currently I have a bunch of lines like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like

newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?

I've just realised -- why on earth are you doing this? Surely there's
a better way than having 50 identical classes. :/

 

[Tim]

I am extending a parser and need to create many classes that are all subclassed from the same object (defined in an external library). When my module is loaded I need all the classes to be created with a particular name but the behavior is all the same. Currently I have a bunch of lines like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like
newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?

I've just realised -- why on earth are you doing this? Surely there's
a better way than having 50 identical classes. :/

The reason is that I'm using a parser that creates a DOM from a LaTeX file.The plasTeX package (http://plastex.sourceforge.net/) provides the libraryto accomplish that. The parser needs to know how to consume some custom-defined tokens (like \Vspace, \Boldpath, etc). The behavior for these is all the same so they're subclassed from one base class, but they need to have these particular names so the parser knows how to consume them when encountered in the source file. That is, for every custom command the parser encounters, it looks for a class of that name in order to tokenize it.

I agree that this is an abnormal circumstance and normally I would just subclass in the normal way. But I think this situation is different enough that it is okay to break a convention.

thanks for thinking about this.
--Tim

 

[Dave Angel]

I am extending a parser and need to create many classes that are all subclassed from the same object (defined in an external library). When my module is loaded I need all the classes to be created with a particular name but the behavior is all the same. Currently I have a bunch of lines like this:

class Vspace(Base.Command): pass
class Boldpath(Base.Command): pass

There are a bunch of lines like that.
Is there a better way? Something like
newclasses = ['Vspace', 'Boldpath', ... ]
for name in newclasses:
tmp = type(name, (Base.Command,) {})
tmp.__name__ = name

Is there a more pythonic way?

I've just realised -- why on earth are you doing this? Surely there's
a better way than having 50 identical classes. :/

The reason is that I'm using a parser that creates a DOM from a LaTeX file. The plasTeX package (http://plastex.sourceforge.net/) provides the library to accomplish that. The parser needs to know how to consume some custom-defined tokens (like \Vspace, \Boldpath, etc). The behavior for these is all the same so they're subclassed from one base class, but they need to have these particular names so the parser knows how to consume them when encountered in the source file. That is, for every custom command the parser encounters, it looks for a class of that name in order to tokenize it.

I agree that this is an abnormal circumstance and normally I would just subclass in the normal way. But I think this situation is different enough that it is okay to break a convention.

thanks for thinking about this.
--Tim

I don't think you broke the convention, that parser did.

 

[Irmen de Jong]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

How does it look for a class?
Can you perhaps override the way it looks for a class of that name?
So that you can instead return something sensible rather than having to define one of 50
almost identical classes...


Irmen

 

[Tim]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

How does it look for a class?
Can you perhaps override the way it looks for a class of that name?
So that you can instead return something sensible rather than having to define one of 50
almost identical classes...
Irmen

hmm, the package author describes inheriting like I was doing in the first place. With a parser, I really don't know any better way but then I'm not aparsing expert. I will delve into the code and try to get a better handle on how the parser finds the definitions it needs in order to tokenize the input.

thanks!
--Tim