How to instatiate a class of which the name is only known at runtime?

M

Marco Herrn

Hi,

I am writing a program that has to instantiate a class from which I
don't know the name until runtime. That leads to two problems for me.

1. How to do the import? I didn't find a way to give a string to the
import statement.

2. How to write such code to instantiate?

The first is a problem, when I don't know where the classes reside.
Currently it is planned that all these classes reside in a specific
subdirectory. Is it possible to say 'import * from subdirectory' ?
What other problems could occur here and how could I avoid them?

And to the second question, I have no idea about how to do this. Any
hints?


Bye
Marco
 
J

John Roth

Marco Herrn said:
Hi,

I am writing a program that has to instantiate a class from which I
don't know the name until runtime. That leads to two problems for me.

1. How to do the import? I didn't find a way to give a string to the
import statement.

As Nick said, you can use the exec statement. You can
also use the _import() function.
2. How to write such code to instantiate?

The first is a problem, when I don't know where the classes reside.

I'm not sure what you mean by this. Modules exist on disk in
a directory. There may be more than one class in a module,
and there is no way of automatically asking for a class and
having the module containing that class loaded.
Currently it is planned that all these classes reside in a specific
subdirectory. Is it possible to say 'import * from subdirectory' ?

No. It is possible to load the names of all the modules in the
subdirectory, though. That doesn't tell you what classes are in
each module unless you either establish some standards or use
a configuration file of some kind.

You could, of course, simply import all of the modules, and then
loop through them looking for the classes.
What other problems could occur here and how could I avoid them?

The biggest problem I can see is name collisions, and that's only
a problem if you don't establish standards, and use the simplest
method, which is to use the identifier that the import statement
adds to the module.
And to the second question, I have no idea about how to do this. Any
hints?

As Nick points out, the easiest way is simply to build an import
statement, and then exec it.

John Roth
 
E

Erik Max Francis

John said:
As Nick said, you can use the exec statement. You can
also use the _import() function.
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name '_import' is not defined

Do you mean __import__?
 
J

John Roth

Erik Max Francis said:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name '_import' is not defined

Do you mean __import__?

Yes.

John Roth
 
M

Max M

Marco said:
I am writing a program that has to instantiate a class from which I
don't know the name until runtime. That leads to two problems for me.


Is there any reason not to use a standard factory pattern for this? It
is the normal approach.


# Factory.py

import class1, class2, class3

def Factory(class_type):
if class_type='class1':
return class1()
elif class_type='class2':
return class2()
elif class_type='class2':
return class3()



# using the factory
from Factory import Factory
myObject = Factory(class2)


regards Max M
 
M

Max M

Max said:
import class1, class2, class3

def Factory(class_type):
if class_type='class1':
return class1()
elif class_type='class2':
return class2()
elif class_type='class2':
return class3()



# using the factory
from Factory import Factory
myObject = Factory(class2)

That last line should read:

myObject = Factory('class2')

Max M
 
P

Peter Otten

Marco said:
1. How to do the import? I didn't find a way to give a string to the
import statement.

2. How to write such code to instantiate?

Off topic: This thread like some others is spread over a few toplevel
entries in my newsreader (KNode 0.7.2). Is this a bug?

Anyway, putting it all together:

import sys

def getClass(classname, modulename):
try:
module = sys.modules[modulename]
except KeyError:
module = __import__(modulename)
return getattr(module, classname)

print getClass("WichmannHill", "random")
print getClass("WichmannHill", "random")()


Peter
 
E

Erik Max Francis

Max said:
Is there any reason not to use a standard factory pattern for this? It
is the normal approach.

Because introspection means you don't have to keep a separate list of
all possible values updated.
 
K

kosh

use

def Factory(class_type):
factory = {
'class1':class1,
'class2':class2,
'class3':class3}
try:
return factory[class_type]()
except KeyError:
pass

That will expand nicely and only needs a single lookup. Change the except
condition to match what you what you want to happen when the class is not
found. I have one factory that can make one of about 50 objects and doing an
if elif structure would suck for that. Also if you want create that data
structure outside that method so that you only create it once although that
is not really an issue since it works very fast.
 
J

John Roth

Peter Otten said:
Off topic: This thread like some others is spread over a few toplevel
entries in my newsreader (KNode 0.7.2). Is this a bug?

It's not a bug, it's a misfeature. Articles that are gated from
the mailing list to the newsgroup have improperly truncated
reference lists, so newsreaders can't put them together properly.

If you look at the headers for a message posted via news, you'll
find something like:

References: <[email protected]>
<[email protected]> <[email protected]>

However, if you look at the headers for a message gated
from the Python mailing list, you'll find:

References: <[email protected]>

Which happens to be the ID of the first message in the tree, (which was,
coincidentally, gated from the mailing list.) regardless of how far down
in the tree it happens to be.

Then if, for some reason, you don't have the message at the top
of the tree, you get this scatter effect.

I have no idea whether this is a problem with Mailman, the
Mail RFCs, or the mail to news gateway.

HTH

John Roth
 
J

Jack Diederich

use

def Factory(class_type):
factory = {
'class1':class1,
'class2':class2,
'class3':class3}
try:
return factory[class_type]()
except KeyError:
pass

metaclasses are your friend, here is one I use frequently for factoires.

class Register(type):
"""A tiny metaclass to help classes register themselves automagically"""

def __init__(cls, name, bases, dict):
if ('register' in dict): # initial dummy class
setattr(cls, 'register', staticmethod(dict['register']))
elif (getattr(cls, 'DO_NOT_REGISTER', 0)):
# we don't want to register this non-concrete class
delattr(cls, 'DO_NOT_REGISTER')
elif (object not in bases):
cls.register(name, cls)
return

class Widget(object):
__metaclass__ = Register
all = {}
def register(name, cls):
Widget.all[name] = cls

def factory(name):
return Widget.all[name]
factory = staticmethod(factory) # just use this class like a namespace

class C1(Widget): pass
class C2(Widget): pass
class C3(Widget): pass

# instantiate a C3 by name
c3_instance = Widget.factory('C3')()

Every class that inherits Widget and doesn't have a DO_NOT_REGISTER attribute
will put it's name:class into the Widget.all dictionary.

I typically use a short hierarchy to break up the code a bit

class WidgetFacory(object):
"""this class only has static factory methods"""
__metaclass__ = Register
all = {}
# register() and factory() as above

class Widget(WidgetFactory): # we inherit 'WidgetFactory' to get the metaclass
"""This class is the base functionality of Widget classes,
This is a virtual base class for Widgets that can't be instatiated
by the WidgetFactory interface"""
DO_NOT_REGISTER = 1 # tell the factory to forget we exist

def __init__(self, *args, **opts):
# your code here

# now define the classes that can be instantiated from the WidgetFactory
class Mallet(Widget): pass
class Hoop(Widget): pass
class Ball(Widget): pass


hmm, the post is about to get longer.
If you repeat the above pattern enough, write a func to do all the above
for you, so your code might look like

class Widget(object): pass # your real definition of the base Widget class

# setup the factory, and rebind the name Widget to a class that inherits it
(WidgetFactory, Widget) = make_factory(Widget)

I use a slightly different technique in my unit tests, cut-n-pasted here
it defines the class-tracker using Register and returns the base object.
It also defines a main() method for the test suite. My test-coverage
class is in here too, but that is extra. It just defines a TestCase
that checks for test coverage versus implemented tests.

Last things first, here is how it is used
""" t_libUtil.py - test 'libUtil' module """
import test # defined below
import libUtil

(Test, main) = test.setup(libUtil)

class SimpleSum(Test):
we_test = [libUtil.simple_sum]
def test_sum(self):
f = libUtil.simple_sum
x = [0,0]
self.assertEqual([0,3], f(x,[1,3]) or x)

if (__name__ == '__main__'):
main()


# module 'test.py'
import unittest

class Register(type):
"""A tiny metaclass that keeps track of all classess"""
def __init__(cls, name, bases, dict):
if ('register' in dict): # initial dummy class
setattr(cls, 'register', staticmethod(dict['register']))
elif (object not in bases):
cls.register(name, cls)
return

def setup(module):
class Test(unittest.TestCase, object): # TestCase is an old-style class
__metaclass__ = Register
alltests = [] # all Test classes
allfuncs = [] # list of all the functions they test

def register(name, cls):
Test.alltests.append(cls)
Test.allfuncs += cls.we_test

class TestCoverage(Test):
"""make sure we have tests for all the funcs and classes in the module"""
we_test = []
def test_coverage(self):
# find all functions and classes in the module
need = []
for (name, val) in module.__dict__.items():
if (callable(val)):
need.append(val)
try:
if (val.__class__.__name__ == name): # this finds classes defined in the module
need.append(val)
except: pass

for (ob) in need:
good = ob in Test.allfuncs
if (not good):
raise self.failureException("No Test coverage: %s" % (str(ob)))

def main():
suite = unittest.TestSuite()
test_classes = Test.alltests
for cls in test_classes:
suite.addTest(unittest.makeSuite(cls))
result = unittest.TestResult()
suite(result)
msg = []
if (result.errors):
msg.append('Failure')
if (result.errors):
msg.append('Error')
print "%s: %d %s" % (module.__name__, result.testsRun, " & ".join(msg) or 'Success')
for (err) in result.errors:
print "\n".join(map(str, err))
for (failure) in result.failures:
print "\n".join(map(str, failure))
return

return (Test, main)


In places like unittest where we already do heavy introspection
using metaclasses to do the introspecition is usually a big win.

long-posted-ly,

-jackdied
 
D

Daniel Klein

import sys

def getClass(classname, modulename):
try:
module = sys.modules[modulename]
except KeyError:
module = __import__(modulename)
return getattr(module, classname)

print getClass("WichmannHill", "random")
print getClass("WichmannHill", "random")()

Any reason why it couldn't be done like this?

modulename = 'random'
classname = 'WichmannHill'

try:
print eval(modulename + '.' + classname + '()')
except AttributeError:
print 'Invalid module and/or class'

Dan
 
P

Peter Otten

Daniel said:
Any reason why it couldn't be done like this?

modulename = 'random'
classname = 'WichmannHill'

try:
print eval(modulename + '.' + classname + '()')
except AttributeError:
print 'Invalid module and/or class'

It won't work :-(
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<string>", line 0, in ?
NameError: name 'random' is not defined

You have to import first, which cannot be done via eval:Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<string>", line 1
import random
^
SyntaxError: invalid syntax
Generally I don't like parsing an expression from a string when I can
achieve the same by "normal" function calls, i. e. I only take eval() as a
quick and dirty resolution or a last resort.

If you want to instantiate more than one instance of the class or other
classes in the same module (the module might contain a plugin interface)
you could easily refine my getClass() to cache modules and classes in a
dictionary, thus avoiding the overhead of repeatedly importing and
evaluating.

Peter
 
T

Tomasz Lisowski

Uzytkownik "Peter Otten said:
Marco said:
1. How to do the import? I didn't find a way to give a string to the
import statement.

2. How to write such code to instantiate?

Off topic: This thread like some others is spread over a few toplevel
entries in my newsreader (KNode 0.7.2). Is this a bug?

Anyway, putting it all together:

import sys

def getClass(classname, modulename):
try:
module = sys.modules[modulename]
except KeyError:
module = __import__(modulename)
return getattr(module, classname)

Hmm...

Why not write it directly:

def getClass(classname, modulename):
try:
module = __import__(modulename)
except ImportError:
return None
else:
return getattr(module, classname)

I have heard, that the __import__ function is smart enough to check the
sys.modules dictionary without actually re-importing the once imported
module. Is it true?

Regards,
Tomasz Lisowski
 
P

Peter Otten

Tomasz said:
import sys

def getClass(classname, modulename):
try:
module = sys.modules[modulename]
except KeyError:
module = __import__(modulename)
return getattr(module, classname)

Hmm...

Why not write it directly:

def getClass(classname, modulename):
try:
module = __import__(modulename)
except ImportError:
return None
else:
return getattr(module, classname)

I have heard, that the __import__ function is smart enough to check the
sys.modules dictionary without actually re-importing the once imported
module. Is it true?

I think you are right with respect to __import__(). However, it's a bad idea
to catch the import error. Your client code should have determined the
module name beforehand, so that something has gone seriously wrong when you
get an ImportError. Returning None instead of a class only confuses the
issue (nobody wants to go back to the C habit of checking every function's
return value for an errorr code). So:

def getClass(classname, modulename):
return getattr(__import__(modulename), classname)

would be a shorter equivalent to my getClass(). It seems that a lookup in
sys.modules is done implicitely by __import__(), as I tested it
successfully with "__main__" as the module name.

Note that all three versions will not work with submodules, e. g. "os.path",
so let's change it one more (last?) time:

def getClass(classname, modulename):
return getattr(__import__(modulename, globals(), locals(), [classname]),
classname)

Peter
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top