simulating #include in python

[Nick Craig-Wood]

We are currently investigating whether to move the data files from our
application into python for ease of maintenance. Each data item turns
into a class definition with some class data. The python approach
looks great, but there is one feature that we'd like to have.

Currently the data files can include other data files. Typically
thats used to import standard definitions from somewhere else (rather
like #include in C - conceptually its a completely textual import).
We use them something like this

include("../../standard_definitions")
include("../shared_definitions")

class A(base_from_standard_definitions): pass
class B(A): pass

include("more_definitions")

The includes act much more like #include than import - all the symbols
from the file before the include() must be available to the included
file and the included file must export all its symbols back to the
parent.

These can of course be re-arranged to work in a pythonic way using
'from x import *' and putting "../.." on sys.path instead of the
includes. However there are over 500 of these files so I'd prefer a
more automatic solution which doesn't require re-arrangement in the
interim. (The re-arrangement is needed because "more_definitions"
above might refer to A, B or anything defined in
standard/shared_definitions leading to mutually recursive imports and
all the pain they cause)

I have implemented a working prototype, but it seems such a horrendous
bodge that there must surely be a better way! I'd really like to be
able to run an __import__ in the context of the file thats running the
include() but I haven't figured that out.

Here is the code (avert your eyes if you are of a sensitive nature ;-)
Any suggestions for improvement would be greatly appreciated!

def include(path):

# Add the include directory onto sys.path
native_path = path.replace("/", os.path.sep)
directory, module_name = os.path.split(native_path)
if module_name.endswith(".py"):
module_name = module_name[:-3]
old_sys_path = sys.path
if directory != "":
sys.path.insert(0, directory)

# Introspect to find the parent
# Each record contains a frame object, filename, line number, function
# name, a list of lines of context, and index within the context.
up = inspect.stack()[1]
frame = up[0]
parent_name = frame.f_globals['__name__']
parent = sys.modules[parent_name]

# Poke all the current definitions into __builtin__ so the module
# uses them without having to import them
old_builtin = __builtin__.__dict__.copy()
overridden = {}
poked = []
for name in dir(parent):
if not (name.startswith("__") and name.endswith("__")):
if hasattr(__builtin__, name):
overridden[name] = getattr(__builtin__, name)
else:
poked.append(name)
setattr(__builtin__, name, getattr(parent, name))

# import the code
module = __import__(module_name, parent.__dict__, locals(), [])

# Undo the modifications to __builtin__
for name in poked:
delattr(__builtin__, name)
for name, value in overridden.items():
setattr(__builtin__, name, value)

# check we did it right! Note __builtin__.__dict__ is read only so
# can't be over-written
if old_builtin != __builtin__.__dict__:
raise AssertionError("Failed to restore __builtin__ properly")

# Poke the symbols from the import back in
for name in dir(module):
if not (name.startswith("__") and name.endswith("__")):
setattr(parent, name, getattr(module, name))

# Restore sys.path
sys.path = old_sys_path

 

[Peter Otten]

I'd really like to be able to run an __import__ in the context of the file
thats running the include() but I haven't figured that out.

execfile()?

Peter

 

[Nick Craig-Wood]

execfile()?

Yes thats exactly what I was looking for - thank you very much!