user modules

[Cameron Walsh]

Hi,

I'm writing a python program to analyse and export volumetric data. To
make development and extension easier, and to make it more useful to the
public when it is released (LGPL), I would like to enable users to place
their own python files in a "user_extensions" directory. These files
would implement a common interface in order for the main program to be
able to read them and execute the necessary code.

My question is what is the best way of implementing this?

I have investigated importing them as modules, but unless the user
modifies the main program I cannot see how the main program can learn of
the existence of specific modules.

For example:

from user_modules import *
# directory 'user_modules' contains __init__.py allowing this
# From here I would need a list of the imported modules, in order to
# execute a common command on each of them, such as

for module in imported_modules:
module.initialise()
module.get_tab_window()


How do I get from the first bit to the second bit, or is there a better
way of obtaining the functionality I need?


--Cameron.

 

[Tuomas]

Hi,

I'm writing a python program to analyse and export volumetric data. To
make development and extension easier, and to make it more useful to the
public when it is released (LGPL), I would like to enable users to place
their own python files in a "user_extensions" directory. These files
would implement a common interface in order for the main program to be
able to read them and execute the necessary code.

My question is what is the best way of implementing this?

I have investigated importing them as modules, but unless the user
modifies the main program I cannot see how the main program can learn of
the existence of specific modules.

For example:

from user_modules import *
# directory 'user_modules' contains __init__.py allowing this
# From here I would need a list of the imported modules, in order to
# execute a common command on each of them, such as

for module in imported_modules:
module.initialise()
module.get_tab_window()


How do I get from the first bit to the second bit, or is there a better
way of obtaining the functionality I need?


--Cameron.

import os

files=os.listdir('user_modules')
tabs=[]
for fle in files:
if fle.endswith('.py'):
module=__import__(fle[0:-3], 'user_modules', None,
['initialise', 'get_tab_window'])
module.initialise()
tabs.append(module.get_tab_window())

*not tested*

print __import__.__doc__
__import__(name, globals, locals, fromlist) -> module

Import a module. The globals are only used to determine the context;
they are not modified. The locals are currently unused. The fromlist
should be a list of names to emulate ``from name import ...'', or an
empty list to emulate ``import name''.
When importing a module from a package, note that __import__('A.B', ...)
returns package A when fromlist is empty, but its submodule B when
fromlist is not empty.

Tuomas

 

[Juho Schultz]

Hi,

I'm writing a python program to analyse and export volumetric data. To
make development and extension easier, and to make it more useful to the
public when it is released (LGPL), I would like to enable users to place
their own python files in a "user_extensions" directory. These files
would implement a common interface in order for the main program to be
able to read them and execute the necessary code.

My question is what is the best way of implementing this?

I have investigated importing them as modules, but unless the user
modifies the main program I cannot see how the main program can learn of
the existence of specific modules.

One simple solution would be a shell script that adds user_extensions
(or whatever) to $PYTHONPATH and then starts your main program.

 

[Juho Schultz]

Cameron Walsh kirjoitti:

Hi,

I'm writing a python program to analyse and export volumetric data. To
make development and extension easier, and to make it more useful to the
public when it is released (LGPL), I would like to enable users to place
their own python files in a "user_extensions" directory. These files
would implement a common interface in order for the main program to be
able to read them and execute the necessary code.

My question is what is the best way of implementing this?

I have investigated importing them as modules, but unless the user
modifies the main program I cannot see how the main program can learn of
the existence of specific modules.

For example:

from user_modules import *
# directory 'user_modules' contains __init__.py allowing this
# From here I would need a list of the imported modules, in order to
# execute a common command on each of them, such as

for module in imported_modules:
module.initialise()
module.get_tab_window()


How do I get from the first bit to the second bit, or is there a better
way of obtaining the functionality I need?


--Cameron.

Sorry... I was typing faster than reading or thinking.

You could have a __init__.py file within user_extensions with
__all__ = ["package1", "package2"]

If you want every python file within some directory in here, you can
auto-generate the __init__.py file in user_extension before importing.
(Or you could have some sort of tester for new .py files detected and
only after you are sure it works, add it.)

from user_extensions import *

would import everything mentioned in __all__. You also have access to
their names through
user_extensions.__all__

The last step would be to put the modules into a list. After the
import,

user_ext_list = [eval(elem) for elem in user_extensions.__all__ ]
for ext in user_ext_list:
ext.initialize()
ext.get_tab_window()

 

[Juho Schultz]

One simple solution would be a shell script that adds user_extensions
(or whatever) to $PYTHONPATH and then starts your main program.

Sorry... I was typing faster than reading or thinking.

You could have a __init__.py file within user_extensions with
__all__ = ["package1", "package2"]

If you want every python file within some directory in here, you can
auto-generate the __init__.py file in user_extension before importing.
(Or you could have some sort of tester for new .py files detected and
only after you are sure it works, add it.)

from user_extensions import *

would import everything mentioned in __all__. You also have access to
their names through
user_extensions.__all__

The last step would be to put the modules into a list. After the
import,

user_ext_list = [eval(elem) for elem in user_extensions.__all__ ]
for ext in user_ext_list:
ext.initialize()
ext.get_tab_window()

 

[Cameron Walsh]

Hi,

I'm writing a python program to analyse and export volumetric data.
To make development and extension easier, and to make it more useful
to the public when it is released (LGPL), I would like to enable users
to place their own python files in a "user_extensions" directory.
These files would implement a common interface in order for the main
program to be able to read them and execute the necessary code.

My question is what is the best way of implementing this?

I have investigated importing them as modules, but unless the user
modifies the main program I cannot see how the main program can learn
of the existence of specific modules.

For example:

from user_modules import *
# directory 'user_modules' contains __init__.py allowing this
# From here I would need a list of the imported modules, in order to
# execute a common command on each of them, such as

for module in imported_modules:
module.initialise()
module.get_tab_window()


How do I get from the first bit to the second bit, or is there a
better way of obtaining the functionality I need?


--Cameron.

import os

files=os.listdir('user_modules')
tabs=[]
for fle in files:
if fle.endswith('.py'):
module=__import__(fle[0:-3], 'user_modules', None,
['initialise', 'get_tab_window'])
module.initialise()
tabs.append(module.get_tab_window())

*not tested*

print __import__.__doc__
__import__(name, globals, locals, fromlist) -> module

Import a module. The globals are only used to determine the context;
they are not modified. The locals are currently unused. The fromlist
should be a list of names to emulate ``from name import ...'', or an
empty list to emulate ``import name''.
When importing a module from a package, note that __import__('A.B', ...)
returns package A when fromlist is empty, but its submodule B when
fromlist is not empty.

Tuomas

Thanks Tuomas, your solution worked a charm, with two changes:
1.) Check I'm not trying to import __init__.py
2.) module=__import__(fle[0:-3], 'user_modules', None,
['initialise', 'get_tab_window'])
had to be changed to:
module=__import__("user_modules.%s" %fle[0:-3], None, None,
['initialise', 'get_tab_window'])

For some reason it wasn't setting the context properly, I'll have to
learn more about the function.

Thanks for teaching me how to read the documentation properly, about how
private keywords like "import" are defined, and for providing a solution
to a problem I'm likely to face many more times.

Best regards,

Cameron.

 

[Cameron Walsh]

One simple solution would be a shell script that adds user_extensions
(or whatever) to $PYTHONPATH and then starts your main program.

Sorry... I was typing faster than reading or thinking.

You could have a __init__.py file within user_extensions with
__all__ = ["package1", "package2"]

If you want every python file within some directory in here, you can
auto-generate the __init__.py file in user_extension before importing.
(Or you could have some sort of tester for new .py files detected and
only after you are sure it works, add it.)

from user_extensions import *

would import everything mentioned in __all__. You also have access to
their names through
user_extensions.__all__

The last step would be to put the modules into a list. After the
import,

user_ext_list = [eval(elem) for elem in user_extensions.__all__ ]
for ext in user_ext_list:
ext.initialize()
ext.get_tab_window()

Thanks Juho,

Your solution also worked a charm. I like the suggestion of a tester
before adding a module to the __all__ list.

Thanks for teaching me about the eval() function, I've been wondering
how to turn a string into a python command.