Automatic import of submodules

Discussion in 'Python' started by Massi, Nov 25, 2011.

  1. Massi

    Massi Guest

    Hi everyone,

    in my project I have the following directory structure:

    plugins
    |
    -- wav_plug
    |
    -- __init__.py
    -- WavPlug.py
    -- mp3_plug
    |
    -- __init__.py
    -- Mp3Plug.py
    ....
    -- etc_plug
    |
    -- __init__.py
    -- EtcPlug.py

    Every .py file contain a class definition whose name is identical to
    to the file name, so in my main script I have to import each submodule
    like that:

    from plugins.wav_plug.WavPlug import WavPlug
    from plugins.wav_plug.Mp3Plug import Mp3Plug

    and so on. This is uncomfortable, since when a new plugin is added I
    have to import it too. So my question is, is it possible to iterate
    through the 'plugins' directory tree in order to automatically import
    the submodules contained in each subdirectory?
    I googled and found that the pkgutil could help, but it is not clear
    how. Any hints?
    Thanks in advance.
    Massi, Nov 25, 2011
    #1
    1. Advertising

  2. Massi

    Dave Angel Guest

    On 11/25/2011 08:00 AM, Massi wrote:
    > Hi everyone,
    >
    > in my project I have the following directory structure:
    >
    > plugins
    > |
    > -- wav_plug
    > |
    > -- __init__.py
    > -- WavPlug.py
    > -- mp3_plug
    > |
    > -- __init__.py
    > -- Mp3Plug.py
    > ...
    > -- etc_plug
    > |
    > -- __init__.py
    > -- EtcPlug.py
    >
    > Every .py file contain a class definition whose name is identical to
    > to the file name, so in my main script I have to import each submodule
    > like that:
    >
    > from plugins.wav_plug.WavPlug import WavPlug
    > from plugins.wav_plug.Mp3Plug import Mp3Plug
    >
    > and so on. This is uncomfortable, since when a new plugin is added I
    > have to import it too. So my question is, is it possible to iterate
    > through the 'plugins' directory tree in order to automatically import
    > the submodules contained in each subdirectory?
    > I googled and found that the pkgutil could help, but it is not clear
    > how. Any hints?
    > Thanks in advance.
    >

    I think the key to the problem is the __import__() function, which takes
    a string and returns a module object. So you make a list of the fully
    qualified module names (filenames less the extension), and loop through
    that list, Then for each one, you can extract items from the module.

    It doesn't make sense to define the class names at your top-level,
    though, since you'd not have any code to reference any new plugin if it
    has a unique class name. So at some point, you're probably going to have
    a list or map of such class objects.



    --

    DaveA
    Dave Angel, Nov 25, 2011
    #2
    1. Advertising

  3. Massi wrote:
    > Hi everyone,
    >
    > in my project I have the following directory structure:
    >
    > plugins
    > |
    > -- wav_plug
    > |
    > -- __init__.py
    > -- WavPlug.py
    > -- mp3_plug
    > |
    > -- __init__.py
    > -- Mp3Plug.py
    > ...
    > -- etc_plug
    > |
    > -- __init__.py
    > -- EtcPlug.py
    >
    > Every .py file contain a class definition whose name is identical to
    > to the file name, so in my main script I have to import each submodule
    > like that:
    >
    > from plugins.wav_plug.WavPlug import WavPlug
    > from plugins.wav_plug.Mp3Plug import Mp3Plug
    >
    > and so on. This is uncomfortable, since when a new plugin is added I
    > have to import it too. So my question is, is it possible to iterate
    > through the 'plugins' directory tree in order to automatically import
    > the submodules contained in each subdirectory?
    > I googled and found that the pkgutil could help, but it is not clear
    > how. Any hints?
    > Thanks in advance.
    >
    >

    Hi,

    Try something like (*untested code*)

    plugins = {}
    classes = {}

    for plugin, className in [('wav_plug', 'WavPlug'), ('mp3_plug', 'Mp3Plug')]:
    plugins[plugin] = __import__(os.path.join('plugins', plugin, className))
    classes[className] = getattr(plugins[plugin], className)

    # raise a keyError if the plugin has not been imported
    wav = classes['wav_plug']()


    Make sure all subdirs have the __init__.py file, including the plugins
    directory.

    JM
    Jean-Michel Pichavant, Nov 25, 2011
    #3
  4. Massi

    alex23 Guest

    On Nov 25, 11:00 pm, Massi <> wrote:
    > plugins
    >     |
    >     -- wav_plug
    >           |
    >           -- __init__.py
    >           -- WavPlug.py
    >     -- mp3_plug
    >           |
    >           -- __init__.py
    >           -- Mp3Plug.py
    > ...
    >     -- etc_plug
    >           |
    >           -- __init__.py
    >           -- EtcPlug.py


    What do you gain by having each plugin as a package? Unless you're
    storing other resources with each plugin, I'd move all your XXXPlug.py
    files into plugins/ I'd also probably call the modules 'wavplug' and
    the class 'WavPlug' to always make it clear to which you're
    referring.

    > Every .py file contain a class definition whose name is identical to
    > to the file name, so in my main script I have to import each submodule
    > like that:
    >
    > from plugins.wav_plug.WavPlug import WavPlug
    > from plugins.wav_plug.Mp3Plug import Mp3Plug
    >
    > and so on. This is uncomfortable, since when a new plugin is added I
    > have to import it too. So my question is, is it possible to iterate
    > through the 'plugins' directory tree in order to automatically import
    > the submodules contained in each subdirectory?


    It's not exactly automatic, but you could move all of those imports
    into plugins/__init__.py, then just do a single

    from plugins import *

    in your main module.
    alex23, Nov 26, 2011
    #4
  5. -----Original Message-----
    From: Massi [mailto:]
    Sent: 25/11/2011 6:30 PM
    To:
    Subject: Automatic import of submodules

    Hi everyone,

    in my project I have the following directory structure:

    plugins
    |
    -- wav_plug
    |
    -- __init__.py
    -- WavPlug.py
    -- mp3_plug
    |
    -- __init__.py
    -- Mp3Plug.py
    ....
    -- etc_plug
    |
    -- __init__.py
    -- EtcPlug.py

    Every .py file contain a class definition whose name is identical to
    to the file name, so in my main script I have to import each submodule
    like that:

    from plugins.wav_plug.WavPlug import WavPlug
    from plugins.wav_plug.Mp3Plug import Mp3Plug

    and so on. This is uncomfortable, since when a new plugin is added I
    have to import it too. So my question is, is it possible to iterate
    through the 'plugins' directory tree in order to automatically import
    the submodules contained in each subdirectory?
    I googled and found that the pkgutil could help, but it is not clear
    how. Any hints?
    Thanks in advance.


    =====
    What you could do is :

    Instead of importing method wise , import the module.

    from plugins.wav_plug import WavPlug

    you can then use like this then
    WavPlug.WavPlug

    This will avoid the-every time importing of each method

    Regards,
    Shambhu
    Shambhu Rajak, Nov 28, 2011
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. TG

    access to submodules

    TG, Jul 19, 2006, in forum: Python
    Replies:
    10
    Views:
    562
  2. Frank Aune

    Python submodules and name imports

    Frank Aune, Aug 23, 2007, in forum: Python
    Replies:
    1
    Views:
    804
    =?iso-8859-1?B?UOFkcmFpZw==?=
    Aug 23, 2007
  3. Donn Ingle
    Replies:
    7
    Views:
    264
    Gabriel Genellina
    Dec 19, 2007
  4. Torsten Mohr

    Noddy with submodules?

    Torsten Mohr, Sep 7, 2009, in forum: Python
    Replies:
    1
    Views:
    303
    Gabriel Genellina
    Sep 8, 2009
  5. Replies:
    1
    Views:
    110
    Peter Otten
    Jan 11, 2013
Loading...

Share This Page