Re: Importing * From a Package

Discussion in 'Python' started by Patrick Doyle, Aug 7, 2007.

  1. > There's no reliable way to differentiate between names which you defined
    > using something other than an import statement, names you defined with an
    > import statement with the intent of publishing them as part of your external
    > API, and names you defined with an import statement which you only intended
    > to use internally. So Python doesn't even try to differentiate betweem them.
    >

    ok, that makes sense

    > >"[It] imports whatever names are defined in the package [including]
    > >any submodules of the package that were explicitly loaded by previous
    > >import statements."

    This is the part that has me confused -- why does "from package import
    *" go on to import names that were explicitly loaded by previous
    import statements?

    Here is a contrived example (contrived by virtue of culling it out of
    an existing code base I was reorganizing into packages when I stumbled
    across this:

    Directory structure:
    ../
    ../SDRGen
    ../SDRGen/__init__.py
    ../SDRGen/TFGenerator.py

    The SDRGen/__init__.py file is empty.
    The SDRGen/TFGenerator.py file contains the following 2 lines:

    class TFGenerator:
    pass


    Now, at the python prompt I type (and get) the following:

    >>> from SDRGen.TFGenerator import *
    >>> TFGenerator

    <class SDRGen.TFGenerator.TFGenerator at 0xb7eae8bc>
    >>> from SDRGen import *
    >>> TFGenerator

    <module 'SDRGen.TFGenerator' from 'SDRGen/TFGenerator.py'

    It looks to me like, when I executed the "from SDRGen import *"
    statement, the import "[included a] submodule of the package that
    [was] explicitly loaded by [a] previous import statement." Granted
    this example is probably not the best example of style and grace in a
    Python program, and I'm in the process of cleaning up things such as
    "from ... import *" and naming files and classes with identical names,
    but when I stumbled across this I found it confusing, to say the
    least.

    Why does Python include the submodules that were explicitly loaded by
    previous imports? Does it go out of it's way to do so? If so, why?
    What purpose does it serve? Or, is it a natural fallout of the manner
    in which imports are processed? If so, could somebody guide my
    intuition as to why this would be such a natural fallout?

    Thanks for the enlightenment...

    --wpd
    Patrick Doyle, Aug 7, 2007
    #1
    1. Advertising

  2. Patrick Doyle

    Duncan Booth Guest

    "Patrick Doyle" <> wrote:

    > Why does Python include the submodules that were explicitly loaded by
    > previous imports? Does it go out of it's way to do so? If so, why?
    > What purpose does it serve? Or, is it a natural fallout of the manner
    > in which imports are processed? If so, could somebody guide my
    > intuition as to why this would be such a natural fallout?
    >


    It is a natural fallout of the manner in which imports are processed.
    When you explicitly load a submodule a reference to the submodule is
    stored in the parent module. When you do 'from module import *' and the
    imported module doesn't define __all__, you create a reference in the
    current module to everything referenced by the imported module (except
    for variables beginning with '_').

    Python makes no distinction at all between the objects named in the
    imported module: they all get imported, i.e. assigned into the current
    scope no matter how they got there.

    If you use your example:

    >>> import SDRGen
    >>> dir(SDRGen)

    ['__builtins__', '__doc__', '__file__', '__name__', '__path__']
    >>> from SDRGen.TFGenerator import *
    >>> dir(SDRGen)

    ['TFGenerator', '__builtins__', '__doc__', '__file__', '__name__',
    '__path__']
    >>>


    The line 'from SDRGen.TFGenerator import *' always creates the name
    TFGenerator in the SDRGen module (and it will create the SDRGen module
    if it has to). A subsequent 'from SDRGen import *' is simply doing the
    equivalent of:

    for name in dir(SDRGen):
    if not name.startswith('_'):
    setattr(current_module, getattr(SDRGen, name))

    except of course the variables name and current_module don't exist.
    Duncan Booth, Aug 7, 2007
    #2
    1. Advertising

  3. On 7 Aug 2007 13:54:21 GMT, Duncan Booth <> wrote:
    > "Patrick Doyle" <> wrote:
    >
    > > Why does Python include the submodules that were explicitly loaded by
    > > previous imports? Does it go out of it's way to do so? If so, why?
    > > What purpose does it serve? Or, is it a natural fallout of the manner
    > > in which imports are processed? If so, could somebody guide my
    > > intuition as to why this would be such a natural fallout?
    > >

    >
    > It is a natural fallout of the manner in which imports are processed.
    > When you explicitly load a submodule a reference to the submodule is
    > stored in the parent module. When you do 'from module import *' and the
    > imported module doesn't define __all__, you create a reference in the
    > current module to everything referenced by the imported module (except
    > for variables beginning with '_').
    >

    Ahhh, I see it now
    >>> import SDRGen
    >>> dir()

    ['SDRGen', '__builtins__', '__doc__', '__name__']

    >>> dir(SDRGen)

    ['__builtins__', '__doc__', '__file__', '__name__', '__path__']

    >>> from SDRGen import *
    >>> dir()

    ['SDRGen', '__builtins__', '__doc__', '__name__']
    >>> dir(SDRGen)

    ['__builtins__', '__doc__', '__file__', '__name__', '__path__']
    (notice, no changes)

    >>> from SDRGen.TFGenerator import TFGenerator
    >>> dir()

    ['SDRGen', 'TFGenerator', '__builtins__', '__doc__', '__name__']
    >>> TFGenerator

    <class SDRGen.TFGenerator.TFGenerator at 0xb7ee80ec>
    (as expected)

    >>> dir(SDRGen)

    ['TFGenerator', '__builtins__', '__doc__', '__file__', '__name__', '__path__']
    >>> SDRGen.TFGenerator

    <module 'SDRGen.TFGenerator' from 'SDRGen/TFGenerator.pyc'>

    and that's what led to my confusion... when my script later did a
    >>> from SDRGen import *


    The 'TFGenerator' name got changed from a reference to the
    'TFGenerator' class to a reference to the 'SDRGen.TFGenerator' module.

    and now it makes sense.

    Thanks.

    --wpd
    Patrick Doyle, Aug 7, 2007
    #3
  4. Patrick Doyle

    greg Guest

    Patrick Doyle wrote:
    > This is the part that has me confused -- why does "from package import
    > *" go on to import names that were explicitly loaded by previous
    > import statements?


    Because there's no easy way for it *not* to do that.
    All it does is grab whatever names are defined in the
    module at the time. It can't tell whether those names
    got there as a result of an import statement or some
    other way.

    A more pertinent question is why it *doesn't* import
    submodules that haven't already been explicitly imported.
    Probably this is just because it could be quite expensive
    to do so -- you would potentially be triggering a lot
    of disk activity to load the previously unused modules
    into memory, many of which you may not be going to use.

    Just in case it's not clear, "importing" can actually
    mean two different things. If the module hasn't been
    imported before, it has to be loaded into memory, which
    is expensive. If it has been imported before, then
    you're just creating another reference to the same
    module object, which is very cheap. An import *
    just does the cheap part, and leaves you to explicitly
    ask for the expensive part if you really want it.

    --
    Greg
    greg, Aug 8, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. mhk
    Replies:
    1
    Views:
    610
    Chris Smith
    Nov 28, 2003
  2. Alex
    Replies:
    0
    Views:
    505
  3. Javabean

    importing for package

    Javabean, Mar 1, 2006, in forum: Java
    Replies:
    4
    Views:
    520
  4. Dave
    Replies:
    2
    Views:
    467
  5. plb
    Replies:
    2
    Views:
    344
Loading...

Share This Page