Why doesn't `from pkg import mod' work after `del pkg.mod'?

[ryles]

According to http://www.python.org/doc/essays/packages.html:

"The import statement first tests whether the item is defined in the
package; if not, it assumes it is a module and attempts to load it."

However, I've noticed that once a module is imported using the
`from pkg import mod' syntax, if its name is deleted from the
package namespace then subsequent imports with `from' will fail.

Here is an example of this type of scenario:

$ ls -l pkg
total 8.0K
-rw-rw-r-- 1 ? general 0 Jul 24 20:21 _impl.py
-rw-rw-r-- 1 ? general 147 Jul 24 20:28 __init__.py
-rw-rw-r-- 1 ? general 0 Jul 24 20:33 public2.py
-rw-rw-r-- 1 ? general 208 Jul 24 20:32 public.py

$ cat pkg/__init__.py
from pkg import _impl

# Add functions which refer to objects in _impl
# ...

# Don't "currently" require this in the namespace anymore.
del _impl

$ cat pkg/public.py
# Implement something with the aid of _impl.

from pkg import public2 # OK, fine.
from pkg import _impl # The module exists, but this will fail.
# Could do import pkg._impl (or a relative import)

$ python
Python 2.6.2 (r262:71600, Jul 16 2009, 14:04:28)
[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pkg/public.py", line 4, in <module>
from pkg import _impl # The module exists, but this will fail.
ImportError: cannot import name _impl

import sys
sys.modules["pkg._impl"]

from pkg import _impl

Traceback (most recent call last):

I had previously noted that once a module is imported from a package
it is automatically added to the package namespace:
['__builtins__', '__doc__', '__file__', '__name__', '__package__',
'__path__']['__builtins__', '__doc__', '__file__', '__name__', '__package__',
'__path__', 'public2']
However, I didn't expect python to give up on importing those names
again
once removed.

Can anyone provide more details on what's occurring here?

Is this behavior intentional (a feature), or, is it in your opinion, a
defect?

 

[Piet van Oostrum]

r> According to http://www.python.org/doc/essays/packages.html:
r> "The import statement first tests whether the item is defined in the
r> package; if not, it assumes it is a module and attempts to load it."

r> However, I've noticed that once a module is imported using the
r> `from pkg import mod' syntax, if its name is deleted from the
r> package namespace then subsequent imports with `from' will fail.

This is incorrectly stated. Also on the initial import it will fail, not
just on subsequent imports.

piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information._impl # this is from a print statement in _impl to show that it did import
Traceback (most recent call last):

According to the documentation
<http://docs.python.org/library/functions.html#__import__> the statement
from pkg import _impl

_temp = __import__('pkg', globals(), locals(), ['_impl'], -1)
_impl = _temp._impl

This fails because _temp (the imported module) doesn't have a binding
for _impl because you deleted it. By the way, if you have a `normal'
package that doesn't do anything with the name _impl, after the import
the pkg module namespace will have got a binding for _impl although it is not
in your code. It will have been put there by the import code.
In the following example pkg2/__init__.py just contains the line
A = 2

piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

from pkg2 import _impl _impl
import sys
dir(sys.modules['pkg2']) ['A', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '_impl']

It is not clear to me, however what the order is of the statements in
__init__.py and the insertion of _impl in the module namespace.

 

[ryles]

r> According tohttp://www.python.org/doc/essays/packages.html:
r> "The import statement first tests whether the item is defined in the
r> package; if not, it assumes it is a module and attempts to load it."
r> However, I've noticed that once a module is imported using the
r> `from pkg import mod' syntax, if its name is deleted from the
r> package namespace then subsequent imports with `from' will fail.

This is incorrectly stated. Also on the initial import it will fail, not
just on subsequent imports.

piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.>>> from pkg import _impl

_impl # this is from a print statement in _impl to show that it did import
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name _impl

Well, since __init__.py is executed first, I would regard it as
causing the initial (successful) import, and yours being the
subsequent one, so that the "initial" import does not fail, the
subsequent one does. Wording aside, though, I think we are on the same
page here.

According to the documentation
<http://docs.python.org/library/functions.html#__import__> the statement
from pkg import _impl

_temp = __import__('pkg', globals(), locals(), ['_impl'], -1)
_impl = _temp._impl

This fails because _temp (the imported module) doesn't have a binding
for _impl because you deleted it.

But supposing we hadn't deleted it, and __init__.py didn't import
_impl, there would still be no binding. Yet the importer would still
import and make the module available (presumably it "knows" to do this
since it was not already in sys.modules). The question really is that
since in our example pkg._impl is still in sys.modules, why won't the
importer add it to the pkg namespace again as it previous had? I would
imagine this is either by design, or is a case that was never
considered.

for _impl because you deleted it. By the way, if you have a `normal'
package that doesn't do anything with the name _impl, after the import
the pkg module namespace will have got a binding for _impl although it is not
in your code. It will have been put there by the import code.

Yes, this was noted further down in the original post with an example.

 

[Piet van Oostrum]

r> According tohttp://www.python.org/doc/essays/packages.html:
r> "The import statement first tests whether the item is defined in the
r> package; if not, it assumes it is a module and attempts to load it."
r> However, I've noticed that once a module is imported using the
r> `from pkg import mod' syntax, if its name is deleted from the
r> package namespace then subsequent imports with `from' will fail.

This is incorrectly stated. Also on the initial import it will fail, not
just on subsequent imports.

piet$ python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
from pkg import _impl

_impl # this is from a print statement in _impl to show that it did import
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name _impl

r> Well, since __init__.py is executed first, I would regard it as
r> causing the initial (successful) import, and yours being the
r> subsequent one, so that the "initial" import does not fail, the
r> subsequent one does. Wording aside, though, I think we are on the same
r> page here.

I was talking about the 'from pkg import _impl' statements because I
thought that's what we were talking about. Of course this does an
implicit 'import pkg' first, but that doesn't fail. In your OP you were
also talking about `from pkg import mod' imports and said that the
subsequent ones of these fails. I noticed that also the first one of
these would fail.

According to the documentation
<http://docs.python.org/library/functions.html#__import__> the statement
from pkg import _impl

_temp = __import__('pkg', globals(), locals(), ['_impl'], -1)
_impl = _temp._impl

This fails because _temp (the imported module) doesn't have a binding
for _impl because you deleted it.

r> But supposing we hadn't deleted it, and __init__.py didn't import
r> _impl, there would still be no binding. Yet the importer would still
r> import and make the module available (presumably it "knows" to do this
r> since it was not already in sys.modules). The question really is that
r> since in our example pkg._impl is still in sys.modules, why won't the
r> importer add it to the pkg namespace again as it previous had? I would
r> imagine this is either by design, or is a case that was never
r> considered.

Yes, that is also something I don't understand.

r> Yes, this was noted further down in the original post with an example.

Sorry, I missed that.