package module import name clash with global package

[George P]

I've run into a strange package related namespace problem.

Follow these steps (on UNIX system) to illustrate:

-------------------------
mkdir /tmp/mypkg
cd /tmp/mypkg
touch __init__.py
echo 'import os\ndef test():\n print os.getcwd()' > os.py
cd /tmp

echo '#\!/usr/bin/env python\nimport mypkg.os\nmypkg.os.test()' >
test_1.py
echo '#\!/usr/bin/env python\nimport os\nprint os.getcwd()' >
test_2.py
chmod a+x test_*.py
-------------------------

This sets up a minimal package mypkg in /tmp that contains a
sub-module named os. It also sets up a test_1.py and test_2.py script
to illustrate the problem. They do equivalent things (print the cwd)
except test_1.py tries and fails to do it from within the mypkg.os
module.

The problem is then that I can't import the global package "os" into
my mypkg.os packaged module. I've done lots of digging around into
this problem and I think I understand the reasons why this happens.

My question is if anyone can think of an alternative way of
fixing/avoiding this?

The best workarounds I have are:
a) Not use os as my submodule name
b) First import global os in test_1.py and then in mypkg/os.py run
the function as sys.modules["os"].__dict__["getcwd"]()

a) is not a perfect solution because, well, in my example I've used
"os" to illustrate the problem, but really, who's to predict what
global modules might be installed on a system? And besides, isn't this
something that namespaces were supposed to solve?

b) well, that's just plain ugly.

Any other ideas?

thanks,
George

 

[Alex Martelli]

a) Not use os as my submodule name
Best.

a) is not a perfect solution because, well, in my example I've used
"os" to illustrate the problem, but really, who's to predict what
global modules might be installed on a system? And besides, isn't this

As long as your package does not use global modulenames coinciding with
the ones it contains, you're fine. You don't need to predict anything:
if your package needs global modules os, sys, and re, don't have modules
named that way in your package. Simple, innit?


Alex

 

[George P]

As long as your package does not use global modulenames coinciding with
the ones it contains, you're fine. You don't need to predict anything:
if your package needs global modules os, sys, and re, don't have modules
named that way in your package. Simple, innit?


Alex

Not all that simple. Let's say I give my submodule some fairly generic
name like utils. I would think that was fine because my module is
really mypkg.utils. But then someone out in the rest of the python
universe creates a package called utils and that eventually becomes
standard and gets installed on all systems. Then if I wanted to access
it within my module, I would have a problem.

The problem is easily overcome in my code - all I'd have to do is to
rename my module to myutils or something, but if any other code has
been written that relied on my package and imported mypkg.util, they'd
suddenly stop working with the new release that has the module renamed
to mypkg.myutil.

I'm not saying that these aren't insurmountable problems, I'm just
looking for a better solution. In C++, I would be able to refer to the
global utils in my code as ::utils, and there would be no more
problems. Does anyone know of a similar workaround in python?

George

 

[Alex Martelli]

Not all that simple. Let's say I give my submodule some fairly generic
name like utils. I would think that was fine because my module is
really mypkg.utils. But then someone out in the rest of the python
universe creates a package called utils and that eventually becomes
standard and gets installed on all systems. Then if I wanted to access
it within my module, I would have a problem.

Right: in this case, in the release of your module where you suddenly
want to use a module of the same name from elsewhere, you have to take a
little more care than just adding an 'import utils'. But there's no
guessing involved: it was your "who could guess" that made no sense.

Your module mypkg.utils which wants to import a module named simply
utils must carefully do something like:

import sys, imp

utils = sys.modules.get('utils')
if not utils: utils = imp.load_module('utils',*imp.find_module('utils'))

The problem is easily overcome in my code - all I'd have to do is to

Sure, see above.

rename my module to myutils or something, but if any other code has
been written that relied on my package and imported mypkg.util, they'd
suddenly stop working with the new release that has the module renamed
to mypkg.myutil.

If you change the published interface of your package, yup, you'll break
client code relying on the previous version of the interface. If
everybody used the preferred form "from mypkg import utils" you'd have
no trouble (just add 'import myutils as utils' to mypkg/__init__.py) but
if they use the popular form "import mypkg.utils" you'd break their
code. Which is why, to avoid risking that, I'd suggest the imp solution
sketched above, rather than running the risk of changing your own
published interface.

I'm not saying that these aren't insurmountable problems, I'm just
looking for a better solution. In C++, I would be able to refer to the
global utils in my code as ::utils, and there would be no more
problems. Does anyone know of a similar workaround in python?

Yep, http://www.python.org/peps/pep-0328.html -- but it looks like only
the multiline-import part of PEP 328 is going to make it into Python
2.4, unless I've missed something, not the absolute and relative imports
which are presumably what you're looking for.


Alex