relative imports improve program organization... suggestions?

[DG]

Alright, I have searched and searched and read many conversations on
the topic of relative and absolute imports and am still not getting
the whole thing through my skull.

Highlights of what I've read:
http://mail.python.org/pipermail/python-list/2007-January/422973.html
http://groups.google.com/group/comp...f2?lnk=gst&q=absolute+import#8751c82cfe1ca3f2
http://www.python.org/dev/peps/pep-0328/
http://docs.python.org/whatsnew/pep-328.html

So my problem and argument:
I want to create a package organized as the following:
pckg/
__init__.py
main.py
moduleA/
__init__.py
A_base.py
A1/
__init__.py
A_inherit1.py
other_A1_files...
A2/
__init__.py
A_inherit2.py
other_A2_files...
moduleB/
...
Explanation:
The main program is located in main.py and it implements the different
modules (A, B). Within the modules the basic organization is; the
base class for all different types of A is directly within the moduleA
directory. All of the different inherited classes of A are within
their own subdirectory with a mess of their own files. This is done
so that a new subclass of A can be added/removed by just adding/
removing the subdirectory and each of these subclasses may have their
own maintainer, but they should all inherit from A_base.py

If I am developing the A1 directory, I want to be able to test
A_inherit1.py by using 'if __name__ == "__main__"' within the
A_inherit1.py file and by typing 'python A_inherit1.py' on the command
line. I prefer this simply to keep all unit tests within the same
directory and same file as the inherited class.

My Problem:
A_inherit1.py has the line:
'from ..A_base import A_Base_Class'
so that I can later declare the inherited class as such:
'A1_Inherited_Class(A_Base_Class):'

*BUT* I get the 'attempted relative import in non-package' error even
when I try the
'from __future__ import absolute_import' command.
I would prefer to be able to test the file without adding anything to
the PYTHONPATH, like I said by using the name == main trick.

So could someone explain to me what the rationale behind not allowing
parent directory relative imports is? And possibly what I can do to
get around it? (I really don't like messing with the sys.path for
something like this)

Thanks,
Danny G

 

[DG]

http://www.velocityreviews.com/forums/t502905-relative-import-broken.html
also gets a bit into this issue, and explains it a bit more
thoroughly.

Still though, I don't why there isn't a 'clean' way to be able to
inherit from a file in the parent directory but still be able to test
the derived class in its own directory without modifying the
PYTHONPATH.

 

[Carl Banks]

My Problem:
A_inherit1.py has the line:
     'from ..A_base import A_Base_Class'
so that I can later declare the inherited class as such:
     'A1_Inherited_Class(A_Base_Class):'

*BUT* I get the 'attempted relative import in non-package' error even
when I try the
'from __future__ import absolute_import' command.
I would prefer to be able to test the file without adding anything to
the PYTHONPATH, like I said by using the name == main trick.

So could someone explain to me what the rationale behind not allowing
parent directory relative imports is?  And possibly what I can do to
get around it?  (I really don't like messing with the sys.path for
something like this)

The problem you are seeing is because when you invoke the script via
the command line it is considered an isolated module called __main__.
In fact if another module were to reimport it it would actually create
a new module with the module's intended name. Not one of the nicer
aspects of Python.

The rationale for doing it this way is mostly historical reasons:
before packages existed it wasn't so confusing. Python 3 (and, I
think, 2.6) has some features to help cope with it better.

Until then, I would say the most straightforward way to workaround
this is to have a helper script. Instead of running "python
A_inherit1.py", you'd run "unittest A_inherit1.py", where unittest is
a Python script that adds the project root to the sys.path, constructs
the actual module name, imports it, and runs a function called
test(). You would have to define test() and use it in lieu of the if
__name__ == '__main__' thing.

Very simple, unchecked, bug-filled example of what the unittest script
would look like:

---------------------
#!/usr/bin/python

import os
import sys

# Get module name

filename = sys.argv[1]
modname = os.path.splitext(filename)[0]

# Starting from cwd, find the project root

pathname = os.getcwd()
pkgpath = []
while True:
dirname,basename = os.path.split(pathname)
if dirname == 'pckg':
break
pkgpath.append(basename)
pathname = dirname

# Insert the project root into sys.path

sys.path.insert(0,pathname)

# Determine the package

pkgpath.reverse()
pkgname = ".".join(pkgpath)

# Import the module

mod = __import__(modpath,None,None,modname)

# Run the unit tests

mod.test()
---------------------

Yes, this does manipulate sys.path. It won't kill you. At least this
way the magic behavior is kept out of the files themselves. As an
added bonus, it allows the unit tests to be run from an interactive
prompt.


Carl Banks

 

[castironpi]

Alright, I have searched and searched and read many conversations on
the topic of relative and absolute imports and am still not getting
the whole thing through my skull.

Highlights of what I've read:http://mail.python.org/pipermail/py.../http://docs.python.org/whatsnew/pep-328.html

So my problem and argument:
I want to create a package organized as the following:
pckg/
     __init__.py
     main.py
     moduleA/
          __init__.py
          A_base.py
          A1/
               __init__.py
               A_inherit1.py
               other_A1_files...
          A2/
               __init__.py
               A_inherit2.py
               other_A2_files...
     moduleB/
          ...
Explanation:
The main program is located in main.py and it implements the different
modules (A, B).  Within the modules the basic organization is; the
base class for all different types of A is directly within the moduleA
directory.  All of the different inherited classes of A are within
their own subdirectory with a mess of their own files.  This is done
so that a new subclass of A can be added/removed by just adding/
removing the subdirectory and each of these subclasses may have their
own maintainer, but they should all inherit from A_base.py

If I am developing the A1 directory, I want to be able to test
A_inherit1.py by using 'if __name__ == "__main__"' within the
A_inherit1.py file and by typing 'python A_inherit1.py' on the command
line.  I prefer this simply to keep all unit tests within the same
directory and same file as the inherited class.

My Problem:
A_inherit1.py has the line:
     'from ..A_base import A_Base_Class'
so that I can later declare the inherited class as such:
     'A1_Inherited_Class(A_Base_Class):'

*BUT* I get the 'attempted relative import in non-package' error even
when I try the
'from __future__ import absolute_import' command.
I would prefer to be able to test the file without adding anything to
the PYTHONPATH, like I said by using the name == main trick.

So could someone explain to me what the rationale behind not allowing
parent directory relative imports is?  And possibly what I can do to
get around it?  (I really don't like messing with the sys.path for
something like this)

Thanks,
Danny G

Didn't read the whole thing, but would imp.load_source( name,
relative_path ) help you at all?

 

[DG]

Carl: Your solution is kind of what I was leaning towards after a bit
of thinking. Since I have to have the modules each have their own
detect() method, then it wouldn't be too hard to have their own test()
method to put them through their paces.

Catrironpi: I will look into this as it might help just get around the
problem, but in my eyes it's still not 'clean'. Simply because people
(including me) would expect the regular 'import' statement over the
method I would think.

Thanks to all who responded.