Problem with python 3.2 and circular imports

[Frank Millman]

Hi all

I thought I was getting the hang of circular imports, but after upgrading to
python 3.2 I am stumped again. I know some people think that circular
imports are always bad, but others suggest that, provided you understand the
potential problems, they can be acceptable.

Assume the following structure -

main.py
/pkg
__init__.py
mod1.py
mod2.py

main.py
from pkg import mod1

mod1.py
import mod2

mod2.py
import mod1

Using python 2.6, running main.py works.

After running 2to3.py on the above directory, mod1.py was changed to 'from .
import mod2' and mod2.py was changed to 'from . import mod1'.

With python 3.2, it now fails with the following traceback -

Traceback (most recent call last):
File "main.py", line 1, in <module>
from pkg import mod1
File "pkg\mod1.py", line 1, in <module>
from . import mod2
File "pkg\mod2.py", line 1, in <module>
from . import mod1
ImportError: cannot import name mod1

I have read the relevant peps and various other material, understood them to
a certain extent, tried several alternatives, but could not find a solution.

I have found a hack that works, but I don't like it very much. I added the
following to '__init__.py' -
import os
import sys
sys.path.insert(0, os.path.dirname(__file__))

This adds the package name into the search path.

Then I changed mod1.py and mod2.py back to the original 'import mod2' and
'import mod1'.

It works, but it seems to be defeating the purpose of PEP 328, which I
thought was an improvement.

Any comments or suggestions will be appreciated.

Frank Millman

 

[Frank Millman]

What are you expecting the result to be?

Simply that mod1 can refer to objects in mod2, and mod2 can refer to objects
in mod1.

If it's about sharing objects between the modules, why not break the
circular dependency: factor out the common code to a module that both
the others import?


Special cases aren't special enough to break the rules. If you think you
have a practical reason to do so, it would be best to make it explicit
when asking for help about this.

I am trying to understand what 'the rule' is. Your advice above suggests
that you are one of those who recommend that circular imports are best
avoided altogether. In an ideal world I would agree. However, the fact is
that, no doubt due to a mental block I have, I do find myself in this
situation from time to time, and I have not seen anything in the
documentation or other literature that says it is absolutely wrong.
Therefore, while I do try to avoid circular imports where possible, I would
also like to know how to manage it in situations where I don't see a simple
alternative.

From everything I have read about how the import mechanism works, I don't

understand *why* the above construct fails. That is actually what I am
asking for help with.

Thanks

Frank

 

[Steven D'Aprano]

Assume the following structure -

main.py
/pkg
__init__.py
mod1.py
mod2.py

main.py
from pkg import mod1

mod1.py
import mod2

mod2.py
import mod1

If you change the "import mod*" lines to "import pkg.mod*" it works for
me in Python 3.1 and 3.2.

According to my understand of PEP 328, "from . import mod*" should work,
but I agree with you that it doesn't.

If you get rid of the circular import, it does work. So I suspect a bug.

 

[Frank Millman]

If you change the "import mod*" lines to "import pkg.mod*" it works for
me in Python 3.1 and 3.2.

According to my understand of PEP 328, "from . import mod*" should work,
but I agree with you that it doesn't.

If you get rid of the circular import, it does work. So I suspect a bug.

Thanks, Steven.

I confirm that 'import pkg.mod* works. Unfortunately I am using sub-packages
as well, which means that to refer to an object in the sub-package I need to
use w.x.y.z every time, which gets to be a lot of typing! I will stick to my
hack of putting the package name in sys.path for now, unless someone comes
up with a better idea.

Frank

 

[Frank Millman]

On February 28 2011 Rafael Durán Castañeda wrote

I'm stil totally stuck with relative imports, i' ve tried the example tree
from PEP 328 without any result:

package/
__init__.py
subpackage1/
__init__.py
moduleX.py
moduleY.py
subpackage2/
__init__.py
moduleZ.py
moduleA.py

Assuming that the current file is either moduleX.py or
subpackage1/__init__.py, following are correct usages of the new
syntax:

from .moduleY import spam
from .moduleY import spam as ham
from . import moduleY
from ..subpackage1 import moduleY
from ..subpackage2.moduleZ import eggs
from ..moduleA import foo
from ...package import bar
from ...sys import path

I always get:

Traceback (most recent call last):
File "moduleY.py", line 1, in <module>
from ..moduleA import a
ValueError: Attempted relative import in non-package

Hi Rafael

I only just noticed your message now. For some reason it did not appear on
google.groups.

I just tried it and it worked for me. This is what I did.

I set up the above structure. I added a 'def spam()' to moduleY, 'def
eggs()' to moduleZ, and 'def foo()' to moduleA.

I pasted all of the 'from ... import ...' lines above into moduleX.

In the same directory that contains the subdirectory 'package' I create a
test script containing the following line -
import package.subpackage1.moduleX

To begin with it failed on the last two lines starting with 'from ...' [how
do you indicate three dots followed by etc etc, which would normally be
indicated by three dots!]. The traceback said 'Attempted relative import
beyond toplevel package'.

I did not want to investigate too deeply so I just commented those lines
out, and now it runs with no errors. I even put a couple of print statements
(or must I call them print functions now) into the modules being imported,
and the messages do appear, so the modules are being imported.

HTH

Frank Millman

 

[Frank Millman]

Thank you for your answer Frank, I think I've found the problem. I was
calling modules from inside subpackages, and I need to use them from
outside, so I have package in PYTHONPATH. is that correct? But now I have
another question: Can I execute an script inside subpackage1 importig
modules from subpackage2?

Hi Rafael

I am no expert, so I cannot answer you directly.

In my case, my program started as a single module.

As it grew, I started to split some parts off and store them in separate
modules in the same directory. I placed 'import' statements in the main
module, and it worked.

Then I found the need for some modules to refer to objects in other modules,
so I needed 'import' statements within the modules. I found myself hitting
problems with circular imports from time to time, but with some help from
this group and re-reading the docs I got over this hurdle.

Only recently has my project got big enough to start thinking about
packages. It adds complexity, but by reading the docs again, and thinking
carefully about the structure, I have so far managed to handle the problems
that occur.

I found the 'modules' chapter in the tutorial a good place to start. Once
you have fully grasped the contents, PEP 328 is a good resource for
understanding what has changed in python 3.x.

BTW, this group prefers 'bottom posting'. You will see that I have placed my
response below yours. If you want to reply to this, please follow the same
practice.

Hope this helps to get you started.

Frank