Cyclic imports

[James]

Hi all,
I'm looking for some advice dealing with cyclic, cross-package
imports.

I've created the following demo file structure:
../a/__init__.py
../a/a.py
../b/__init__.py
../b/b.py
../main.py

a.py imports a class from b.py and vice versa, and main.py imports
them both.

However, because a.py has not been completely read by the time it gets
to the:
from b import b
line, the
from a import a
line in b.py fails with an import error (stack trace at the end).

I hope that's clear: basically, because of the cyclic import, I see
incomplete modules which can't be imported.

Can I have this module hierarchy in some other way?

Thanks,
James

Stack trace from example:
File "main.py", line 1, in <module>
from a import a
File "/Users/james/tmp/python/a/a.py", line 1, in <module>
from b import b
File "/Users/james/tmp/python/b/b.py", line 1, in <module>
from a import a
ImportError: cannot import name a

 

[Dan Bishop]

Hi all,
I'm looking for some advice dealing with cyclic, cross-package
imports.

I've created the following demo file structure:
./a/__init__.py
./a/a.py
./b/__init__.py
./b/b.py
./main.py

a.py imports a class from b.py and vice versa, and main.py imports
them both.

However, because a.py has not been completely read by the time it gets
to the:
from b import b
line, the
from a import a
line in b.py fails with an import error (stack trace at the end).

I hope that's clear: basically, because of the cyclic import, I see
incomplete modules which can't be imported.

Can I have this module hierarchy in some other way?

# a.py
import b
# refer to b.b

# b.py
import a
# refer to a.a

 

[James]

# a.py

import b
# refer to b.b

# b.py
import a
# refer to a.a

Thanks Dan, but that still doesn't work for me I'm afraid...

I renamed the modules avoid name overloading -- a.py is now:
import b

class A():
print('b.b_mod:', b.b_mod)

b is now defined, but b.b_mod isn't:
File "main.py", line 1, in <module>
from a import a_mod
File "/Users/james/tmp/python/a/a_mod.py", line 3, in <module>
class A():
File "/Users/james/tmp/python/a/a_mod.py", line 4, in A
print('b.b_mod:', b.b_mod)
AttributeError: 'module' object has no attribute 'b_mod'

I must be missing something here - I've tried adding the modules to
__all__ in __init__.py but that didn't help either.

James

 

[Carl Banks]

Thanks Dan, but that still doesn't work for me I'm afraid...

I renamed the modules avoid name overloading -- a.py is now:
import b

class A():
print('b.b_mod:', b.b_mod)

b is now defined, but b.b_mod isn't:
File "main.py", line 1, in <module>
from a import a_mod
File "/Users/james/tmp/python/a/a_mod.py", line 3, in <module>
class A():
File "/Users/james/tmp/python/a/a_mod.py", line 4, in A
print('b.b_mod:', b.b_mod)
AttributeError: 'module' object has no attribute 'b_mod'

I must be missing something here - I've tried adding the modules to
__all__ in __init__.py but that didn't help either.

The above print statement runs at module import time, so you're seeing
something that might not be an issue in functioning code.

For instance, if you rewrite A like the following:

class A(object):
def print_b(self):
print('b.b_mod:', b.b_mod)

If you call the print_b method from outside the module (say, from
main.py), it should work.


What lessons did we learn here?

Be aware of the difference between code that runs at module import
time, and code that runs after the module is imported.

In code that runs after the module has been imported (basically
anything defined in a function that isn't called by code that runs at
module time), you can expect the variables defined in the imported
module to be available.

In code that runs at module import time, there is no such guarantee,
so you have to arrange your modules so that code that depends on
another module is run after the dependent module. (One common time
where this happens is when subclassing a class from another module:
the module with the base class needs to run first.)

If you have circular imports involved, making sure the modules import
in a certain order can be quite hairy. (Faced with this problem, I
found it necessary to write an import hook to pre-import certain
modules.)


Carl Banks

 

[James]

In code that runs after the module has been imported (basically

anything defined in a function that isn't called by code that runs at
module time), you can expect the variables defined in the imported
module to be available.

If you have circular imports involved, making sure the modules import
in a certain order can be quite hairy.  (Faced with this problem, I
found it necessary to write an import hook to pre-import certain
modules.)

Ok, thanks for all the advice: I now have the class architecture I was
looking for, by using much less specific imports, as Dan originally
suggested.

Re-factoring the use of partially loaded modules into functions wasn't
an option for me, unfortunately, as the classes were being used as
super classes or class variables. So, the code is a bit more verbose
than before, but works!

James