Weird import failure with "nosetests --processes=1"

R

Roy Smith

I've got a minimal test script:

-----------------------------
$ cat test_foo.py
import pyza.models
print pyza.models

def test_foo():
pass
-----------------------------

pyza.models is a package. Under normal conditions, I can import it fine:

$ python
Python 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.<module 'pyza.models' from
'/home/roy/deploy/current/pyza/models/__init__.pyc'>
But when I run nosetests in parallel mode, the import fails in a way
which has me baffled. What's going on here?

$ nosetests --processes=1 -s test_foo:test_foo
<module 'pyza.models' from
'/home/roy/deploy/current/pyza/models/__init__.pyc'>
E
======================================================================
ERROR: Failure: AttributeError ('module' object has no attribute
'models')
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/roy/production/python/local/lib/python2.7/site-packages/nose/loade
r.py", line 390, in loadTestsFromName
addr.filename, addr.module)
File
"/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
ter.py", line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File
"/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
ter.py", line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/roy/songza/pyza/djapi/test_foo.py", line 2, in <module>
print pyza.models
AttributeError: 'module' object has no attribute 'models'
 
H

Hans Mulder

I've got a minimal test script:

-----------------------------
$ cat test_foo.py
import pyza.models
print pyza.models

def test_foo():
pass
-----------------------------

pyza.models is a package. Under normal conditions, I can import it fine:

$ python
Python 2.7.3 (default, Aug 1 2012, 05:14:39)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.<module 'pyza.models' from
'/home/roy/deploy/current/pyza/models/__init__.pyc'>
But when I run nosetests in parallel mode, the import fails in a way
which has me baffled. What's going on here?

$ nosetests --processes=1 -s test_foo:test_foo
<module 'pyza.models' from
'/home/roy/deploy/current/pyza/models/__init__.pyc'>
E
======================================================================
ERROR: Failure: AttributeError ('module' object has no attribute
'models')
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/roy/production/python/local/lib/python2.7/site-packages/nose/loade
r.py", line 390, in loadTestsFromName
addr.filename, addr.module)
File
"/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
ter.py", line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File
"/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
ter.py", line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/roy/songza/pyza/djapi/test_foo.py", line 2, in <module>
print pyza.models
AttributeError: 'module' object has no attribute 'models'


That is baffling indeed. It looks like nose is adding some
directory to sys.path, which contains a module pyza.py instead
of a package.

One thing you could try, is changing line 2 of you script to just

print pyza

, to see if pyza is being loaded from
/home/roy/deploy/current/pyza/__init__.pyc, as you'd expect. If it
isn't, you'll be
one step closer to a solution.

Another idea might be to delete all *.pyc files below
/home/roy/deploy/current/ and /home/roy/songza/pyza/djapi/.
That would help if a .pyc file is somehow out of sync with
the corresponding .py file. That's not supposed to happen,
but if it does, you can get weird results.

One last idea: put these lines at the top of test_foo.py

import pdb
pdb.set_trace()

Running under nosetest should then drop you in the debugger.
Single step into the next statement ("import pyza.models") to
see where this gets you. It should import pyza, and then
pyza.models. Add some print statements to the __init__.py
files of those packages, so that there's somewhere for pdb
to stop and prompt.


Hope this helps,

-- HansM
 
R

Roy Smith

Hans Mulder said:
That is baffling indeed. It looks like nose is adding some
directory to sys.path, which contains a module pyza.py instead
of a package.

We finally figured it out. As it turns out, that's pretty close.
Another idea might be to delete all *.pyc files below
/home/roy/deploy/current/ and /home/roy/songza/

I've got

sys.path.insert(0, '/home/roy/deploy/current')

in my virtualenv's usercustomize.py. Also, though historical accident,
I had (but don't any longer)

PYTHONPATH=/home/roy/songza

in my .profile. And /home/roy/deploy/current is a symlink to
/home/roy/songza! So, I had two different paths to the same directory.
Why this only showed up as a problem when running under nose in
multiprocess mode, I have no clue. And how it ended up thinking pyza
was a module instead of a package, I also have no idea.

Crazy.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top