Weird import failure with "nosetests --processes=1"

Discussion in 'Python' started by Roy Smith, Nov 29, 2012.

  1. Roy Smith

    Roy Smith Guest

    I've got a minimal test script:

    -----------------------------
    $ cat test_foo.py
    import pyza.models
    print pyza.models

    def test_foo():
    pass
    -----------------------------

    pyza.models is a package. Under normal conditions, I can import it fine:

    $ python
    Python 2.7.3 (default, Aug 1 2012, 05:14:39)
    [GCC 4.6.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import pyza.models
    >>> print pyza.models

    <module 'pyza.models' from
    '/home/roy/deploy/current/pyza/models/__init__.pyc'>
    >>>


    But when I run nosetests in parallel mode, the import fails in a way
    which has me baffled. What's going on here?

    $ nosetests --processes=1 -s test_foo:test_foo
    <module 'pyza.models' from
    '/home/roy/deploy/current/pyza/models/__init__.pyc'>
    E
    ======================================================================
    ERROR: Failure: AttributeError ('module' object has no attribute
    'models')
    ----------------------------------------------------------------------
    Traceback (most recent call last):
    File
    "/home/roy/production/python/local/lib/python2.7/site-packages/nose/loade
    r.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
    File
    "/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
    ter.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
    File
    "/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
    ter.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
    File "/home/roy/songza/pyza/djapi/test_foo.py", line 2, in <module>
    print pyza.models
    AttributeError: 'module' object has no attribute 'models'

    ----------------------------------------------------------------------
    Ran 1 test in 0.107s

    FAILED (errors=1)
    Roy Smith, Nov 29, 2012
    #1
    1. Advertising

  2. Roy Smith

    Hans Mulder Guest

    On 29/11/12 04:13:57, Roy Smith wrote:
    > I've got a minimal test script:
    >
    > -----------------------------
    > $ cat test_foo.py
    > import pyza.models
    > print pyza.models
    >
    > def test_foo():
    > pass
    > -----------------------------
    >
    > pyza.models is a package. Under normal conditions, I can import it fine:
    >
    > $ python
    > Python 2.7.3 (default, Aug 1 2012, 05:14:39)
    > [GCC 4.6.3] on linux2
    > Type "help", "copyright", "credits" or "license" for more information.
    >>>> import pyza.models
    >>>> print pyza.models

    > <module 'pyza.models' from
    > '/home/roy/deploy/current/pyza/models/__init__.pyc'>
    >>>>

    >
    > But when I run nosetests in parallel mode, the import fails in a way
    > which has me baffled. What's going on here?
    >
    > $ nosetests --processes=1 -s test_foo:test_foo
    > <module 'pyza.models' from
    > '/home/roy/deploy/current/pyza/models/__init__.pyc'>
    > E
    > ======================================================================
    > ERROR: Failure: AttributeError ('module' object has no attribute
    > 'models')
    > ----------------------------------------------------------------------
    > Traceback (most recent call last):
    > File
    > "/home/roy/production/python/local/lib/python2.7/site-packages/nose/loade
    > r.py", line 390, in loadTestsFromName
    > addr.filename, addr.module)
    > File
    > "/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
    > ter.py", line 39, in importFromPath
    > return self.importFromDir(dir_path, fqname)
    > File
    > "/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
    > ter.py", line 86, in importFromDir
    > mod = load_module(part_fqname, fh, filename, desc)
    > File "/home/roy/songza/pyza/djapi/test_foo.py", line 2, in <module>
    > print pyza.models
    > AttributeError: 'module' object has no attribute 'models'
    >
    > ----------------------------------------------------------------------
    > Ran 1 test in 0.107s
    >
    > FAILED (errors=1)



    That is baffling indeed. It looks like nose is adding some
    directory to sys.path, which contains a module pyza.py instead
    of a package.

    One thing you could try, is changing line 2 of you script to just

    print pyza

    , to see if pyza is being loaded from
    /home/roy/deploy/current/pyza/__init__.pyc, as you'd expect. If it
    isn't, you'll be
    one step closer to a solution.

    Another idea might be to delete all *.pyc files below
    /home/roy/deploy/current/ and /home/roy/songza/pyza/djapi/.
    That would help if a .pyc file is somehow out of sync with
    the corresponding .py file. That's not supposed to happen,
    but if it does, you can get weird results.

    One last idea: put these lines at the top of test_foo.py

    import pdb
    pdb.set_trace()

    Running under nosetest should then drop you in the debugger.
    Single step into the next statement ("import pyza.models") to
    see where this gets you. It should import pyza, and then
    pyza.models. Add some print statements to the __init__.py
    files of those packages, so that there's somewhere for pdb
    to stop and prompt.


    Hope this helps,

    -- HansM
    Hans Mulder, Nov 29, 2012
    #2
    1. Advertising

  3. Roy Smith

    Roy Smith Guest

    In article <50b78e26$0$6945$4all.nl>,
    Hans Mulder <> wrote:

    > That is baffling indeed. It looks like nose is adding some
    > directory to sys.path, which contains a module pyza.py instead
    > of a package.


    We finally figured it out. As it turns out, that's pretty close.

    > Another idea might be to delete all *.pyc files below
    > /home/roy/deploy/current/ and /home/roy/songza/


    I've got

    sys.path.insert(0, '/home/roy/deploy/current')

    in my virtualenv's usercustomize.py. Also, though historical accident,
    I had (but don't any longer)

    PYTHONPATH=/home/roy/songza

    in my .profile. And /home/roy/deploy/current is a symlink to
    /home/roy/songza! So, I had two different paths to the same directory.
    Why this only showed up as a problem when running under nose in
    multiprocess mode, I have no clue. And how it ended up thinking pyza
    was a module instead of a package, I also have no idea.

    Crazy.
    Roy Smith, Nov 30, 2012
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ilias Lazaridis
    Replies:
    0
    Views:
    552
    Ilias Lazaridis
    Feb 1, 2005
  2. Replies:
    0
    Views:
    453
  3. dorayme
    Replies:
    1
    Views:
    602
    richard
    Jan 21, 2011
  4. richard
    Replies:
    0
    Views:
    571
    richard
    Jan 21, 2011
  5. botfood

    weird failure with Net::FTP

    botfood, Jan 1, 2005, in forum: Perl Misc
    Replies:
    11
    Views:
    177
    botfood
    Jan 4, 2005
Loading...

Share This Page