problem with packages and path

[Daniel]

Hello,

I'm writing some unit tests for my python software which uses
packages. Here is the basic structure:

mypackage
__init__.py
module1
__init__.py
mod1.py
module2
__init__.py
mod2.py
unittests
__init__.py
alltests.py
test1.py
test2.py

within alltests.py I would expect to be able to "import
mypackage.unittests.test1". In fact within PyScripter this works as
expected. However, when I execute the code from the command line, I
get the following error:

ImportError: No module named mypackage.unittests.test1

I've read that "When importing the package, Python searches through
the directories on sys.path looking for the package subdirectory." -
http://docs.python.org/tut/node8.html#SECTION008400000000000000000

So I tried adding the following code to my alltests.py, which should
add the directory for mypackage to the path (and it does).

import os, sys
newpath = os.path.normpath( os.path.join( __file__, "../../" ))
sys.path.append(newpath)

I still get the same error. Can someone please point me in the right
direction? Thanks in advance:

Daniel

 

[Paul Boddie]

I'm writing some unit tests for my python software which uses
packages.  Here is the basic structure:

mypackage
[...]

  unittests
    __init__.py
    alltests.py
    test1.py
    test2.py

within alltests.py I would expect to be able to "import
mypackage.unittests.test1".  In fact within PyScripter this works as
expected.  However, when I execute the code from the command line, I
get the following error:

ImportError: No module named mypackage.unittests.test1

One thing to note: if you are running alltests.py directly (which you
don't mention explicitly) then even if you do so from the directory
containing the root of the package hierarchy (mypackage), it will be
the directory containing alltests.py (unittests) which will appear on
the PYTHONPATH/sys.path. Sometimes it's easy to take this behaviour
for granted when running programs.

A question: what happens if you prepend the directory containing the
root of package hierarchy to sys.path using insert (at element 0)
rather than append?

Paul

 

[Daniel]

I'm writing some unit tests for my python software which uses
packages.  Here is the basic structure:

mypackage

[...]

  unittests
    __init__.py
    alltests.py
    test1.py
    test2.py

within alltests.py I would expect to be able to "import
mypackage.unittests.test1".  In fact within PyScripter this works as
expected.  However, when I execute the code from the command line, I
get the following error:

ImportError: No module named mypackage.unittests.test1

One thing to note: if you are running alltests.py directly (which you
don't mention explicitly) then even if you do so from the directory
containing the root of the package hierarchy (mypackage), it will be
the directory containing alltests.py (unittests) which will appear on
the PYTHONPATH/sys.path. Sometimes it's easy to take this behaviour
for granted when running programs.

A question: what happens if you prepend the directory containing the
root of package hierarchy to sys.path using insert (at element 0)
rather than append?

Paul

I changed it to 'sys.path.insert(0, newpath)', as you suggest, but it
doesn't fix the error.

I did notice that the directory containing the file alltests.py ends
up in the path. I've also noticed that (in WinXP at least) the
sys.path modification inserts a '..' in the path, rather than the full
path to 'mypackage'. I'm not sure if this makes a difference, but it
seems like I should be able to add either '..' or 'path/to/mypackage'
to the path and have it find my packages.

Any more suggestions?

 

[Daniel]

I'm writing some unit tests for my python software which uses
packages.  Here is the basic structure:

mypackage

[...]

  unittests
    __init__.py
    alltests.py
    test1.py
    test2.py

within alltests.py I would expect to be able to "import
mypackage.unittests.test1".  In fact within PyScripter this works as
expected.  However, when I execute the code from the command line, I
get the following error:

ImportError: No module named mypackage.unittests.test1

One thing to note: if you are running alltests.py directly (which you
don't mention explicitly) then even if you do so from the directory
containing the root of the package hierarchy (mypackage), it will be
the directory containing alltests.py (unittests) which will appear on
the PYTHONPATH/sys.path. Sometimes it's easy to take this behaviour
for granted when running programs.

A question: what happens if you prepend the directory containing the
root of package hierarchy to sys.path using insert (at element 0)
rather than append?

Paul

I changed it to 'sys.path.insert(0, newpath)', as you suggest, but it
doesn't fix the error.

I did notice that the directory containing the file alltests.py ends
up in the path. I've also noticed that (in WinXP at least) the
sys.path modification inserts a '..' in the path, rather than the full
path to 'mypackage'. I'm not sure if this makes a difference, but it
seems like I should be able to add either '..' or 'path/to/mypackage'
to the path and have it find my packages.

Any more suggestions?

 

[Marco Bizzarri]

Hello,

I'm writing some unit tests for my python software which uses
packages. Here is the basic structure:

mypackage
__init__.py
module1
__init__.py
mod1.py
module2
__init__.py
mod2.py
unittests
__init__.py
alltests.py
test1.py
test2.py

within alltests.py I would expect to be able to "import
mypackage.unittests.test1". In fact within PyScripter this works as
expected. However, when I execute the code from the command line, I
get the following error:

ImportError: No module named mypackage.unittests.test1

1) What is the command you're using to run the alltest.py module?

2) what is the result of:
- python -c "import mypackage"
- python -c "import mypackage.unittests"

Regards
Marco

 

[Daniel]

1) What is the command you're using to run the alltest.py module?

2) what is the result of:
   - python -c "import mypackage"
  - python -c "import mypackage.unittests"

Regards
Marco

I have tried running both commands above from the mypackage directory
and unittests directory. I get the following response universtally.

C:\mypackage>dir
Volume in drive C is Default

Directory of C:\mypackage

08/29/2008 11:04 AM <DIR> .
08/29/2008 11:04 AM <DIR> ..
08/29/2008 11:05 AM <DIR> module1
08/29/2008 11:05 AM <DIR> module2
08/29/2008 11:06 AM <DIR> unittests
08/29/2008 11:04 AM 0 __init__.py
1 File(s) 0 bytes
5 Dir(s) 55,402,070,016 bytes free

C:\mypackage>dir unittests
Volume in drive C is Default

Directory of C:\mypackage\unittests

08/29/2008 11:06 AM <DIR> .
08/29/2008 11:06 AM <DIR> ..
08/29/2008 11:05 AM 0 alltests.py
08/29/2008 11:05 AM 0 test1.py
08/29/2008 11:05 AM 0 test2.py
08/29/2008 11:04 AM 0 __init__.py
4 File(s) 0 bytes
2 Dir(s) 55,401,988,096 bytes free

C:\mypackage>python -c "import mypackage"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named mypackage

C:\mypackage>python -c "import mypackage.unittests"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named mypackage.unittests

 

[Paul Boddie]

I have tried running both commands above from the mypackage directory
and unittests directory. I get the following response universtally.

C:\mypackage>dir
Volume in drive C is Default

Directory of C:\mypackage

08/29/2008 11:04 AM <DIR> .
08/29/2008 11:04 AM <DIR> ..
08/29/2008 11:05 AM <DIR> module1
08/29/2008 11:05 AM <DIR> module2
08/29/2008 11:06 AM <DIR> unittests
08/29/2008 11:04 AM 0 __init__.py
1 File(s) 0 bytes
5 Dir(s) 55,402,070,016 bytes free

If you run unittests\alltests.py from here, sys.path will initially
refer to this directory (C:\mypackage) and to anything else set up in
your environment. The extra path modification that you put in
alltests.py will append (or insert) the following...

os.path.normpath( os.path.join( __file__, "../../" ))
-> os.path.normpath( os.path.join( "C\\:mypackage\\unittests\
\alltests.py", "../../" ))
-> os.path.normpath( "C\\:mypackage" )
-> "C\\:mypackage"

Since the import statement then tries to access mypackage, which isn't
itself in the C:\mypackage directory - it's in the C:\ directory - the
import fails. The previous, automatically added entry in sys.path is
duplicated by the path modifications shown above, so that won't help
here, either.

C:\mypackage>dir unittests
Volume in drive C is Default

Directory of C:\mypackage\unittests

08/29/2008 11:06 AM <DIR> .
08/29/2008 11:06 AM <DIR> ..
08/29/2008 11:05 AM 0 alltests.py
08/29/2008 11:05 AM 0 test1.py
08/29/2008 11:05 AM 0 test2.py
08/29/2008 11:04 AM 0 __init__.py
4 File(s) 0 bytes
2 Dir(s) 55,401,988,096 bytes free

Here, sys.path should refer to C:\mypackage\unittests somewhere, which
won't help Python find mypackage. The path modification in alltests.py
should perform as above, producing C:\mypackage which won't help.

I guess the easiest thing to do is to change the path modification
code to use "../../.." which should produce a reference to C:\ instead
of C:\mypackage. The confusion with this particular piece of the code
is the way ".." doesn't cause os.path.join to operate on the directory
C:\mypackage\unittests but instead on the file C:\mypackage\unittests
\alltests.py - that's almost to be expected, however, since
os.path.join doesn't really know that its first argument is really a
file or that ".." is being combined with a file.

What I tend to do, especially with __file__, is to first use
os.path.split to remove the filename - strictly speaking, it's the
leafname - and then to apply the kind of modifications mentioned
above, although there's probably no real difference or, indeed, any
advantage with either approach.

Paul

 

[Daniel]

If you run unittests\alltests.py from here, sys.pathwill initially
refer to this directory (C:\mypackage) and to anything else set up in
your environment. The extrapathmodification that you put in
alltests.py will append (or insert) the following...

   os.path.normpath( os.path.join( __file__, "../../" ))
-> os.path.normpath( os.path.join( "C\\:mypackage\\unittests\
\alltests.py", "../../" ))
-> os.path.normpath( "C\\:mypackage" )
-> "C\\:mypackage"

Since the import statement then tries to access mypackage, which isn't
itself in the C:\mypackage directory - it's in the C:\ directory - the
import fails. The previous, automatically added entry in sys.pathis
duplicated by thepathmodifications shown above, so that won't help
here, either.




Here, sys.pathshould refer to C:\mypackage\unittests somewhere, which
won't help Python find mypackage. Thepathmodification in alltests.py
should perform as above, producing C:\mypackage which won't help.

I guess the easiest thing to do is to change thepathmodification
code to use "../../.." which should produce a reference to C:\ instead
of C:\mypackage. The confusion with this particular piece of the code
is the way ".." doesn't cause os.path.join to operate on the directory
C:\mypackage\unittests but instead on the file C:\mypackage\unittests
\alltests.py - that's almost to be expected, however, since
os.path.join doesn't really know that its first argument is really a
file or that ".." is being combined with a file.

What I tend to do, especially with __file__, is to first use
os.path.split to remove the filename - strictly speaking, it's the
leafname - and then to apply the kind of modifications mentioned
above, although there's probably no real difference or, indeed, any
advantage with either approach.

Paul

Thanks Paul. That did the trick.

Daniel