order of importing modules

[Catherine Moroney]

In what order does python import modules on a Linux system? I have a
package that is both installed in /usr/lib64/python2.5/site-packages,
and a newer version of the same module in a working directory.

I want to import the version from the working directory, but when I
print module.__file__ in the interpreter after importing the module,
I get the version that's in site-packages.

I've played with the PYTHONPATH environmental variable by setting it
to just the path of the working directory, but when I import the module
I still pick up the version in site-packages.

/usr/lib64 is in my PATH variable, but doesn't appear anywhere else. I
don't want to remove /usr/lib64 from my PATH because that will break
a lot of stuff.

Can I force python to import from my PYTHONPATH first, before looking
in the system directory?

Catherine

 

[Dan Stromberg]

In what order does python import modules on a Linux system?  I have a
package that is both installed in /usr/lib64/python2.5/site-packages,
and a newer version of the same module in a working directory.

I want to import the version from the working directory, but when I
print module.__file__ in the interpreter after importing the module,
I get the version that's in site-packages.

I've played with the PYTHONPATH environmental variable by setting it
to just the path of the working directory, but when I import the module
I still pick up the version in site-packages.

/usr/lib64 is in my PATH variable, but doesn't appear anywhere else.  I
don't want to remove /usr/lib64 from my PATH because that will break
a lot of stuff.

Can I force python to import from my PYTHONPATH first, before looking
in the system directory?

Catherine

Please import sys and inspect sys.path; this defines the search path
for imports.

By looking at sys.path, you can see where in the search order your
$PYTHONPATH is going.

It might actually be better to give your script a command line option
that says "Throw the following directory at the beginning of
sys.path".

 

[Catherine Moroney]

I've looked at my sys.path variable and I see that it has
a whole bunch of site-package directories, followed by the
contents of my $PYTHONPATH variable, followed by a list of
misc site-package variables (see below).

I've verified that if I manually reverse the order of sys.path
I can then import the proper version of the module that I want.
But this is not a permanent solution for me as this will mess up
other people are who working with the same code.

But, I'm curious as to where the first bunch of 'site-package'
entries come from. The
/usr/lib64/python2.5/site-packages/pyhdfeos-1.0_r57_58-py2.5-linux-x86_64.egg
is not present in any of my environmental variables yet it shows up
as one of the first entries in sys.path.

A colleague of mine is running on the same system as I am, and he
does not have the problem of the
/usr/lib64/python2.5/site-packages/pyhdfeos-1.0_r57_58-py2.5-linux-x86_64.egg
variable showing up as one of the first entries in sys.path.

Thanks for the education,

Catherine

 

[Catherine Moroney]

I've looked at my sys.path variable and I see that it has
a whole bunch of site-package directories, followed by the
contents of my $PYTHONPATH variable, followed by a list of
misc site-package variables (see below).

I've verified that if I manually reverse the order of sys.path
I can then import the proper version of the module that I want.
But this is not a permanent solution for me as this will mess up
other people are who working with the same code.

But, I'm curious as to where the first bunch of 'site-package'
entries come from. The
/usr/lib64/python2.5/site-packages/pyhdfeos-1.0_r57_58-py2.5-linux-x86_64.egg
is not present in any of my environmental variables yet it shows up
as one of the first entries in sys.path.

A colleague of mine is running on the same system as I am, and he
does not have the problem of the
/usr/lib64/python2.5/site-packages/pyhdfeos-1.0_r57_58-py2.5-linux-x86_64.egg
variable showing up as one of the first entries in sys.path.

Thanks for the education,

Catherine

 

[Chris Rebert]

I've looked at my sys.path variable and I see that it has
a whole bunch of site-package directories, followed by the
contents of my $PYTHONPATH variable, followed by a list of
misc site-package variables (see below).

But, I'm curious as to where the first bunch of 'site-package'
entries come from.  The
/usr/lib64/python2.5/site-packages/pyhdfeos-1.0_r57_58-py2.5-linux-x86_64..egg
is not present in any of my environmental variables yet it shows up
as one of the first entries in sys.path.

You probably have a .pth file somewhere that adds it (since it's an
egg, probably site-packages/easy-install.pth).
See http://docs.python.org/install/index.html#modifying-python-s-search-path

Cheers,
Chris

 

[Dan Stromberg]

I don't know where the site-packages directories are coming from -
maybe a site.py or sitecustomize.py? Sometimes you can strace with a
very large -s to see where something like this is coming from. -o is
your friend for saving the output to a file in, EG, /tmp.

What I usually do is to put specific versions of modules I need, that
are different from what the OS is providing, in the same CWD as my
script during testing. Then in my experience, CPython imports them
instead. When it comes time for a production run, you can drop the
special modules into the OS directories somewhere.

However, another option is to add a --test command line argument to
your script, and make the option just do something like:

if '--test' in sys.argv:
sys.path.insert(0, os.path.expanduser('~/magic-modules'))

....for example (and that in sys.argv test could probably be improved
to fit your program's specifics).