allow scripts to use .pth files?

[Alan Isaac]

Suppose I have a directory `scripts`.
I'd like the scripts to have access to a package
that is not "installed", i.e., it is not on sys.path.
On this list, various people have described a variety
of tricks they use, but nobody has proposed a
pretty way to allow this.
I am therefore assuming there is not one. (?)

How about allowing a `scripts.pth` file in such a `scripts`
directory, to work like a path configuration file?
(But to be used only when __name__=="__main__".)
Drawbacks?

Alan Isaac

 

[half.italian]

Suppose I have a directory `scripts`.
I'd like the scripts to have access to a package
that is not "installed", i.e., it is not on sys.path.
On this list, various people have described a variety
of tricks they use, but nobody has proposed a
pretty way to allow this.
I am therefore assuming there is not one. (?)

How about allowing a `scripts.pth` file in such a `scripts`
directory, to work like a path configuration file?
(But to be used only when __name__=="__main__".)
Drawbacks?

Alan Isaac

import sys
sys.path.append("../scripts")

import Module_from_scripts_dir

~Sean

 

[Alan Isaac]

import sys
sys.path.append("../scripts")
import Module_from_scripts_dir

That is not actually an answer to the question. In any case,
path manipulation in scripts is often pronounced "not pretty".
(And would also have to be done in every script file.)

Cheers,
Alan Isaac

 

[samwyse]

Suppose I have a directory `scripts`.
I'd like the scripts to have access to a package
that is not "installed", i.e., it is not on sys.path.
On this list, various people have described a variety
of tricks they use, but nobody has proposed a
pretty way to allow this.
I am therefore assuming there is not one. (?)

How about allowing a `scripts.pth` file in such a `scripts`
directory, to work like a path configuration file?
(But to be used only when __name__=="__main__".)
Drawbacks?

Alan Isaac

before i can adequately shoot down your proposal, i need more
information.

first, how does the interpreter know to look in 'scripts' for your
'scripts.pyh' file and not in '.' or '~' or sys.argv[0] or someplace
else? and you do know, don't you, that if you run 'scripts/
myscript.py' then 'scripts' is automagically prepended to the search
path for modules?

second, what's for format of this proposed file? does it contain the
name of a single directory? is it one directory name per line? is it
intended to be os-specific or will the same file work on windows,
unix, vms and macos?

third, is there anything special about the name? should i put a
myscripts.pyh file in the myscripts directory? what if i have
multiple .pyh files?

if i may make some assumptions about the answers to the above, then
this incantation might do somewhat more than what you've asked for
(but what you actually want may be different):

if __name__ == '__main__':
import sys
from os.path import split, join, expanduser
for d in '.', split(sys.argv[0])[0], expanduser('~'):
scripts_pyh = join(d, 'scripts.pyh')
try:
for each_line in open(scripts_pyh).readlines():
sys.path.append(each_line)
except:
pass


personally, though, i would be more likely to use this and skip all of
that 'scripts.pyh' nonsense:

if __name__ == '__main__':
import sys, os.path
for d in '.', os.path.expanduser('~'):
if os.path.isdir(d): sys.path.append(d)

 

[John Machin]

Suppose I have a directory `scripts`.
I'd like the scripts to have access to a package
that is not "installed", i.e., it is not on sys.path.
On this list, various people have described a variety
of tricks they use, but nobody has proposed a
pretty way to allow this.
I am therefore assuming there is not one. (?)

How about allowing a `scripts.pth` file in such a `scripts`
directory, to work like a path configuration file?
(But to be used only when __name__=="__main__".)
Drawbacks?

Alan Isaac

before i can adequately shoot down your proposal, i need more
information.

first, how does the interpreter know to look in 'scripts' for your
'scripts.pyh' file and not in '.' or '~' or sys.argv[0] or someplace
else?

I got the impression that the OP was suggesting that the interpreter
look in the directory in which it found the script.

and you do know, don't you, that if you run 'scripts/
myscript.py' then 'scripts' is automagically prepended to the search
path for modules?

I got the impression that the problem was that the package was not
only not on sys.path but also not in the same directory as the script
that wanted to import it. Otherwise the OP's script.p?h file need only
contain ".\n" (or the path to the directory in which it resided!!),
and I doubt that he was proposing something so silly.

I'm curious whether you think that the OP's use of ".pth" was a typo,
and whether you have read this:
http://docs.python.org/lib/module-site.html

Cheers,
John

 

[Alan Isaac]

I got the impression that the OP was suggesting that the interpreter
look in the directory in which it found the script.

Right.

I got the impression that the problem was that the package was not
only not on sys.path but also not in the same directory as the script
that wanted to import it.
Right.


I'm curious whether you think that the OP's use of ".pth" was a typo,
and whether you have read this:
http://docs.python.org/lib/module-site.html

You seem to understand what I'm getting at.
Thanks John.

Alan Isaac (the OP above)

 

[samwyse]

I got the impression that the OP was suggesting that the interpreter
look in the directory in which it found the script. [...]
I got the impression that the problem was that the package was not
only not on sys.path but also not in the same directory as the script
that wanted to import it. Otherwise the OP's script.p?h file need only
contain ".\n" (or the path to the directory in which it resided!!),
and I doubt that he was proposing something so silly.

And as I'm sure you realize, those two impression slightly contradict
each other. Anyway, a small modification to my second approach would
also work in the case looking for packages in a directory that's
located somewhere relative to the one where the script was found:

if __name__ == '__main__':
import sys, os.path
base = sys.path[0]
for d in 'lib', 'modules':
d2 = os.path.join(base, d)
if os.path.isdir(d2):
sys.path.append(d2)
base = os.path.join(base, '..')
for d in 'modules', 'lib':
d2 = os.path.join(base, d)
if os.path.isdir(d2):
sys.path.append(d2)
# for debugging
print repr(sys.path)

(BTW, this is such a fun script to type. My fingers keep typing
'os.path' where I mean 'sys.path' and vice versa.)

 

[John Machin]

I got the impression that the OP was suggesting that the interpreter
look in the directory in which it found the script. [...]
I got the impression that the problem was that the package was not
only not on sys.path but also not in the same directory as the script
that wanted to import it. Otherwise the OP's script.p?h file need only
contain ".\n" (or the path to the directory in which it resided!!),
and I doubt that he was proposing something so silly.

And as I'm sure you realize, those two impression slightly contradict
each other.

Your sureness is misplaced. Please explain.

 

[samwyse]

I'm curious whether you think that the OP's use of ".pth" was a typo,
and whether you have read this:
http://docs.python.org/lib/module-site.html

I've read it, but not recently; the syntax of the .pyh files was in
the back of my head. I had forgotten about the sitecustomize module,
though. Unfortunately for the OP, while the documentation states,
"After these path manipulations, an attempt is made to import a module
named sitecustomize," the import is apparently done *before* the path
to the script is prepended to sys.path. (My name isn't Luke, so I
don't feel the need to read the source.)

I'm guessing that the OP's real question is, "How does one do site
customizations when one doesn't have write access to the site
directories?" Lots of programs can be installed and run from ~/bin,
but locating ~/lib can be hard, at least for arbitrary values of bin
and lib. At this point, someone usually tells me to write a PEP;
perhaps the OP would like to try his hand? In keeping with the spirit
of the site customizations, I'd specify that all .pyh files get
imported, not just one matching the name of the script. (This is more
robust in the face of hard-links, packages consisting of multiple
scripts, etc.) I'd also suggest an attempt to import a module named
mycustomize after the path to the invoked script is prepended to
sys.path.

 

[samwyse]

I got the impression that the OP was suggesting that the interpreter
look in the directory in which it found the script. [...]
I got the impression that the problem was that the package was not
only not on sys.path but also not in the same directory as the script
that wanted to import it. Otherwise the OP's script.p?h file need only
contain ".\n" (or the path to the directory in which it resided!!),
and I doubt that he was proposing something so silly.

And as I'm sure you realize, those two impression slightly contradict
each other.

Your sureness is misplaced. Please explain.

Oops, you're right. See my other post.