organizing your scripts, with plenty of re-use

[Buck]

Perhaps I've simply not followed this thread closely enough but could you  
let us know a little bit more about how you intend / expect the scripts to  
be used?

If there's a standard directory you expect them to be dropped into by your  
users (e.g. $HOME/scripts) ...

We don't have any such convention. The production code is at a well-
known path, but I need the code to be fully relocatable (cp -r /known/
path ~/mydir) for testing purposes.

Here's a scenario. A user does a cvs checkout into some arbitrary
directory and sees this:

project/
+-- python/
+-- animals.py
+-- mammals/
+-- horse.py
+-- otter.py
+-- reptiles/
+-- gator.py
+-- newt.py
+-- misc/
+-- lungs.py
+-- swimming.py

These are all runnable scripts that "just work" with no extra effort
or knowlege, both in the testing scenario above, and for normal users
that run it from some central location (maybe "/tools/mycompany/bin/
mammals").

The frustrating thing, for me, is that all these requirements are met
if you leave the scripts in jumbled into a flat directory. As soon as
you start organizing things, you need a good amount of boilerplate in
each script to make things work anything like they did with the flat
directory.

I wouldn't mind having to write 'from mammals.horse import Horse' or
'from .animals import Animal' but it's not that simple.

 

[Buck]

The Original Poster is confusing installation difficulties with code
organization -- his problem is that users have special requirements for
installation, and he's trying to work around those requirements by
organizing his code differently.

I believe you have it backwards here. My installation requirements (no
installation at all) are met with a flat organization, but as soon as
I try to improve that organization, I can't meet the installation
requirements.

As far as I can see, whether or not he uses a package, he will still have
the same problem with installation, namely, that his users aren't
developers, plus for non-technical reasons he can't provide an installer
and has to have users check code out of CVS.

That's probably true... I was hoping it wasn't.

a simple setup script that
modifies the user's .tcshrc to include the appropriate PYTHONPATH will
solve his other problem.

I haven't seen a program that needs separate installation procedure
for each user. Maybe mail clients?

But I have seen plenty of programs that can be installed with a simple
copy. Maybe that's just not true for Python projects except with the
simplest of organizations

 

[Carl Banks]

We don't have any such convention. The production code is at a well-
known path, but I need the code to be fully relocatable (cp -r /known/
path ~/mydir) for testing purposes.

Here's a scenario. A user does a cvs checkout into some arbitrary
directory and sees this:

project/
+-- python/
    +-- animals.py
    +-- mammals/
        +-- horse.py
        +-- otter.py
    +-- reptiles/
        +-- gator.py
        +-- newt.py
    +-- misc/
        +-- lungs.py
        +-- swimming.py

These are all runnable scripts that "just work" with no extra effort
or knowlege, both in the testing scenario above, and for normal users
that run it from some central location (maybe "/tools/mycompany/bin/
mammals").

The frustrating thing, for me, is that all these requirements are met
if you leave the scripts in jumbled into a flat directory. As soon as
you start organizing things, you need a good amount of boilerplate in
each script to make things work anything like they did with the flat
directory.

I wouldn't mind having to write 'from mammals.horse import Horse' or
'from .animals import Animal' but it's not that simple.

Ok, presumably each of these files has something like the following
code inside them, since they can all be both imported and run as
scripts, right?

if __name__ == '__main__':
main()

So follow these steps.

1. Remove this section from all of the files. If the section contains
anything nontrivial, move the nontrivial stuff into your main()
function, or whatever function it is that call. I'll just assume that
the function is called main.

2. In every directory and subdirectory below python, touch an
__init__.py file. It can be empty. You don't need one for the python
directory itself.

3. Change all your imports to package imports. I recommend using
absolute imports only, but that's up to you. "import otter" would
become "from mammals import otter", and so on.

4. Add the following basic script to your Python directory. This is a
script, NOT A MODULE, so don't ever import it. In fact, don't even
give it a py extension if you're not on Windows. I'll call the script
"sa", for "simulate animals". This script accepts the name of a
module to import on the command line, checks whether the top level
module is acceptable, imports the module, then calls the main function
in that module. Enhance as required.

#!/usr/bin/python
import sys
allowed_top_levels = set(('animals','mammals','reptiles','misc'))
modname = sys.argv[1]
modpath = modname.split('.')
if modpath[0] not in allowed_top_levels:
raise RuntimeError('invalid top level module specified')
mod = __import__(modname)
for pkgname in modpath[1:]:
mod = getattr(mod,pkgname)
mod.main() # or main(*sys.argv[2:]), or however you call it


5. Tell your users to stop entering commands like this:

$ ./python/mammal/otter.py


And start entering commands like this:

$ sa mammal.otter


With a little care this method will work fine, and will be installable
with cp -r.


Carl Banks

 

[Steven D'Aprano]

Here's a scenario. A user does a cvs checkout into some arbitrary
directory and sees this:

project/
+-- python/
+-- animals.py
+-- mammals/
+-- horse.py
+-- otter.py
+-- reptiles/
+-- gator.py
+-- newt.py
+-- misc/
+-- lungs.py
+-- swimming.py

These are all runnable scripts that "just work" with no extra effort or
knowlege, both in the testing scenario above, and for normal users that
run it from some central location (maybe "/tools/mycompany/bin/
mammals").

The frustrating thing, for me, is that all these requirements are met if
you leave the scripts in jumbled into a flat directory.

I bet that's not true. I bet that they Just Work only if the user cd's
into the directory first. In other words, if you have all your scripts in
the directory /tools/mycompany/bin/scripts, this will work:

$ cd /tools/mycompany/bin/scripts
$ animals.py

but this won't:

$ cd /home/username
$ /tools/mycompany/bin/scripts/animals.py


In the first case, it works because the current working directory is
included in the PYTHONPATH, and all the modules you need are there. In
the second, it doesn't because the modules aren't in either the current
directory or any other directory in the PYTHONPATH.

That's my prediction.

As soon as you
start organizing things, you need a good amount of boilerplate in each
script to make things work anything like they did with the flat
directory.

You shouldn't need that much boilerplate. A little, perhaps, but not that
much.

Although I have defended the practice of making modules executable, I do
recognise that for complex packages this becomes difficult quickly. It
sounds like you would benefit greatly from separating the interface from
the backend. You should arrange matters so that the users see something
like this:

project/
+-- animal
+-- mammal
+-- reptile
+-- backend/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py


where the front end is made up of three scripts "animal", "mammal" and
"reptile", and the entire backend is in a package. Each front end script
manages a small amount of boilerplate, something like this:

#!/usr/bin/python
import os, sys

if __name__ == '__main__':
# find out where we are, and add it to the path
location = __import__('__main__').__file__
location = os.path.dirname(location)
if location not in sys.path:
sys.path.append(location)

import animals
animals.main()

That's not a lot of boilerplate for a script.

The backend modules rely on the path being setup correctly. For example,
animals.py might do:

import mammals.horse
horse.ride('like the wind')


Calling the backend modules directly is not supported.

 

[Gabriel Genellina]

En Sat, 10 Oct 2009 05:57:08 -0300, Steven D'Aprano

I bet that's not true. I bet that they Just Work only if the user cd's
into the directory first. In other words, if you have all your scripts in
the directory /tools/mycompany/bin/scripts, this will work:

$ cd /tools/mycompany/bin/scripts
$ animals.py

but this won't:

$ cd /home/username
$ /tools/mycompany/bin/scripts/animals.py


In the first case, it works because the current working directory is
included in the PYTHONPATH, and all the modules you need are there. In
the second, it doesn't because the modules aren't in either the current
directory or any other directory in the PYTHONPATH.

That's my prediction.

Mmm, I predict you won't have much success in your new fortune teller
career...
You got it backwards. At least on Windows, the current directory *isn't*
on the Python path, but the directory containing the script *is* included.
So both alternatives above work.

You shouldn't need that much boilerplate. A little, perhaps, but not that
much.


project/
+-- animal
+-- mammal
+-- reptile
+-- backend/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py


where the front end is made up of three scripts "animal", "mammal" and
"reptile", and the entire backend is in a package. Each front end script
manages a small amount of boilerplate, something like this:

[code omited]

You need even less code. In fact, you don't need any boilerplate code.
This Just Works (tm):

<code>
#!/usr/bin/python

from backend import animals
animals.main()
</code>

That's all. No need to set PYTHONPATH, nor alter sys.path, nor install the
whole tree in a specific place.
The good thing is that, if the backend package is properly installed
somewhere in the Python path, and the animal script is copied onto any
suitable place, it still works with no modifications.

 

[Gabriel Genellina]

En Sat, 10 Oct 2009 05:57:08 -0300, Steven D'Aprano

I bet that's not true. I bet that they Just Work only if the user cd's
into the directory first. In other words, if you have all your scripts in
the directory /tools/mycompany/bin/scripts, this will work:

$ cd /tools/mycompany/bin/scripts
$ animals.py

but this won't:

$ cd /home/username
$ /tools/mycompany/bin/scripts/animals.py


In the first case, it works because the current working directory is
included in the PYTHONPATH, and all the modules you need are there. In
the second, it doesn't because the modules aren't in either the current
directory or any other directory in the PYTHONPATH.

That's my prediction.

Mmm, I predict you won't have much success in your new fortune teller
career...
You got it backwards. At least on Windows, the current directory *isn't*
on the Python path, but the directory containing the script *is* included.
So both alternatives above work.

You shouldn't need that much boilerplate. A little, perhaps, but not that
much.


project/
+-- animal
+-- mammal
+-- reptile
+-- backend/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py


where the front end is made up of three scripts "animal", "mammal" and
"reptile", and the entire backend is in a package. Each front end script
manages a small amount of boilerplate, something like this:

[code omited]

You need even less code. In fact, you don't need any boilerplate code.
This Just Works (tm):

<code>
#!/usr/bin/python

from backend import animals
animals.main()
</code>

That's all. No need to set PYTHONPATH, nor alter sys.path, nor install the
whole tree in a specific place.
The good thing is that, if the backend package is properly installed
somewhere in the Python path, and the animal script is copied onto any
suitable place, it still works with no modifications.

 

[Ethan Furman]

En Sat, 10 Oct 2009 05:57:08 -0300, Steven D'Aprano



Mmm, I predict you won't have much success in your new fortune teller
career...
You got it backwards. At least on Windows, the current directory
*isn't* on the Python path, but the directory containing the script
*is* included.
So both alternatives above work.

Are you sure?

------
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.['', <-- current directory?
'C:\\WINDOWS\\system32\\python25.zip',
'C:\\Python25\\DLLs',
'C:\\Python25\\lib',
'C:\\Python25\\lib\\plat-win',
'C:\\Python25\\lib\\lib-tk',
'C:\\Python25',
'C:\\Python25\\lib\\site-packages',
'C:\\Python25\\lib\\site-packages\\win32',
'C:\\Python25\\lib\\site-packages\\win32\\lib',
'C:\\Python25\\lib\\site-packages\\Pythonwin']

------
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

c:\temp>type print_path.py
import sys
print sys.path

c:\temp>cd \

C:\>python \temp\print_path.py
['C:\\temp', <-- hmmm, not the current directory
'C:\\WINDOWS\\system32\\python25.zip',
'c:\\python25\\DLLs',
'c:\\python25\\lib',
'c:\\python25\\lib\\plat-win',
'c:\\python25\\lib\\lib-tk',
'c:\\python25',
'c:\\python25\\lib\\site-packages',
'c:\\python25\\lib\\site-packages\\win32',
'c:\\python25\\lib\\site-packages\\win32\\lib',
'c:\\python25\\lib\\site-packages\\Pythonwin']

 

[Steven D'Aprano]

Mmm, I predict you won't have much success in your new fortune teller
career...
You got it backwards. At least on Windows, the current directory *isn't*
on the Python path, but the directory containing the script *is*
included. So both alternatives above work.

Oops. Serves me right for making what I thought was a sure bet before
testing

It's the same for Linux too, and it seems to hold at least back to Python
1.5. On the ignominy of it all! I guess I'll have to give up the fortune-
telling and get a proper job

 

[Buck]

The good thing is that, if the backend package is properly installed  
somewhere in the Python path ... it still works with no modifications.

I'd like to get to zero-installation if possible. It's easy with
simple python scripts, why not packages too? I know the technical
reasons, but I haven't heard any practical reasons.

If the reasons are purely technical, it smells like a PEP to me.

 

[Buck]

#!/usr/bin/python
import os, sys

if __name__ == '__main__':
    # find out where we are, and add it to the path
    location = __import__('__main__').__file__
    location = os.path.dirname(location)
    if location not in sys.path:
        sys.path.append(location)

    import animals
    animals.main()

That's not a lot of boilerplate for a script.

Does this keep working if a user decides to symlink it to their ~/
bin/? I think it doesn't (untested). Adding that capability adds a few
lines (from my boilerplate previously):
#continue working even if the script is symlinked and then compiled
if f.endswith(".pyc"): f = f[:-1]
if islink(f): f = realpath(f)

My issues with boilerplate are several. It adds possible points of
bugs to the script. It's done differently by everybody, which has
various implications. The supported features are (obviously)
unstandardized. It's untested by the community and so may have bugs
even after a year or two of use.

It also violates the Don't Repeat Yourself (DRY) principle. To fix
this, I might make a packagehelper module or some such, but as soon as
I start organizing my scripts into a hierarchy (the topic of the OP),
I have a catch 22 in that packagehelper is a module that helps me find
modules. This means I need a copy of packagehelper in each script
directory, again violating DRY.

There's a couple other things, but I'll stop myself here.
--Buck

 

[Gabriel Genellina]

I'd like to get to zero-installation if possible. It's easy with
simple python scripts, why not packages too? I know the technical
reasons, but I haven't heard any practical reasons.

If the reasons are purely technical, it smells like a PEP to me.

That's what I meant to say. It IS a zero-installation schema, and it also
works if you properly install the package. Quoting Steven D'Aprano
(changing names slightly):

"""You would benefit greatly from separating the interface from
the backend. You should arrange matters so that the users see something
like this:

project/
+-- animal
+-- mammal
+-- reptile
+-- somepackagename/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py


where the front end is made up of three scripts "animal", "mammal" and
"reptile", and the entire backend is in a package.""" [ignore the rest]

By example, the `animal` script would contain:

from somepackagename import animals
animals.main()

or perhaps something more elaborate, but in any case, the script imports
whatever it needs from the `somepackagename` package.

The above script can be run:

a) directly from the `project` directory; this could be a checked out copy
from svn, or a tar file extracted in /tmp, or whatever. No need to install
anything, it just works.

b) alternatively, you may install somepackagename into site-packages (or
the user site directory, or any other location along the Python path), and
copy the scripts into /usr/bin (or any other location along the system
PATH), and it still works.

The key is to put all the core functionality into a package, and place the
package where Python can find it. Also, it's a good idea to use relative
imports from inside the package. There is no need to juggle with sys.path
nor even set PYTHONPATH nor import __main__ nor play any strange games; it
Just Works (tm).

 

[Gabriel Genellina]

I'd like to get to zero-installation if possible. It's easy with
simple python scripts, why not packages too? I know the technical
reasons, but I haven't heard any practical reasons.

If the reasons are purely technical, it smells like a PEP to me.

That's what I meant to say. It IS a zero-installation schema, and it also
works if you properly install the package. Quoting Steven D'Aprano
(changing names slightly):

"""You would benefit greatly from separating the interface from
the backend. You should arrange matters so that the users see something
like this:

project/
+-- animal
+-- mammal
+-- reptile
+-- somepackagename/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py


where the front end is made up of three scripts "animal", "mammal" and
"reptile", and the entire backend is in a package.""" [ignore the rest]

By example, the `animal` script would contain:

from somepackagename import animals
animals.main()

or perhaps something more elaborate, but in any case, the script imports
whatever it needs from the `somepackagename` package.

The above script can be run:

a) directly from the `project` directory; this could be a checked out copy
from svn, or a tar file extracted in /tmp, or whatever. No need to install
anything, it just works.

b) alternatively, you may install somepackagename into site-packages (or
the user site directory, or any other location along the Python path), and
copy the scripts into /usr/bin (or any other location along the system
PATH), and it still works.

The key is to put all the core functionality into a package, and place the
package where Python can find it. Also, it's a good idea to use relative
imports from inside the package. There is no need to juggle with sys.path
nor even set PYTHONPATH nor import __main__ nor play any strange games; it
Just Works (tm).

 

[Gabriel Genellina]

#!/usr/bin/python
import os, sys

if __name__ == '__main__':
    # find out where we are, and add it to the path
    [...]

That's not a lot of boilerplate for a script.

My issues with boilerplate are several. [...]

As pointed out before, all this boilerplate code is actually unnecesary.
Just import the module you want from its package, and you're done
(usually).

 

[Stef Mientki]

I'd like to get to zero-installation if possible. It's easy with
simple python scripts, why not packages too? I know the technical
reasons, but I haven't heard any practical reasons.

If the reasons are purely technical, it smells like a PEP to me.

That's what I meant to say. It IS a zero-installation schema, and it
also works if you properly install the package. Quoting Steven
D'Aprano (changing names slightly):

"""You would benefit greatly from separating the interface from
the backend. You should arrange matters so that the users see something
like this:

project/
+-- animal
+-- mammal
+-- reptile
+-- somepackagename/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py


where the front end is made up of three scripts "animal", "mammal" and
"reptile", and the entire backend is in a package.""" [ignore the rest]

By example, the `animal` script would contain:

from somepackagename import animals
animals.main()

or perhaps something more elaborate, but in any case, the script
imports whatever it needs from the `somepackagename` package.

The above script can be run:

a) directly from the `project` directory; this could be a checked out
copy from svn, or a tar file extracted in /tmp, or whatever. No need
to install anything, it just works.

b) alternatively, you may install somepackagename into site-packages
(or the user site directory, or any other location along the Python
path), and copy the scripts into /usr/bin (or any other location along
the system PATH), and it still works.

The key is to put all the core functionality into a package, and place
the package where Python can find it. Also, it's a good idea to use
relative imports from inside the package. There is no need to juggle
with sys.path nor even set PYTHONPATH nor import __main__ nor play any
strange games; it Just Works (tm).

please don't get angry,
I'm not a programmer, I'm just a human ;-)

Hierarchical choices are done on todays knowledge, tomorrow we might
have different views and want/need to arrange things in another way.
An otter may become a reptile ;-)
So from the human viewpoint the following should be possible (and is for
example possible in Delphi)
- I can move the complete project anywhere I like and it should still
work without any modifications (when I move my desk I can still do my work)
- I can move any file in he project to any other place in the project
and again everything should work without any modifications ( when I
rearrange my books, I can still find a specific book)

In my humble opinion if these actions are not possible, there must be
redundant information in the collection. The only valid reason for
redundant information is to perform self healing (or call it error
correction), and here we have a catch-22.

cheers,
Stef Mientki

 

[Carl Banks]

I'd like to get to zero-installation if possible. It's easy with
simple python scripts, why not packages too? I know the technical
reasons, but I haven't heard any practical reasons.

No it's purely technical. Well mostly technical (there's a minor
issue of how a script would figure out its "root"). No language is
perfect, not even Python, and sometimes you just have to deal with
things the way they are.

We're trying to help you with workarounds, but it seems like you just
want to vent more than you want an actual solution.

If it makes you feel better, go ahead and vent. If you want a
solution, try instead to sit down and implement the advice Steven,
Gabriel, or I gave you.

If the reasons are purely technical, it smells like a PEP to me.

Good luck with that. I'd wholeheartedly support a good alternative, I
just want to warn you that it's not a simple issue to fix, it would be
involve spectacular and highly backwards-incompatible changes.


Carl Banks

 

[Ethan Furman]

Gabriel Genellina wrote:
[snip]

That's what I meant to say. It IS a zero-installation schema, and it
also works if you properly install the package. Quoting Steven
D'Aprano (changing names slightly):

"""You would benefit greatly from separating the interface from
the backend. You should arrange matters so that the users see something
like this:

project/
+-- animal
+-- mammal
+-- reptile
+-- somepackagename/
+-- __init__.py
+-- animals.py
+-- mammals/
+-- __init__.py
+-- horse.py
+-- otter.py
+-- reptiles/
+-- __init__.py
+-- gator.py
+-- newt.py
+-- misc/
+-- __init__.py
+-- lungs.py
+-- swimming.py

[snip]

The key is to put all the core functionality into a package, and place
the package where Python can find it. Also, it's a good idea to use
relative imports from inside the package. There is no need to juggle
with sys.path nor even set PYTHONPATH nor import __main__ nor play any
strange games; it Just Works (tm).

please don't get angry,
I'm not a programmer, I'm just a human ;-)

Hierarchical choices are done on todays knowledge, tomorrow we might
have different views and want/need to arrange things in another way.
An otter may become a reptile ;-)
So from the human viewpoint the following should be possible (and is for
example possible in Delphi)

- I can move the complete project anywhere I like and it should still
work without any modifications (when I move my desk I can still do my work)

Move a complete package anywhere along the PYTHONPATH and it will still
work. Check.

- I can move any file in he project to any other place in the project
and again everything should work without any modifications ( when I
rearrange my books, I can still find a specific book)

Move any file in any directory to any other spot in that same directory
and it will still work. Check. ;-)

Humans are a lot smarter than computers. Even 'just humans'. ;-) If
you move your book, then can't find it on the last shelf it used to be
on, you look on other shelves, you look on your desk, you look on the
coffe table, you look in your car, etc, etc, and so forth. If you move
a file in a package to somewhere else, and you don't tell the package
where it's at, it's not going to start looking all over the hard-drive
for it. If that were the case you would have to be extra careful to
have every module's name be distinct, and then what's the point of
having packages?

~Ethan~

 

[greg]

- I can move the complete project anywhere I like and it should still
work without any modifications (when I move my desk I can still do my work)

Gabriel's organisation satisfies that.

- I can move any file in he project to any other place in the project
and again everything should work without any modifications

That's not a reasonable thing to expect from Python. The
position of a module in the module naming hierarchy is
implied by the location of its file in the directory
hierarchy. When you move the file, you change the name
of the module, so you have to change any import statements
that refer to it accordingly.

For example, if you decide that otters are now lizards
and move mammals/otter.py into the lizards directory,
the module is now called lizards.otter instead of
mammals.otter.

( when I
rearrange my books, I can still find a specific book)

Only because you've updated the index in your brain that
maps book titles to their locations. If you don't do that,
you lose track of the book and have to search for it.

The Python equivalent of this is updating import statements
to reflect the new location of the module.

In my humble opinion if these actions are not possible, there must be
redundant information in the collection.

Actually, it's because Python *doesn't* keep redundant
information. To be able to move files around without changing
anything else, Python would have to keep an index somewhere
mapping module names to files. But there is no such index --
the directory structure *is* the index.

 

[Dennis Lee Bieber]

coffe table, you look in your car, etc, etc, and so forth. If you move
a file in a package to somewhere else, and you don't tell the package
where it's at, it's not going to start looking all over the hard-drive
for it. If that were the case you would have to be extra careful to
have every module's name be distinct, and then what's the point of
having packages?

Heh... Digging up some ancient history but... TRS-80 TRSDOS6 (and
some earlier incarnations too) WOULD search all active drives if no
drive letter was specified (no subdirectories, of course -- and I never
had a hard drive on mine [$5000 for a 5MB drive?], just the two
floppies). And for creating files, again if no drive were specified, it
would create the file on the first drive that was not write-protected.

Made it easy to work with write-protected system floppies while not
having to specify drives for user data.

 

[Gabriel Genellina]

En Tue, 13 Oct 2009 03:48:00 -0300, Dennis Lee Bieber

coffe table, you look in your car, etc, etc, and so forth. If you move
a file in a package to somewhere else, and you don't tell the package
where it's at, it's not going to start looking all over the hard-drive
for it. If that were the case you would have to be extra careful to
have every module's name be distinct, and then what's the point of
having packages?

Heh... Digging up some ancient history but... TRS-80 TRSDOS6 (and
some earlier incarnations too) WOULD search all active drives if no
drive letter was specified (no subdirectories, of course -- and I never
had a hard drive on mine [$5000 for a 5MB drive?], just the two
floppies). And for creating files, again if no drive were specified, it
would create the file on the first drive that was not write-protected.

In the old MSDOS era, there was the APPEND command. It was used to set a
list of directories or subdirectories that were searched for data files
(in a way similar as PATH works for executable files). For example, after
APPEND c:\pirulo\data;doc
you could edit a file like c:\foo\doc\readme.txt directly from the c:\foo
directory:

c:\foo> edit readme.txt

and because of "doc" being in the APPEND path, edit would find it. I think
Stef would enjoy using it - just listing each subdirectory would make the
disk tree completely flat as seen by the application.

I think this functionality was removed by the time Windows 95 came out
because it was very dangerous. It was extremely easy to open (or even
remove!) the wrong file.

 

[Buck]

No it's purely technical.  Well mostly technical (there's a minor
issue of how a script would figure out its "root").  No language is
perfect, not even Python, and sometimes you just have to deal with
things the way they are.

Python is the closest I've seen. I'd like to deal with this wart if we
can.

We're trying to help you with workarounds, but it seems like you just
want to vent more than you want an actual solution.

Steven had the nicest workaround (with the location = __import__
('__main__').__file__ trick), but none of them solve the problem of
the OP: organization of runnable scripts. So far it's been required to
place all runnable scripts directly above any used packages. The
workaround that Gabriel has been touting requires this too.

Maybe it seems like I'm venting when I shoot down these workarounds,
but my real aim is to find some sort of consensus, either that there
is a solution, or an unsolved problem. I'd be delighted with a
solution, but none have been acceptable so far (as I explained in
aggravating detail earlier).

If I can find consensus that this is a real problem, not just my
personal nit-pick, then I'd be willing to donate my time to design,
write and push through a PEP for this purpose. I believe it can be
done neatly with just three new standard functions, but it's premature
to discuss that.

Good luck with that.  I'd wholeheartedly support a good alternative, Thanks.

I just want to warn you that it's not a simple issue to fix, it would be
involve spectacular and highly backwards-incompatible changes.
--Carl Banks

I don't believe that's true, but I think that's a separate discussion.