How to install local module other than in "site-packages"?

J

Jive Dadson

(My apologies if this question shows up twice. I posted it quite a
while ago, and it's yet to show up.)

This is no doubt a beginner's question, but I've searched for the
answer for quite a while, to no avail. I'm running Python 2.6 under
Windows XP.

How do I install a module that I wrote, without putting it in the
site-packages directory for a specific release? I have stuff that, to
the best of my knowledge and belief, ought to work under any release. I
do not want to have multiple copies of it. When I edit one of the
modules, I want it to "take" for every release. Is that clear? Hope so.
 
D

Diez B. Roggisch

Am 18.01.10 01:07, schrieb Jive Dadson:
(My apologies if this question shows up twice. I posted it quite a
while ago, and it's yet to show up.)

This is no doubt a beginner's question, but I've searched for the answer
for quite a while, to no avail. I'm running Python 2.6 under Windows XP.

How do I install a module that I wrote, without putting it in the
site-packages directory for a specific release? I have stuff that, to
the best of my knowledge and belief, ought to work under any release. I
do not want to have multiple copies of it. When I edit one of the
modules, I want it to "take" for every release. Is that clear? Hope so.

Setting the PYTHONPATH environment variable should work.

Diez
 
J

Jive Dadson

Diez said:
Am 18.01.10 01:07, schrieb Jive Dadson:

Setting the PYTHONPATH environment variable should work.

Diez

I was trying that just now. So far, no joy. Can you type up a DUMMIES
version?

There was no environment variable PYTHONPATH. I added one to my "User
Variables." It's still not finding anything. And yes, I closed and
re-opened my Python shell before I tested it.
 
D

Diez B. Roggisch

Am 18.01.10 01:33, schrieb Jive Dadson:
I was trying that just now. So far, no joy. Can you type up a DUMMIES
version?

There was no environment variable PYTHONPATH. I added one to my "User
Variables." It's still not finding anything. And yes, I closed and
re-opened my Python shell before I tested it.

Did "echo %PYTHONPATH%" yield anything? Or is it part of


?

Diez
 
J

Jive Dadson

Ben said:
Nevertheless, the compiled byte-code version will be specific to the
Python version that compiled it. You need to install the module to a
version-specific directory for that reason at least.


You'll need (I'd love to be corrected on this) to come up with a
deployment discipline: a build process that deploys the module from its
(single, canonical) source location to the appropriate locations when
you decide to deploy it.

Thanks for your help.

Well that majorly sucks. I was hoping the "foo.pth" thing would save
me. There ought to be a way. But even if they did invent one, it
wouldn't help me, because I am stuck at version 2.6. Don't even get me
started on that. I really like Python, but the way every release breaks
everything is a royal pain. The biggest problem is C-extensions of stuff
that does not come with the Python release. I've complained about it
before, but it didn't help. I guess I got myself started on that. :)

Since I cannot imagine facing the challenge of going to 3.0, I guess it
doesn't matter if I have my stuff in a folder that's specific to 2.6.
 
J

Jive Dadson

Diez said:
Did "echo %PYTHONPATH%" yield anything? Or is it part of



?

Diez

Yes and no in that order. Never mind. Ben Fenny talked me out of it
anyway. Grrrrr.
 
J

Jive Dadson

Diez said:
Did "echo %PYTHONPATH%" yield anything? Or is it part of



?

Diez

Update: It's working now. I guess I hadn't reloaded something that I
need to. Thanks for your help.
 
J

Jive Dadson

Okay, I might go this route anyway. It's almost working.

I created a directory (folder in MS-speak) named Modules, and put its
path in the PYTHONPATH env variable.

I can now put a file foo.py into the directory Modules, and it will load
foo.py when I say "import foo."

Now I put a folder into Modules called myModule and in that I put
bar.py. It does not find bar.py when I say either import bar or import
myModule.bar. I tried the myModule.pth trick but it didn't help. Maybe
I did it wrong.
 
B

Benjamin Kaplan

Okay, I might go this route anyway.  It's almost working.

I created a directory (folder in MS-speak) named Modules, and put its path
in the PYTHONPATH env variable.

I can now put a file foo.py into the directory Modules, and it will load
foo.py when I say "import foo."

Now I put a folder into Modules called myModule and in that I put bar.py.
 It does not find bar.py when I say either import bar or import
myModule.bar.  I tried the myModule.pth trick but it didn't help.  Maybe I
did it wrong.

Just put a blank file called __init__.py inside myModule

In Python, packages have to have an __init__.py file in them, even if
it's empty. Whatever you put in __init__.py gets imported when you
"import myModule".

http://docs.python.org/tutorial/modules.html#packages
 
J

Jive Dadson

Thankee. I had just figgered that out. I wrote everything up in a
message titled "The answer," but I accidentally created a new thread
with it. I'll post it in this thread.
 
J

Jive Dadson

Okay, with your help I've figured it out. Instructions are below, but
read the caveat by Ben Fenny in this thread. All this stuff is good for
one default version of Python only. The PYTHONPATH described below, for
example, cannot specify a version number. Yes, that's a pain in the
butt, but there's no way around it. If you switch versions, you may
have to delete all the .pyc files that will show up in the module
folders. Python ought to check them to see if they are valid, but I do
not know if it does so.

These instructions are for MS Windows.

1) Create your modules folder. Let's say it's named "Modules." The
documentation calls it a "package."

2) In an explorer window or on the desktop, right click on My Computer,
and select Properties.

3) Select the Advanced tab, and click on Environment Variables near the
bottom.

4) Look for an environment variable named PYTHONPATH.

a) If you do not find one, create one using the New button(s). I
don't know if it has to be in User Variables or System Variables. To
save time experimenting, I just put one in both. For the value, put the
full path of the folder Modules.

b) If there's already a PYTHONPATH, Edit it, adding a semi-colon
and the full path of folder Module to the end.

5) Put your module folders into the folder Module.

6) (Here's a really arcane bit.) Into each module folder, put a file
named __init__.py. It will be executed when you load the module. It
can be empty, but it has to be there or else the module folder will be
ignored.
 
A

Alf P. Steinbach

* Jive Dadson:
Okay, with your help I've figured it out. Instructions are below, but
read the caveat by Ben Fenny in this thread. All this stuff is good for
one default version of Python only. The PYTHONPATH described below, for
example, cannot specify a version number. Yes, that's a pain in the
butt, but there's no way around it. If you switch versions, you may
have to delete all the .pyc files that will show up in the module
folders. Python ought to check them to see if they are valid, but I do
not know if it does so.

These instructions are for MS Windows.

1) Create your modules folder. Let's say it's named "Modules." The
documentation calls it a "package."

2) In an explorer window or on the desktop, right click on My Computer,
and select Properties.

3) Select the Advanced tab, and click on Environment Variables near the
bottom.

4) Look for an environment variable named PYTHONPATH.

a) If you do not find one, create one using the New button(s). I
don't know if it has to be in User Variables or System Variables. To
save time experimenting, I just put one in both. For the value, put the
full path of the folder Modules.

The User variables /override/ the System variables for a given user. The System
variables provide defaults for all users. One notable exception is PATH, where
the User PATH items are appended to the System PATH.

Anyways, with many items in a variable like PATH it can be impractical to use
Windows' functionality, which presents it all on one line, which for a PATH with
many items can be exceeding long -- like a many thousand characters[1] line.

So for your convenience, below is a little Windows app (as an HTML application,
it's just some HTML and VBScript and CSS) that presents the user PATH items one
per line in a Notepad-like window. It should be no problem modifying it to
instead edit PYTHONPATH, or even a choice of environment variable. As my late
father used to say, when you don't have a tool, you make it.

Maybe now I should recode this in Python. But on the other hand, one shouldn't
spend time fixing what works. So ... enjoy! :)

Note: it's crucial to use [.HTA] filename extension.
Also, it's quite possible that this doesn't work in Windows Vista (at least not
without configuration of Windows), but it works in XP and earlier.


<code file="userpath.hta">
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Strict//EN">
<!-- If this works, then it was written by Alf P. Steinbach.
Otherwise it's someone impersonating me.
And yes, if you've seen a comment like this before: I/he stole it. :) -->
<html>
<head>
<meta name=vs_targetSchema
content="http://schemas.microsoft.com/intellisense/ie5">
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
<meta http-equiv="MSThemeCompatible" content="yes">

<hta:application scroll="no"/>
<title>Simple user-environment %path% editor</title>

<style>
body { margin: 0px; padding: 0px; }

#menuArea
{
width: 100%; height: 1.8em; white-space: nowrap; padding-top: 2px;
background: #E0E0E0;
}

#clientArea
{
width: 100%; position: absolute; top: 1.8em; bottom: 0;
}

#pathEditor
{
display: inline-block;
width: 100%; height: 100%;
border: 0;
position: absolute; top: 0;
overflow: scroll;
}
</style>

<script type="text/vbscript" language="vbscript">
option explicit
dim wshShell
dim wshUserEnv
dim originalText

sub loadText
pathEditor.innerText = replace( wshUserEnv( "PATH" ), ";",
vbNewline)
end sub

sub saveText
dim text
dim button

text = pathEditor.innerText
text = replace( text, vbNewLine, ";" )
button = MsgBox( _
text, vbYesNo + vbQuestion, "Save this as new %path% value?" _
)
if button = vbYes then wshUserEnv( "PATH" ) = text
end sub

sub onBtnLoad
loadText
end sub

sub onBtnLoadOriginal
pathEditor.innerText = originalText
end sub

sub onBtnSave
saveText
end sub

sub onLoaded
set wshShell = createObject( "WScript.Shell" )
set wshUserEnv = wshShell.environment( "USER" )
loadText
originalText = pathEditor.innerText
MsgBox _
"Type one path per line (no semicolons)", _
vbInformation, _
"How to use:"
end sub
</script>
</head>

<body onload="onLoaded">
<div id="menuArea">
&nbsp; <button onClick="onBtnLoad">Load current</button>
&nbsp; <button onClick="onBtnLoadOriginal">Reload original</button>
&nbsp; <button onClick="onBtnSave">Save as current ...</button>
</div>
<div id="clientArea">
<textarea id="pathEditor" wrap="off">
</textarea>
</div>
</body>
</html>
</code>


b) If there's already a PYTHONPATH, Edit it, adding a semi-colon and
the full path of folder Module to the end.

5) Put your module folders into the folder Module.

6) (Here's a really arcane bit.) Into each module folder, put a file
named __init__.py. It will be executed when you load the module. It
can be empty, but it has to be there or else the module folder will be
ignored.


Cheers & hth.,

- Alf


Notes:
[1] Windows XP and I believe also Vista places a limit of about 8 KiB on the
length of any environment variable.
 
J

Jive Dadson

alex23 said:
>
> Actually, if you're using Python 2.6+/3.x, you can effectively skip
> steps 1-5, as these versions now support user site-packages.
>
> Rather than create a Module folder and modify your PYTHONPATH, add (if
> it doesn't exist already) the following folder:
> %APPDATA%/Python/Python26/site-packages
>
> Modules can sit directly in the folder, or within packages.
>
> For more details: http://www.python.org/dev/peps/pep-0370/

That requires a directory whose name embeds the Python version number,
which is the evil from which I flee, or rather sought to flee. Imagine
if all your C++ code had to go into directories that were named for some
specific C++ compiler. It's just WRONG. It's a maintenance nightmare
to have a bunch of different source files that all have to be updated
whenever you fix a bug or add a feature.
 
A

alex23

Jive Dadson said:
That requires a directory whose name embeds the Python version number,
which is the evil from which I flee, or rather sought to flee.  Imagine
if all your C++ code had to go into directories that were named for some
specific C++ compiler.  It's just WRONG.  It's a maintenance nightmare
to have a bunch of different source files that all have to be updated
whenever you fix a bug or add a feature.

With the PEP 370 approach, you can just designate a folder the core
one and symlink all other versions to it. Or better yet, use version
control to push updates to each folder.

With all of the versions sharing one folder, as you'd prefer, it would
be a lot more difficult to actually achieve version-level granularity
if and when you need it. It's not WRONG, it's just providing a level
of control that you don't need at this point.
 
D

Dave Angel

Jive said:
That requires a directory whose name embeds the Python version number,
which is the evil from which I flee, or rather sought to flee.
Imagine if all your C++ code had to go into directories that were
named for some specific C++ compiler. It's just WRONG. It's a
maintenance nightmare to have a bunch of different source files that
all have to be updated whenever you fix a bug or add a feature.
As others have pointed out, you need a "deploy" script, which in your
case would copy the files from source control to the appropriate
production folder. That's analogous to the compile, link and deploy
steps of C++. And if you want to be even more analogous, copy just the
..pyc files, after building them.

Certainly you have lots more files in your version control system which
are not intended to be copied to the "Modules" folder, such as the test
suite.

DaveA
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,755
Messages
2,569,537
Members
45,020
Latest member
GenesisGai

Latest Threads

Top