Multiple versions of Python coexisting in the same OS

[Edward Diener]

Are there any documents about multiple versionsof Python coexisting in
the same OS ( Windows in my case ) and what pitfalls to look out for ? I
have already run into a number of them. I installed Python 2.7 and 3.1.2
into completely folders, but immediately ran into serious problems
executing a Python script.

The first problem is that just invoking Python will start whichever
version is first in the PATH, and this is true from the command line or
internally in a Python script.

The second problem is that invoking a script ( some xxx.py ) will start
whichever version of Python is associated with the .py extension.

The third problem is if some software expects its scripts, which it puts
in some Python subdirectory to be in the PATH.

There may be other considerations but overall having to versions
coexisting has turned out to be a big headache involving both changes in
the PATH and in the .py association.

Does anybody know of other things to look out for ?

 

[Mark Lawrence]

Are there any documents about multiple versionsof Python coexisting in
the same OS ( Windows in my case ) and what pitfalls to look out for ? I
have already run into a number of them. I installed Python 2.7 and 3.1.2
into completely folders, but immediately ran into serious problems
executing a Python script.

The first problem is that just invoking Python will start whichever
version is first in the PATH, and this is true from the command line or
internally in a Python script.

The second problem is that invoking a script ( some xxx.py ) will start
whichever version of Python is associated with the .py extension.

The third problem is if some software expects its scripts, which it puts
in some Python subdirectory to be in the PATH.

There may be other considerations but overall having to versions
coexisting has turned out to be a big headache involving both changes in
the PATH and in the .py association.

Does anybody know of other things to look out for ?

I found this only yesterday and found it extremely helpful, find the
post by Gabriel Genellina.

http://www.eggheadcafe.com/software/aspnet/35716114/maintain-2-versions-of-py.aspx

HTH.

Mark Lawrence

 

[Edward Diener]

I found this only yesterday and found it extremely helpful, find the
post by Gabriel Genellina.

http://www.eggheadcafe.com/software/aspnet/35716114/maintain-2-versions-of-py.aspx

I found the solutions too exotic for actual use, and completely
ineffectual for the cases I originally cited. The people in that thread
seem to have completely forgotten that Python can be invoked externally
and internally both through executing 'python(w) xxx' and through
executing a file with the file extension(s) associated with Python. They
seem to have forgotten this can be within scripts or any other program
using Python, both written by themselves and by others, and not just by
their typing 'python(w) xxx' somewhere. Their solutions seem to believe
that only they will externally be i9nvoking Python and only for their
own written scripts, as opposed to the many libraries using Python as
well as the Python distribution itself.

The best solution is some program which changes the PATH and the Python
file type associations depending on which version of Python one wants to
use on one's own system when more than one Python version must coexist
with others. I will probably write such a program for myself.

Are the .py and .pyc extensions the only ones which are associated with
Python or are there others, for a normal Python installation in Windows ?

 

[Edward Diener]

I found this only yesterday and found it extremely helpful, find the
post by Gabriel Genellina.

http://www.eggheadcafe.com/software/aspnet/35716114/maintain-2-versions-of-py.aspx

I found the solutions too exotic for actual use, and completely
ineffectual for the cases I originally cited. The people in that thread
seem to have completely forgotten that Python can be invoked externally
and internally both through executing 'python(w) xxx' and through
executing a file with the file extension(s) associated with Python. They
seem to have forgotten this can be within scripts or any other program
using Python, both written by themselves and by others, and not just by
their typing 'python(w) xxx' somewhere. Their solutions seem to believe
that only they will externally be i9nvoking Python and only for their
own written scripts, as opposed to the many libraries using Python as
well as the Python distribution itself.

The best solution is some program which changes the PATH and the Python
file type associations depending on which version of Python one wants to
use on one's own system when more than one Python version must coexist
with others. I will probably write such a program for myself.

Are the .py and .pyc extensions the only ones which are associated with
Python or are there others, for a normal Python installation in Windows ?

 

[Chris Rebert]

The best solution is some program which changes the PATH and the Python file
type associations depending on which version of Python one wants to use on
one's own system when more than one Python version must coexist with others.
I will probably write such a program for myself.

Are the .py and .pyc extensions the only ones which are associated with
Python or are there others, for a normal Python installation in Windows ?

There's also .pyw

Cheers,
Chris

 

[Steven D'Aprano]

There's also .pyw

Also .pyo

..py = Python source code, usually associated with command line Python
..pyc = Python byte code
..pyo = Python optimized byte code
..pyw = is Windows only, and shouldn't open a console window.

 

[Gelonida]

Hi Edward,

I found the solutions too exotic for actual use, and completely
ineffectual for the cases I originally cited. The people in that thread
seem to have completely forgotten that Python can be invoked externally
and internally both through executing 'python(w) xxx' and through
executing a file with the file extension(s) associated with Python. They
seem to have forgotten this can be within scripts or any other program
using Python, both written by themselves and by others, and not just by
their typing 'python(w) xxx' somewhere. Their solutions seem to believe
that only they will externally be i9nvoking Python and only for their
own written scripts, as opposed to the many libraries using Python as
well as the Python distribution itself.

The best solution is some program which changes the PATH and the Python
file type associations depending on which version of Python one wants to
use on one's own system when more than one Python version must coexist
with others. I will probably write such a program for myself.

Hi Edward,

changing the path and is perfect for people who use consoles.
(under linux there's virtuelenv for his and it's great)

changing the file association is perfect for people who'd know at which
time they want to use which version of python.

The usecase, that I'm nore aware of however is somethig like having some
scripts / directories, that should use one version of python
and others that shoud use another.
In unix you do this normally with the 'shebang line'
( e.g. #!/usr/bin/env/python2.6 )

There the windows solution could be something like a small 'pystarter'
program, which would decide depending on the file's location / the
file's first line which python should be started.

 

[Edward Diener]

Hi Edward,


Hi Edward,

changing the path and is perfect for people who use consoles.
(under linux there's virtuelenv for his and it's great)

changing the file association is perfect for people who'd know at which
time they want to use which version of python.

The problem with this is that you forget that a script can invoke Python
internally. So whether one uses the console or file association method
of invoking Python externally, any already written script can use either
internally.

The usecase, that I'm nore aware of however is somethig like having some
scripts / directories, that should use one version of python
and others that shoud use another.
In unix you do this normally with the 'shebang line'
( e.g. #!/usr/bin/env/python2.6 )

There the windows solution could be something like a small 'pystarter'
program, which would decide depending on the file's location / the
file's first line which python should be started.

This does not work when Python is invoked internally via a file
association. That was the point of my saying that the simple solutions
do not work.

 

[Edward Diener]

Also .pyo

.py = Python source code, usually associated with command line Python
.pyc = Python byte code
.pyo = Python optimized byte code
.pyw = is Windows only, and shouldn't open a console window.

Thanks ! I had forgotten about .pyo and .pyw under Windows.

 

[Thomas Jollans]

The problem with this is that you forget that a script can invoke Python
internally. So whether one uses the console or file association method
of invoking Python externally, any already written script can use either
internally.

Maybe it's just me, but I think that a script that does this is quite
simply badly written: it *will* break on systems that have multiple
Python versions.

If you let .py scripts specify which interpreter they'd like to be run
with in the first line, with some sort of pystarter script that you map
to the file extensions (or by using UNIX), this should work equally well
no matter if the script being run by a script or by a user.

If you invoke the Python interpreter directly, you should NEVER EVER
assume that the interpreter is in a certain place, or on the PATH. It's
a stupid idea: what if it's not? Instead, if you really must, invoke
sys.executable instead of guessing.

Obviously, Windows doesn't have fork(), but still, I don't see any
reason to leave the comfort of your own running interpreter: you can use
runpy to run python scripts. If you want them to run in the background,
you can use threads, or, if you require (or want) separate processes,
use multiprocessing.

 

[News123]

On 7/25/2010 6:07 AM, Gelonida wrote:

This does not work when Python is invoked internally via a file
association. That was the point of my saying that the simple solutions
do not work.

I'm not sure I understand. The ida is of course, that the file
association would point to the pystarter and that pystarter would
depending on directory / first line of the script
identify the correct executable to be started with.



Perhaps you could once more explain, what your intended solution would be.

 

[Gelonida]

This does not work when Python is invoked internally via a file
association. That was the point of my saying that the simple solutions
do not work.

What do you mean with invoking python internally?

call another python script? from a python script.

You could start it again via a pystarter.
or just with python (assuming the starter adapts the path)


The problem, that I currently encounter are some scripts, which want to
use python UNO (open office delivers ther own version of python)
and others which want to use libraries of my 'normal' python.

having the file association point to a python starter and let the python
starter choose could be an option.

However I never tried so far.

the fast and easy option is to just have specific file suffixes for the
open office python scripts (like e.g ) .oopy

 

[Joel Goldstick]

Are there any documents about multiple versionsof Python coexisting in
the same OS ( Windows in my case ) and what pitfalls to look out for ? I
have already run into a number of them. I installed Python 2.7 and 3.1.2
into completely folders, but immediately ran into serious problems
executing a Python script.

The first problem is that just invoking Python will start whichever
version is first in the PATH, and this is true from the command line or
internally in a Python script.

The second problem is that invoking a script ( some xxx.py ) will start
whichever version of Python is associated with the .py extension.

The third problem is if some software expects its scripts, which it puts
in some Python subdirectory to be in the PATH.

There may be other considerations but overall having to versions
coexisting has turned out to be a big headache involving both changes in
the PATH and in the .py association.

Does anybody know of other things to look out for ?

There is this:
http://pypi.python.org/pypi/virtualenv

We have a good group in NYC for python and django. That's where I
learned about virtualevn. I admit I'm not fully up to speed on it, but
it lets you set up multiple 'virtual environments' for python to do
exactly what I think you are asking about

Joel Goldstick

 

[Edward Diener]

There is this:
http://pypi.python.org/pypi/virtualenv

It appears to be only for Linux.

 

[Edward Diener]

Maybe it's just me, but I think that a script that does this is quite
simply badly written: it *will* break on systems that have multiple
Python versions.

Whether it is badly written or not in your opinion it is legal and
happens all the time. Are you going to refuse to use any script, no
matter for what library or for what purpose, that internally invokes
Python either through a 'python' command or through a file with a Python
extension ? And how would you find out if a script did this or not ? Are
going to search every script in every distribution and library to
determine if it does this ? And when you find out a script does this,
what will you do ?

Be real. saying you do not like scripts that internally invoke Python
does not solve anything if you have multiple coexisting versions of
Python installed.

 

[Thomas Jollans]

It appears to be only for Linux.

I don't know where you get that impression from. I don't know how well
it works on which platforms, but the fact that there is a "Note for
Windows:" does suggest that it does work on windows.

 

[Thomas Jollans]

Whether it is badly written or not in your opinion it is legal and
happens all the time. Are you going to refuse to use any script, no
matter for what library or for what purpose, that internally invokes
Python either through a 'python' command or through a file with a Python
extension ? And how would you find out if a script did this or not ? Are
going to search every script in every distribution and library to
determine if it does this ? And when you find out a script does this,
what will you do ?

Be real. saying you do not like scripts that internally invoke Python
does not solve anything if you have multiple coexisting versions of
Python installed.

I doubt many scripts do it. The fact of the matter is: many systems have
multiple Python versions installed in parallel, and it probably will
break somewhere, which will get noticed, and probably fixed.

If a script uses sys.executable instead of "python", there is no
problem, at all.

 

[Edward Diener]

I'm not sure I understand. The ida is of course, that the file
association would point to the pystarter and that pystarter would
depending on directory / first line of the script
identify the correct executable to be started with.



Perhaps you could once more explain, what your intended solution would be.

How does a 'pystarter' program know where the file's location is which
is being invoked ? As to the first file line this is completely
unrealistic. What are you going to do, alter the first file line of
every script in a Python distribution and every script in every library
installed in a Python distribution ? Sorry, but a less intrusive
solution is much better and much less of a headache to say the least.

My intended solution would be a simple program which understands where
each co-existing Python distribution is installed on a system and what
the "name" of that distribution is. Then you tell the program which
Python distribution should be the current one by its "name", the current
one meaning the distribution which you want to be invoked at any given
time. The program then changes the PATH so that any references to the
Python directory and its subdirectories point to the "name" Python
directory tree, and changes the file associations so that the "name"
Python executables handle the Python associations.

This does have the weakness that I can not use more than one Python
distribution while Python is executing scripts. But I can personally
live with that since I have never encountered a situation where I must
use more than one Python distribution at the same time.

 

[News123]

Am 25.07.2010 21:32, schrieb Thomas Jollans:

sys.executable will not work with scripts converted with py2exe,
as sys.executable will not be the executable of the python interpreter,
but with the main executable's name.

 

[News123]

How does a 'pystarter' program know where the file's location is which
is being invoked ?

the file's location would be somewhere in sys.argv
probably in sys.argv[1].
converting it to an abs path would return a directory which the python
file belongs to.

As to the first file line this is completely
unrealistic. What are you going to do, alter the first file line of
every script in a Python distribution and every script in every library
installed in a Python distribution ? Sorry, but a less intrusive
solution is much better and much less of a headache to say the least.

Well I would at least do it for all of my self written scripts.

It could allow a soft transition from 2.6 to 2.7 to 3.0 without having
to upgrade all scripts at the same time.

My intended solution would be a simple program which understands where
each co-existing Python distribution is installed on a system and what
the "name" of that distribution is. Then you tell the program which
Python distribution should be the current one by its "name", the current
one meaning the distribution which you want to be invoked at any given
time. The program then changes the PATH so that any references to the
Python directory and its subdirectories point to the "name" Python
directory tree, and changes the file associations so that the "name"
Python executables handle the Python associations.


This does have the weakness that I can not use more than one Python
distribution while Python is executing scripts. But I can personally
live with that since I have never encountered a situation where I must
use more than one Python distribution at the same time.

I guess it's rather difficult to find a solution which suits all.

The above minor weakness, that you mention would be a killer for me.

Currently I'm looking for solutions, where I can start python scripts
requireing different python versions at the same time.

Currently I'm staring the scripts manually from two different cmd line
windows with a different path name and an explicit python call,

Thus my idea of having a pystarter with a config file
mentioning which directories (tools) should use which python executable