Sharing Python installation between architectures

P

Paul Smith

One thing I always liked about Perl was the way you can create a single
installation directory which can be shared between archictures. Say
what you will about the language: the Porters have an enormous amount of
experience and expertise producing portable and flexible interpreter
installations.

By this I mean, basically, multiple architectures (Linux, Solaris,
MacOSX, even Windows) sharing the same $prefix/lib/python2.7 directory.
The large majority of the contents there are completely portable across
architectures (aren't they?) so why should I have to duplicate many
megabytes worth of files?

The only parts of the install which are not shareable (as far as I can
tell) are the .so dynamic objects (and the python executable itself
obviously).

If the default sys.path included platform-specific directories as well
as the generic lib-dynload, it would be possible.


I do see that there are "plat-*" directories available in the default
path. Is it possible to make use of these (say, by renaming each
architecture's lib-dynload to the appropriate plat-* name)?


If that works, the remaining issue is the site-packages directory.
There is no ability (that I can see) to separate out the shareable vs.
non-sharable aspects of the add-on site-packages.


Any comments or suggestions? Am I overestimating the amount of sharing
that's possible? Thanks!
 
A

Albert van der Horst

One thing I always liked about Perl was the way you can create a single
installation directory which can be shared between archictures. Say
what you will about the language: the Porters have an enormous amount of
experience and expertise producing portable and flexible interpreter
installations.

By this I mean, basically, multiple architectures (Linux, Solaris,
MacOSX, even Windows) sharing the same $prefix/lib/python2.7 directory.
The large majority of the contents there are completely portable across
architectures (aren't they?) so why should I have to duplicate many
megabytes worth of files?

The solution is of course to replace all duplicates by hard links.
A tool for this is useful in a lot of other circumstances too.
In a re-installation of the whole or parts, the hard links
will be removed, and the actual files are only removed if they aren't needed
for any of the installations, so this is transparent for reinstallation.
After a lot of reinstallation you want to run the tool again.

This is of course only possible on real file systems (probably not on FAT),
but your files reside on a server, so chances are they are on a real file
system.

(The above is partly in jest. It is a real solution to storage problems,
but storage problems are unheard of in these days of Tera byte disks.
It doesn't help with the clutter, which was probably the main motivation.)

Symbolic links are not as transparent, but they may work very well too.
Have the common part set apart and replace everything else by symbolic links.

There is always one more way to skin a cat.

Groetjes Albert
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top