distutils alternatives?

[Lonnie Princehouse]

I'm in a situation where I need to distribute several interdependent
Python packages. There are good reasons not to combine them all into
one package. Distutils doesn't seem to be able to bundle a
heterogeneous mix of multiple packages and modules, and so I've
currently got people launching three installers in the proper sequence
in order to get software installed, which seems needlessly complicated
and confusing. There must be a better way. What I really want is a
way for the installer to automatically download and install
dependencies...

First, does anyone know of an alternative to distutils that does this?

Second, this is exactly what Gentoo's Portage software does, except
for Gentoo ebuilds instead of Python packages. It also happens to be
written in Python. Can anyone more familiar with Portage's internals
comment on the feasibility of harnessing emerge to work as a Python
package installer/distributer for systems which aren't running Gentoo?
It is also foreseeable that something like this could be linked to
PyPI, etc.

 

[Robert Kern]

I'm in a situation where I need to distribute several interdependent
Python packages. There are good reasons not to combine them all into
one package. Distutils doesn't seem to be able to bundle a
heterogeneous mix of multiple packages and modules, and so I've
currently got people launching three installers in the proper sequence
in order to get software installed, which seems needlessly complicated
and confusing. There must be a better way. What I really want is a
way for the installer to automatically download and install
dependencies...

The SciPy project has similar needs and extended distutils to do just
that. Check out SciPy CVS at http://www.scipy.org/cvs/ .

The package you're looking for is scipy_core/scipy_distutils . It's a
bit arcane, but it does what you describe and uses distutils as the
base, so you get all the compiler and platform support already there,
and the modifications to existing setup.py scripts should be managable.
I don't think there's much documentation, if any, for using it outside
of SciPy, but it's certainly general.

A few steps to get you started: do a CVS checkout of the scipy_core
directory. Move the scipy_distutils subdirectory out and blast away
scipy_core (which should now only contain stuff you don't need). You can
then distribute scipy_distutils with your packages (as another toplevel
package just under the directory with your uber setup.py that takes care
of all of the packages).

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I'm in a situation where I need to distribute several interdependent
Python packages. There are good reasons not to combine them all into
one package. Distutils doesn't seem to be able to bundle a
heterogeneous mix of multiple packages and modules, and so I've
currently got people launching three installers in the proper sequence
in order to get software installed, which seems needlessly complicated
and confusing.

What do you mean by "launch installers"? Invoking "python setup.py
install" does not "launch" anything in the traditional sense.

However, if this is really what you talk about, I suggest a shell script:

#!/bin/sh
(cd dir1;python setup.py install)
(cd dir2;python setup.py install)
(cd dir3;python setup.py install)

Then create a tarball containing all packages in separate
subdirectories, and add a toplevel install script.

Regards,
Martin

 

[David Fraser]

I'm in a situation where I need to distribute several interdependent
Python packages. There are good reasons not to combine them all into
one package. Distutils doesn't seem to be able to bundle a
heterogeneous mix of multiple packages and modules, and so I've
currently got people launching three installers in the proper sequence
in order to get software installed, which seems needlessly complicated
and confusing. There must be a better way. What I really want is a
way for the installer to automatically download and install
dependencies...

What we tend to do is alter the setup script of the packages that we
want to include in our package, so that it can be imported into our
package and run from there. distutils is unfortunately tricky sometimes,
but with most packages this has been possible.
- define variables in the packagesetup script for all the bits that need
to be included
- in the packagesetup script, do if __name__ == '__main__': setup(...)
- in the main setup script, import the other package setup scripts you need
- include the neccesary bits into the right place.
It would be really nice if distutils were built to do this kind of thing
out of the box ...

First, does anyone know of an alternative to distutils that does this?

Second, this is exactly what Gentoo's Portage software does, except
for Gentoo ebuilds instead of Python packages. It also happens to be
written in Python. Can anyone more familiar with Portage's internals
comment on the feasibility of harnessing emerge to work as a Python
package installer/distributer for systems which aren't running Gentoo?
It is also foreseeable that something like this could be linked to
PyPI, etc.

This would be fantastic.
An important issue to resolve would be to link into the native package
management system on each platform (Windows - MSI, various for other
platforms). You want to be able to have a single installer that installs
multiple native packages, and an uninstaller that can intelligently
determine dependencies and uninstall them.

David

 

[John Roth]

I believe that there was some thought at one time
to doing something like this. It eventually evolved
into PyPI as a first step. I haven't heard of anyone
working on the next step, though.

Distutils does what it does (and I won't say
adequately - I had to write a program to create
the distutils setup.py for PyFIT). I'd think that
putting a separate layer on top to handle multiple
downloads and installs would be a better approach
than attempting to integrate it all into distutils.

John Roth

 

[Lonnie Princehouse]

What do you mean by "launch installers"? Invoking "python setup.py

install" does not "launch" anything in the traditional sense.

The installers I refer to are the graphical Windows installers created
by bdist_wininst. Can't really chain them together in a script
easily, since they're interactive. I already have a nice install
script for *nix, but it requires a little more expertise on the user's
part (i.e. how to competently use a command line)

The packages aren't distributed as source on Windows, partially for
intellectual property reasons, but mostly because there are C
extensions and we can't assume Windows users have a compiler. Before
I start writing my own installer gadget, I thought I should go fishing
on c.l.py to see if such a thing already existed =)