MySQLdb install vs. "setuptools"

[John Nagle]

MySQLdb won't install as non-root on Python 2.6 because
its "setup.py" file requires "setuptools". "setuptools",
unlike "distutils", isn't part of the Python 2.6 distribution.

IMPORTANT PACKAGES SHOULD NOT USE "setuptools". Use the
standard "distutils". "setuptools" and "eggs" create more
problems than they solve. "setuptools" has many built-in
assumptions about where things are supposed to be, and they're
usually wrong.

There's an "ez_setup.py" available for MySQLdb. This tries to obtain and
install "setuptools". But it wants too many privileges just to do a "build":

The following error occurred while trying to add or remove files in the
installation directory:

[Errno 2] No such file or directory:
'/usr/local/lib/python2.6/site-packages/test-easy-install-22015.write-test'

That's because "setuptools" has a built-in ego trip of insisting that
it should be installed globally. I'm running a freshly-built, but not
"installed" version of Python 2.6. I'm trying to install a test version
of Python2.6 with some packages without running as root on a live server.

It's not just MySQLdb that's broken by "setuptools". See these
examples of other installation problems created by "setuptools":

http://blog.awarelabs.com/2008/installing-mysqldb-without-python-egg-problems/
http://mail.python.org/pipermail/pythonmac-sig/2009-March/021148.html
http://plone.org/documentation/error/permission-denied-python-eggs

As soon as "setuptools" becomes involved, unnecessary headaches appear.
DO NOT USE SETUPTOOLS. Either use "distutils", or go all the way and
provide a RPM for Linux and a .exe for Windows.

John Nagle

 

[Antoine Pitrou]

Really, this shouldn't happen if you really are using a
non-root version of Python:

[Errno 2] No such file or directory:
'/usr/local/lib/python2.6/site-packages/test-easy-install-22015.write-test'

I don't think setuptools is dumb enough to hardcode things like
"/usr/local/lib/python2.6/", so the error is probably yours here.
Perhaps you should double-check you did everything fine before posting
such a rant.

Of course, if by "freshly-built version of Python", you mean you didn't
run "make install" in any way, then it's your problem. Give
"./configure" an appropriate non-root prefix and don't forget to run
"make install" at the end.

 

[David Cournapeau]

  MySQLdb won't install as non-root on Python 2.6 because
its "setup.py" file requires "setuptools".  "setuptools",
unlike "distutils", isn't part of the Python 2.6 distribution.

  IMPORTANT PACKAGES SHOULD NOT USE "setuptools".  Use the
standard "distutils".  "setuptools" and "eggs" create more
problems than they solve.  "setuptools" has many built-in
assumptions about where things are supposed to be, and they're
usually wrong.

Yes, setuptools is often a pain.

The magic incantation you want is something like python setup.py
install --prefix=someprefix --single-version-externally-managed
--record=/dev/null. I myself have a small script which detects whether
setup.py uses setuptools or not, and add the necessary options in
setuptools case for a sane behavior.

David

 

[John Nagle]

Yes, setuptools is often a pain.

The magic incantation you want is something like python setup.py
install --prefix=someprefix --single-version-externally-managed
--record=/dev/null. I myself have a small script which detects whether
setup.py uses setuptools or not, and add the necessary options in
setuptools case for a sane behavior.

David

The "setup.py" file for MySQLdb has, as its first Python
line, "from setuptools import ...". No combination of options will
get around that.

John Nagle

 

[John Nagle]

Really, this shouldn't happen if you really are using a
non-root version of Python:

[Errno 2] No such file or directory:
'/usr/local/lib/python2.6/site-packages/test-easy-install-22015.write-test'

I don't think setuptools is dumb enough to hardcode things like
"/usr/local/lib/python2.6/", so the error is probably yours here.
Perhaps you should double-check you did everything fine before posting
such a rant.

Of course, if by "freshly-built version of Python", you mean you didn't
run "make install" in any way, then it's your problem. Give
"./configure" an appropriate non-root prefix and don't forget to run
"make install" at the end.

Actually, a "built" but "uninstalled" Python works fine. If it
didn't, "make test" wouldn't work. "sys.path" correctly points to the
library directories created during "build".

On the other hand, options to "./configure" apparently don't work
right in Python 2.6 through 3.x. "--libdir" and "--bindir" don't actually
do anything. See "http://bugs.python.org/issue858809" (an open bug). So custom
"configure" is currently broken.

The real problem here remains the unnecessary use of "setuptools".
It's Debian distro policy not to use "setuptools":

http://www.debian.org/doc/packaging-manuals/python-policy/ap-packaging_tools.html

Also read "Setuptools is not a decent software package management".

http://workaround.org/easy-install-debian

The fundamental problem is that "setuptools" is more than an installer but
less than a widely-supported system-wide package manager like "yum".

Now, how to get the dependency on "setuptools" out of MySQLdb?

John Nagle

 

[John Nagle]

I tried this change on MySQLdb's "setup.py":

diff setup.py setup-nodistutils.py
5c5
< from setuptools import setup, Extension
---

> from distutils.core import setup, Extension

The build then runs. The resulting MySQLdb runs under the uninstalled
Python and connects to the database properly.

There's no need for "setuptools" here at all. It just gets in the way.

John Nagle

 

[Antoine Pitrou]

Actually, a "built" but "uninstalled" Python works fine.
If it
didn't, "make test" wouldn't work.

That's a completely unrelated thing. The main reason "make test" works
with an uninstalled Python is simply so that the core developers' life
is easier. It doesn't mean all Python functionalities work in that
context.

On the other hand, options to "./configure" apparently don't work
right in Python 2.6 through 3.x. "--libdir" and "--bindir" don't actually
do anything.

You can use "--prefix" instead, it works.

The real problem here remains the unnecessary use of "setuptools".

No, again, the real problem is just that you are trying to install
modules for an uninstalled Python.
You may hate setuptools with a passion, but it's unconstructive and
useless to put the blame on it without any solid argument.
Do yourself a favour: pass the proper options to "./configure", and
install Python.

Regards

Antoine.

 

[Antoine Pitrou]

It's nice that some of the options work. Note that someone who
used "--bindir", expecting it to work, might end up overwriting their
existing Python installation unintentionally, which would break system
administration tools like cPanel and "yum".

Well, usually you don't type "sudo" unintentionally...

The ongoing low quality of Python distribution mechanisms, and the
denial of that fact, is a major part of why Python, after 20 years, is far
less available than Perl.

"Far less available"? How so?

The latest production versions of Red Hat
Enterprise Linux and CentOS, the major server distributions, still ship
with Python 2.4.3, a five year old version of Python.

And of course nothing in this is Python's or setuptools' fault, since
it's just Redhat's policy, so I wonder what you're trying to tell us.

(not to mention that it doesn't have anything to do with the original
topic anymore, either)

 

[John Nagle]

That's a completely unrelated thing. The main reason "make test" works
with an uninstalled Python is simply so that the core developers' life
is easier. It doesn't mean all Python functionalities work in that
context.
>

You can use "--prefix" instead, it works.

It's nice that some of the options work. Note that someone who
used "--bindir", expecting it to work, might end up overwriting their
existing Python installation unintentionally, which would break system
administration tools like cPanel and "yum".

cPanel support recommends against installing a new Python other
than through "yum".

http://forums.cpanel.net/f5/mailman-breaks-stable-upcp-due-python-upgrade-126453.html
http://forums.cpanel.net/f5/upgrade-python-whm-113593.html

They don't trust other install mechanisms. With good cause.

No, again, the real problem is just that you are trying to install
modules for an uninstalled Python.
You may hate setuptools with a passion, but it's unconstructive and
useless to put the blame on it without any solid argument.
Do yourself a favour: pass the proper options to "./configure", and
install Python.

Regardssetu

Antoine.

Having demonstrated in a previous post that 1) the use of "setuptools"
was completely unnecessary, and 2) it's quite possible to load and use
MySQLdb in an "uninstalled" Python, we can dismiss the above argument.

The ongoing low quality of Python distribution mechanisms, and the
denial of that fact, is a major part of why Python, after 20 years, is far
less available than Perl. The latest production versions of Red Hat
Enterprise Linux and CentOS, the major server distributions, still ship
with Python 2.4.3, a five year old version of Python.

John Nagle

 

[David Cournapeau]

   It's nice that some of the options work.  Note that someone who
used "--bindir", expecting it to work, might end up overwriting their
existing Python installation unintentionally, which would break system
administration tools like cPanel and "yum".

   cPanel support recommends against installing a new Python other
than through "yum".

http://forums.cpanel.net/f5/mailman-breaks-stable-upcp-due-python-upgrade-126453.html
http://forums.cpanel.net/f5/upgrade-python-whm-113593.html

   They don't trust other install mechanisms.  With good cause.

This has absolutely nothing to do with how python is installed. It is
common sense that you should upgrade an installed package through the
package manager mechanism (here yum), and is true for any software,
python or not.

 The latest production versions of Red Hat
Enterprise Linux and CentOS, the major server distributions, still ship
with Python 2.4.3, a five year old version of Python.

Yes, by definition RHEL ships softwares that does not change for a
long time. By your argument, the linux kernel and gcc are broken
because they are 4 years old on RHEL 5 (linux 2.6.18. gcc 4.0.*). That
does not make any sense.

cheers,

David