ubuntu dist-packages

Discussion in 'Python' started by Robin Becker, Aug 26, 2009.

  1. Robin Becker

    Robin Becker Guest

    I was surprised a couple of days ago when trying to assist a colleage with his
    python setup on a ubuntu 9.04 system.

    We built our c-extensions and manually copied them into place, but site-packages
    wasn't there. It seems that ubuntu now wants stuff to go into
    lib/python2.6/dist-packages.

    What is the relation between dist-packages/site-packages if any? Is this just a
    name change or is there some other problem being addressed?

    For developers is it best just to create one's own private installations from
    the original tarballs?
    --
    Robin Becker
    Robin Becker, Aug 26, 2009
    #1
    1. Advertising

  2. Robin Becker wrote:

    > I was surprised a couple of days ago when trying to assist a colleage with
    > his python setup on a ubuntu 9.04 system.
    >
    > We built our c-extensions and manually copied them into place, but
    > site-packages wasn't there. It seems that ubuntu now wants stuff to go
    > into lib/python2.6/dist-packages.
    >
    > What is the relation between dist-packages/site-packages if any? Is this
    > just a name change or is there some other problem being addressed?
    >
    > For developers is it best just to create one's own private installations
    > from the original tarballs?


    I don't know much about the atrocities distributions commit to
    python-installations (ripping out distutils "because it's a developer-only
    thing", trying to save a few bytes here and there by unifying
    install-locations between interpreter versions and whatnot), but I think
    the main problem is that you don't use distutils - or setuptools (that gets
    often a bad rap even if for most cases it works flawless)

    If you'd use that to build & install your extension, it would figure out
    where to place the resulting package (or egg) by itself.

    Diez
    Diez B. Roggisch, Aug 26, 2009
    #2
    1. Advertising

  3. Robin Becker

    Jorgen Grahn Guest

    On Wed, 26 Aug 2009 12:46:13 +0200, Diez B. Roggisch
    <> wrote:
    > Robin Becker wrote:
    >
    >> I was surprised a couple of days ago when trying to assist a colleage with
    >> his python setup on a ubuntu 9.04 system.
    >>
    >> We built our c-extensions and manually copied them into place, but
    >> site-packages wasn't there. It seems that ubuntu now wants stuff to go
    >> into lib/python2.6/dist-packages.

    ....

    > I don't know much about the atrocities distributions commit to
    > python-installations (ripping out distutils "because it's a developer-only
    > thing",


    Well, if you are thinking about Debian Linux, it's not as much
    "ripping out" as "splitting into a separate package with a non-obvious
    name". Annoying at times, but hardly an atrocity.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
    Jorgen Grahn, Aug 26, 2009
    #3
  4. Robin Becker

    Robin Becker Guest

    Jorgen Grahn wrote:
    > On Wed, 26 Aug 2009 12:46:13 +0200, Diez B. Roggisch

    .........
    > Well, if you are thinking about Debian Linux, it's not as much
    > "ripping out" as "splitting into a separate package with a non-obvious
    > name". Annoying at times, but hardly an atrocity.

    so where is the official place for user installed stuff on ubuntu/debian ie will
    there be dist-packages and site-packages.
    --
    Robin Becker
    Robin Becker, Aug 26, 2009
    #4
  5. Robin Becker

    Robin Becker Guest

    Jorgen Grahn wrote:
    > On Wed, 26 Aug 2009 12:46:13 +0200, Diez B. Roggisch

    .........
    > Well, if you are thinking about Debian Linux, it's not as much
    > "ripping out" as "splitting into a separate package with a non-obvious
    > name". Annoying at times, but hardly an atrocity.

    so where is the official place for user installed stuff on ubuntu/debian ie will
    there be dist-packages and site-packages.
    --
    Robin Becker
    Robin Becker, Aug 26, 2009
    #5
  6. Robin Becker <> writes:

    > I was surprised a couple of days ago when trying to assist a colleage
    > with his python setup on a ubuntu 9.04 system.
    >
    > We built our c-extensions and manually copied them into place, but
    > site-packages wasn't there. It seems that ubuntu now wants stuff to go
    > into lib/python2.6/dist-packages.
    >
    > What is the relation between dist-packages/site-packages if any? Is
    > this just a name change or is there some other problem being
    > addressed?
    >
    > For developers is it best just to create one's own private
    > installations from the original tarballs?


    From /usr/lib/python2.6/site.py:

    ,----
    | For Debian and derivatives, this sys.path is augmented with directories
    | for packages distributed within the distribution. Local addons go
    | into /usr/local/lib/python<version>/dist-packages, Debian addons
    | install into /usr/{lib,share}/python<version>/dist-packages.
    | /usr/lib/python<version>/site-packages is not used.
    `----




    Florian
    --
    <http://www.florian-diesch.de/software/pdfrecycle/>
    Florian Diesch, Aug 26, 2009
    #6
  7. Robin Becker

    Jorgen Grahn Guest

    On Wed, 26 Aug 2009 17:20:35 +0100, Robin Becker <> wrote:
    > Jorgen Grahn wrote:
    >> On Wed, 26 Aug 2009 12:46:13 +0200, Diez B. Roggisch

    > ........
    >> Well, if you are thinking about Debian Linux, it's not as much
    >> "ripping out" as "splitting into a separate package with a non-obvious
    >> name". Annoying at times, but hardly an atrocity.


    > so where is the official place for user installed stuff on
    > ubuntu/debian ie will
    > there be dist-packages and site-packages.


    I don't know, but I know Debian has a group of people working out how
    to package Python and software written in Python. Those guys have a
    home page somewhere at debian.org -- there should be information
    there, and/or under /usr/share/doc/python* on your system.

    Another answer is "let distutils do its job and do not worry", but
    sometimes you need to know the rules and the reasons ...

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
    Jorgen Grahn, Aug 26, 2009
    #7
  8. Robin Becker

    Paul Boddie Guest

    On 26 Aug, 17:48, Jorgen Grahn <> wrote:
    >
    > Well, if you are thinking about Debian Linux, it's not as much
    > "ripping out" as "splitting into a separate package with a non-obvious
    > name". Annoying at times, but hardly an atrocity.


    Indeed. Having seen two packages today which insisted on setuptools,
    neither really needing it, and with one actively trying to download
    stuff from the Internet (fifteen seconds warning - how generous!) when
    running setup.py, it seems to me that it isn't the distribution
    packagers who need to be re-thinking how they install Python software.

    Generally, distributions have to manage huge amounts of software and
    uphold reasonable policies without creating unnecessary maintenance.
    Sadly, until very recently (and I'm still not entirely sure if there's
    really been an attitude change) the Pythonic packaging brigade has
    refused to even consider the needs of one of the biggest groups of
    consumers of the upstream code. Consequently, distributions will
    always devise different ways of storing installed Python software,
    documentation and resources, mostly because the Pythonic tools have
    been deficient, particularly in the management of the latter
    categories.

    Paul
    Paul Boddie, Aug 26, 2009
    #8
  9. Robin Becker

    Robin Becker Guest

    Florian Diesch wrote:
    ..........
    >
    >>From /usr/lib/python2.6/site.py:

    >
    > ,----
    > | For Debian and derivatives, this sys.path is augmented with directories
    > | for packages distributed within the distribution. Local addons go
    > | into /usr/local/lib/python<version>/dist-packages, Debian addons
    > | install into /usr/{lib,share}/python<version>/dist-packages.
    > | /usr/lib/python<version>/site-packages is not used.
    > `----


    the above is not present in my windows documentation (or indeed site.py) at all
    so it seems they just decided to change the name. Anyone trying to debug why
    their distutils or setuptools or whichever python packager is going wrong will
    have yet another detail to remember. In addition, as any one who has done such
    trivial changes will already know, they forgot to do it globally eg my 0.4.1.0
    version of the "Debian Python Policy" document explicitly mentions site-packages.
    --
    Robin Becker
    Robin Becker, Aug 27, 2009
    #9
  10. Robin Becker

    Robin Becker Guest

    Florian Diesch wrote:
    ..........
    >
    >>From /usr/lib/python2.6/site.py:

    >
    > ,----
    > | For Debian and derivatives, this sys.path is augmented with directories
    > | for packages distributed within the distribution. Local addons go
    > | into /usr/local/lib/python<version>/dist-packages, Debian addons
    > | install into /usr/{lib,share}/python<version>/dist-packages.
    > | /usr/lib/python<version>/site-packages is not used.
    > `----


    the above is not present in my windows documentation (or indeed site.py) at all
    so it seems they just decided to change the name. Anyone trying to debug why
    their distutils or setuptools or whichever python packager is going wrong will
    have yet another detail to remember. In addition, as any one who has done such
    trivial changes will already know, they forgot to do it globally eg my 0.4.1.0
    version of the "Debian Python Policy" document explicitly mentions site-packages.
    --
    Robin Becker
    Robin Becker, Aug 27, 2009
    #10
  11. Paul Boddie wrote:

    > On 26 Aug, 17:48, Jorgen Grahn <> wrote:
    >>
    >> Well, if you are thinking about Debian Linux, it's not as much
    >> "ripping out" as "splitting into a separate package with a non-obvious
    >> name". Annoying at times, but hardly an atrocity.

    >
    > Indeed. Having seen two packages today which insisted on setuptools,
    > neither really needing it, and with one actively trying to download
    > stuff from the Internet (fifteen seconds warning - how generous!) when
    > running setup.py, it seems to me that it isn't the distribution
    > packagers who need to be re-thinking how they install Python software.
    >
    > Generally, distributions have to manage huge amounts of software and
    > uphold reasonable policies without creating unnecessary maintenance.
    > Sadly, until very recently (and I'm still not entirely sure if there's
    > really been an attitude change) the Pythonic packaging brigade has
    > refused to even consider the needs of one of the biggest groups of
    > consumers of the upstream code. Consequently, distributions will
    > always devise different ways of storing installed Python software,
    > documentation and resources, mostly because the Pythonic tools have
    > been deficient, particularly in the management of the latter
    > categories.


    You mean it's the problem of the python packaging that it can't deal with
    RPMs, debs, tgzs, OSX bundles, MSIs and
    <put-in-the-next-big-packaging-thing-here>?

    Multiplied by the various packaging philosophies the respective distros
    build based on these have?

    I'm a Python-developer. I develop libraries and tools for Python, and want
    others to be able to install these - as I want to install things *other*
    python developers created.

    Setuptools let's me do that (most of the time. And I mean most).

    If somebody thinks he wants to include these in whatever form he prefers -
    fine with me. But it's hardly *my* problem, or that of the Python world in
    general, to fulfill the requirements some other people come up with.


    Diez
    Diez B. Roggisch, Aug 27, 2009
    #11
  12. Robin Becker

    Paul Boddie Guest

    On 27 Aug, 15:27, "Diez B. Roggisch" <> wrote:
    >
    > You mean it's the problem of the python packaging that it can't deal with
    > RPMs, debs, tgzs, OSX bundles, MSIs and
    > <put-in-the-next-big-packaging-thing-here>?


    No, it's the problem of the Pythonic packaging brigade that package
    retrieval, building and installing is combined into one unsatisfactory
    whole. Certainly, it's annoying to discover when building some
    extension that the Python headers are missing, but the rhetorical
    vehicle used in the Python community is to frame the distributions as
    people who like to move stuff around unnecessarily and to corrupt
    other people's work. In fact, the distributions have proven themselves
    to be quite competent at managing huge numbers of software packages in
    a semi-automated fashion, so it's baffling that people doing work on
    Pythonic packaging tools wouldn't want to learn as much as they can
    about how those people manage it.

    For me, when making Debian packages, it has become easier to use the
    debhelper stuff to install things like documentation and resources
    than it is to figure out which special options have to be passed to
    the particular distutils incarnation of the setup function in order to
    get such stuff put in the right place, especially since distutils
    probably still employs an ad-hoc 1990s proprietary UNIX "oh just dump
    that stuff in some directory or other and forget about it" mentality.
    Really, distutils should be all about *dist*ribution and even then get
    out of the way as much as possible - the hard stuff is extracting the
    appropriate knowledge about the built Python interpreter in order to
    build extensions. Installation should be left to something which has
    an opinion about where things should be placed on a particular system,
    and which has the capability to know how to uninstall stuff - a
    capability which never seems to get any closer in the distutils scene.

    Paul
    Paul Boddie, Aug 27, 2009
    #12
  13. On Thu, Aug 27, 2009 at 8:27 AM, Diez B. Roggisch<> wrote:
    > Paul Boddie wrote:
    >
    >> On 26 Aug, 17:48, Jorgen Grahn <> wrote:
    >>>
    >>> Well, if you are thinking about Debian Linux, it's not as much
    >>> "ripping out" as "splitting into a separate package with a non-obvious
    >>> name". Annoying at times, but hardly an atrocity.

    >>
    >> Indeed. Having seen two packages today which insisted on setuptools,
    >> neither really needing it, and with one actively trying to download
    >> stuff from the Internet (fifteen seconds warning - how generous!) when
    >> running setup.py, it seems to me that it isn't the distribution
    >> packagers who need to be re-thinking how they install Python software.
    >>
    >> Generally, distributions have to manage huge amounts of software and
    >> uphold reasonable policies without creating unnecessary maintenance.
    >> Sadly, until very recently (and I'm still not entirely sure if there's
    >> really been an attitude change) the Pythonic packaging brigade has
    >> refused to even consider the needs of one of the biggest groups of
    >> consumers of the upstream code. Consequently, distributions will
    >> always devise different ways of storing installed Python software,
    >> documentation and resources, mostly because the Pythonic tools have
    >> been deficient, particularly in the management of the latter
    >> categories.

    >
    > You mean it's the problem of the python packaging that it can't deal with
    > RPMs, debs, tgzs, OSX bundles, MSIs and
    > <put-in-the-next-big-packaging-thing-here>?


    Of course it is - not because distutils should know about them but on
    the contrary because it should be possible to tweak the installation
    parameters to accomodate the various packaging solutions. Autotools,
    cmake, etc... do not need anything about rpm, debian, msi, and yet,
    they can be used to that purpose.

    cheers,

    David
    David Cournapeau, Aug 27, 2009
    #13
  14. Robin Becker <> writes:

    > Florian Diesch wrote:
    > .........
    >>
    >>>From /usr/lib/python2.6/site.py:

    >>
    >> ,----
    >> | For Debian and derivatives, this sys.path is augmented with directories
    >> | for packages distributed within the distribution. Local addons go
    >> | into /usr/local/lib/python<version>/dist-packages, Debian addons
    >> | install into /usr/{lib,share}/python<version>/dist-packages.
    >> | /usr/lib/python<version>/site-packages is not used.
    >> `----

    >
    > the above is not present in my windows documentation (or indeed
    > site.py) at all so it seems they just decided to change the
    > name. Anyone trying to debug why their distutils or setuptools or
    > whichever python packager is going wrong will have yet another detail
    > to remember.


    setuptools works fine for me in Ubuntu 9.04; eggs go to
    /usr/local/lib/python2.6/dist-packages/ like they should according to site.py

    > In addition, as any one who has done such trivial changes
    > will already know, they forgot to do it globally eg my 0.4.1.0 version
    > of the "Debian Python Policy" document explicitly mentions
    > site-packages.


    It's a change for the Python 2.6 package, see
    <http://packages.debian.org/changelogs/pool/main/p/python2.6/python2.6_2.6.2-2/changelog>
    and /usr/lib/python2.5/site.py

    Python 2.6 is only in Debian experimental, I guess there is a new
    version of the Debian Python Policy coming.



    Florian
    --
    <http://www.florian-diesch.de/software/pdfrecycle/>
    Florian Diesch, Aug 27, 2009
    #14
  15. Robin Becker

    David Lyon Guest

    On Thu, 27 Aug 2009 07:05:51 -0700 (PDT), Paul Boddie <>
    wrote:
    > No, it's the problem of the Pythonic packaging brigade that package
    > retrieval, building and installing is combined into one unsatisfactory
    > whole.


    Brigade? That implies a disciplined and systematic approach..

    I would suggest a better term is just guerrilla fighters.

    There's no central planning.. no looking out for the greater good..

    No sense of future.. forgive my pessimism..

    > In fact, the distributions have proven themselves
    > to be quite competent at managing huge numbers of software packages in
    > a semi-automated fashion, so it's baffling that people doing work on
    > Pythonic packaging tools wouldn't want to learn as much as they can
    > about how those people manage it.


    That's a serious and true allegation. I think the answer is that the
    people that are running it aren't interested in cleaning things up
    for the linux or windows platform.

    Consensus seems to be just "remove" anything that relates to windows
    or linux as an approach to harmonising it.

    I believe that it should be harmonised.


    > For me, when making Debian packages, it has become easier to use the
    > debhelper stuff to install things like documentation and resources
    > than it is to figure out which special options have to be passed to
    > the particular distutils incarnation of the setup function in order to
    > get such stuff put in the right place, especially since distutils
    > probably still employs an ad-hoc 1990s proprietary UNIX "oh just dump
    > that stuff in some directory or other and forget about it" mentality.


    It doesn't. Actually, it can't even do that very well.

    90's.... which '90s ? 1890's ?

    > Really, distutils should be all about *dist*ribution and even then get
    > out of the way as much as possible - the hard stuff is extracting the
    > appropriate knowledge about the built Python interpreter in order to
    > build extensions.


    Exactly.

    > Installation should be left to something which has
    > an opinion about where things should be placed on a particular system,
    > and which has the capability to know how to uninstall stuff - a
    > capability which never seems to get any closer in the distutils scene.


    (sigh) - it is true...

    David
    David Lyon, Aug 31, 2009
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. SPE - Stani's Python Editor
    Replies:
    1
    Views:
    440
    SPE - Stani's Python Editor
    Apr 29, 2008
  2. Jerzy Jalocha N
    Replies:
    1
    Views:
    564
    Diez B. Roggisch
    Nov 20, 2008
  3. Jerzy Jalocha N
    Replies:
    0
    Views:
    835
    Jerzy Jalocha N
    Nov 25, 2008
  4. Replies:
    2
    Views:
    311
  5. Jason Friedman
    Replies:
    0
    Views:
    138
    Jason Friedman
    Dec 27, 2012
Loading...

Share This Page