setup.py install and bdist_egg

[Jasiu]

Hey,

I work at a company where I'm lucky enough to write web apps using
Python and WSGI . We develop more and more stuff in Python and it's
becoming a mess of dependencies, so we thought we would create a
guideline for developers that describes the whole process of deploying
a Python app on a server. Since all of our servers run Debian, all of
our software has to be installed as Debian packages - other
departments that don't use Python (shame on them! ) already follow
this policy.

Here is what we have figured so far:
1) Write your code.
2) Prepare setup.py, and put names of required eggs in there.
3) Write a Debian wrapper that uses CDBS Python class, and put names
of required Debian packages in there.
4) Build Debian package.
5) Done! Deploy on server and have fun .

This guideline already works pretty well for us, but I thought I could
tweak it a little. I want to make the Debian package wrapper as thin
as possible so that if we ever stop using Debian, we will still be
able to deploy all of our software as eggs. This goal turned to be
pretty challenging. I have a few questions.

1) Debian has a debian/dirs file where you can list all the
directories that the package should create, for example those in /var/
log, /var/lib . I already figured out that I can use setup.py
'install_data' keyword for that, even for empty directories. Then I
can use 'setup.py install' command and things work great. But... How
about eggs? I'd like my egg to somehow contain or create such
directories. For example, I have a config file that I want to place
in /etc directory. I can do that using 'setup.py install', but
'setup.py bdist_egg' will just create an egg containing etc/
directory. Can I do something about it? Also, why doesn't an egg
contain empty dirs that I place in install_data?

2) What about file permissions? Can they be set by setup.py? My
software runs as www-data user and so /var/log and /var/lib
directories should have proper owner, group and permissions. I didn't
figure out a way to change permissions using setup.py. In Debian I can
use postinst script for that.

If anything sounds unclear, blame my bad english .

Regards,

Mike

 

[Hyuga]

Hey,

I work at a company where I'm lucky enough to write web apps using
Python and WSGI . We develop more and more stuff in Python and it's
becoming a mess of dependencies, so we thought we would create a
guideline for developers that describes the whole process of deploying
a Python app on a server. Since all of our servers run Debian, all of
our software has to be installed as Debian packages - other
departments that don't use Python (shame on them! ) already follow
this policy.

Here is what we have figured so far:
1) Write your code.
2) Prepare setup.py, and put names of required eggs in there.
3) Write a Debian wrapper that uses CDBS Python class, and put names
of required Debian packages in there.
4) Build Debian package.
5) Done! Deploy on server and have fun .

This guideline already works pretty well for us, but I thought I could
tweak it a little. I want to make the Debian package wrapper as thin
as possible so that if we ever stop using Debian, we will still be
able to deploy all of our software as eggs. This goal turned to be
pretty challenging. I have a few questions.

Glad to see I'm not the only one with this sort of challenge. Though
it sounds like you're already a step ahead of me in that you're
properly maintaining requirements/dependencies for your Python
packages. When I build my Debian packages I make sure that *they*
have the correct dependencies. But my setup.py files do not have all
their dependencies listed, so it's a pain to get everything set up
correctly when not doing it through the Debian package system.

1) Debian has a debian/dirs file where you can list all the
directories that the package should create, for example those in /var/
log, /var/lib . I already figured out that I can use setup.py
'install_data' keyword for that, even for empty directories. Then I
can use 'setup.py install' command and things work great. But... How
about eggs? I'd like my egg to somehow contain or create such
directories. For example, I have a config file that I want to place
in /etc directory. I can do that using 'setup.py install', but
'setup.py bdist_egg' will just create an egg containing etc/
directory. Can I do something about it? Also, why doesn't an egg
contain empty dirs that I place in install_data?

My approach to this has been to use a postinst script to copy files
out of the eggs (or when I'm not installing as an egg, from /usr/
share) to the correct locations in /etc. However, this doesn't work
so well if you're not using a Debian package. But see below:

2) What about file permissions? Can they be set by setup.py? My
software runs as www-data user and so /var/log and /var/lib
directories should have proper owner, group and permissions. I didn't
figure out a way to change permissions using setup.py. In Debian I can
use postinst script for that.

I'm also using a postinst script for configuring my system, but I want
to move away from that. I already have a command-line tool for easily
administering certain aspects of the system. So my plan is to
implement a "deploy" command that creates all the directories with the
correct permissions, generates the config files, and performs all the
configuration my postinst script currently does. Then the postinst
script simply needs to call the deploy command. That way I can take
advantage of debconf without being dependent on it.

If anything sounds unclear, blame my bad english .

Sounds perfectly clear to me!