distutils (packaging) newbie questions please....

  • Thread starter Christian Seberino
  • Start date
C

Christian Seberino

I really want to learn distutils to professionally package some open
source software.

I had some distutils questions after reading python.org distutil docs
on packaging code
if you don't mind. I would really appreciate some help.....


0. I'm confused about use of the word "package" in setup.py.
packages = ["<package1>", "<package2>", ...]
Does package here just mean "a new directory of source code to
consider"???

1. What exactly goes into the __init__.py files? I know what to put in
setup.py
and I know __init__.py is used for packages but I don't know what
to put IN this file.

2. About the data_files option for setup.py....

data_files = [ "<installation directory>", [ "<file1>",
"<file2>", ...] ]

Is there a way to AVOID having to type EVERY single data file?? Can
I specify
just the directory or use wild cards??

3. python setup.py sdist will make a package of SOURCE code and
using "bdist" will put BINARIES in package. What does this
mean???
Will bdist packages unpack to pyc (bytecode) files and sdist will
unpack
py (noncompiled) files???? What if I wanted py and pyc files
installed
from the package?? (i.e. source and compiled versions)

Thanks!

Chris
 
M

Mark Hahn

I just went through this with a python shareware app
(http://EzPicMailer.com) and after a bit of wandering around lost, I came up
with a packaging combination that worked great for me. I can highly
recommend using:

1) McMillan utility instead of distutils
http://www.mcmillan-inc.com/install1.html

2) Inno setup for actual installer http://www.jrsoftware.org/isinfo.php

After setting up a few .bat files, I can now release in a matter of a minute
or so. Oh, and I didn't have to set up any __init__.py files or anything
like that.
 
G

Greg Krohn

Christian Seberino said:
I really want to learn distutils to professionally package some open
source software.

I had some distutils questions after reading python.org distutil docs
on packaging code
if you don't mind. I would really appreciate some help.....
Click to expand...

I've never used distutil, so I can onlt help a little bit here.
1. What exactly goes into the __init__.py files? I know what to put in
setup.py
and I know __init__.py is used for packages but I don't know what
to put IN this file.
Click to expand...

You don't NEED to put anything in it, it works fine as an empty file. You
CAN put code in there to initialize your package, but personally I've never
had to.
2. About the data_files option for setup.py....

data_files = [ "<installation directory>", [ "<file1>",
"<file2>", ...] ]

Is there a way to AVOID having to type EVERY single data file?? Can
I specify
just the directory or use wild cards??
Click to expand...

Sure, it's just a list of filenames. You can use glob.glob('*.dat') like
this:

data_files = = ['./images', glob.glob('*.png')]


Hopefully someone else can help you with the rest.


greg
 
A

Alex Martelli

Christian said:
I really want to learn distutils to professionally package some open
source software.

I had some distutils questions after reading python.org distutil docs
on packaging code
if you don't mind. I would really appreciate some help.....


0. I'm confused about use of the word "package" in setup.py.
packages = ["<package1>", "<package2>", ...]
Does package here just mean "a new directory of source code to
consider"???
Click to expand...

No, it means a Python package. The concept of "package" (a module
that may contain other modules, and corresponds to a directory with
an __init__.py file) exists in Python quite independently from
distutils -- the distutils just SUPPORT the concept, if the code
you want to package-up uses Python packages.

1. What exactly goes into the __init__.py files? I know what to put in
setup.py
and I know __init__.py is used for packages but I don't know what
to put IN this file.
Click to expand...

Again, not a distutils-related question. A Python package's __init__.py
has the module-body of the package (remember, any package IS a module).
If you need no Python statements whatsoever in that body, __init__.py
may be empty (it must still exist, or else Python will refuse to
consider that directory as a package). Normally you do want, at least,
to assign the appropriate list to __all__ .

In p. 124 of "Python in a Nutshell", i.e. the section named "Packages",
I summarize the issue as follows:
"""
A package named P resides in a subdirectory, also called P, of some
directory in sys.path. The module body of P is in the file P/__init__.py.
You must have a file named P/__init__.py, even if it's empty (representing
an empty module body), in order to indicate to Python that directory P
is indeed a package.
"""

Do you find this unclear, or was it hard for you to find -- or didn't
you even considering checking out the Nutshell book before setting
yourself the task to "professionally package" some Python software?
Feedback on this (private or public as you wish) may help me do a
better job next time. [Note that if the issue is with the purchase
price of the Nutshell book, there IS a perfectly legal way to read it
for free: the "Safari" online book service of O'Reilly offers the
first two weeks free, so you can join it, and read Python in a Nutshell,
which is among the over 1000 books on Safari, paying nothing as long
as you cancel before two weeks are up].

2. About the data_files option for setup.py....

data_files = [ "<installation directory>", [ "<file1>",
"<file2>", ...] ]

Is there a way to AVOID having to type EVERY single data file?? Can
I specify just the directory or use wild cards??
Click to expand...

You can form the list of file names in any way you wish, including,
for example, using the glob module from Python's standard library.
setup.py is a Python script, and any Python construct is available
to you as the author of that script.

For example, suppose we wanted to package up (for installation under
directory foodir) all the files in our directory named datadir EXCEPT
those whose file-extension is '.not' . That's easy...:

"""
from distutils.core import setup

import glob

thefiles = []
for afile in glob.glob('datadir/*'):
if afile.endswith('.not'): continue
thefiles.append(afile)

print "Packaging up", thefiles

setup(name='justdata', data_files = [ [ 'foodir', thefiles ] ] )
"""

Here's an example of how this works:

[alex@lancelot tryp]$ ls datadir
aa.yes bb.not cc.yes
[alex@lancelot tryp]$ python setup.py bdist
Packaging up ['datadir/aa.yes', 'datadir/cc.yes']
running bdist
running bdist_dumb
running build
installing to build/bdist.linux-i686/dumb
running install
running install_data
creating build/bdist.linux-i686/dumb
creating build/bdist.linux-i686/dumb/usr
creating build/bdist.linux-i686/dumb/usr/local
creating build/bdist.linux-i686/dumb/usr/local/foodir
copying datadir/aa.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
copying datadir/cc.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
tar -cf /x/tryp/dist/justdata-0.0.0.linux-i686.tar .
gzip -f9 /x/tryp/dist/justdata-0.0.0.linux-i686.tar
removing 'build/bdist.linux-i686/dumb' (and everything under it)
[alex@lancelot tryp]$ tar tvzf dist/justdata-0.0.0.linux-i686.tar.gz
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/foodir/
-rw-rw-r-- alex/alex 0 2003-10-01 14:13:48 ./usr/local/foodir/aa.yes
-rw-rw-r-- alex/alex 0 2003-10-01 14:13:55 ./usr/local/foodir/cc.yes
[alex@lancelot tryp]$

3. python setup.py sdist will make a package of SOURCE code and
using "bdist" will put BINARIES in package. What does this
mean???
Will bdist packages unpack to pyc (bytecode) files and sdist will
unpack
py (noncompiled) files???? What if I wanted py and pyc files
installed
from the package?? (i.e. source and compiled versions)
Click to expand...

Hmmm, if the issue was so unclear (and important) to you, was it
SO hard to try things out...? Say that setup.py is:

from distutils.core import setup

setup(name='onemodule', py_modules=['hello'])

and hello.py in the same directory is:

def greet(name='World'):
return 'Hello, %s!' % name

Now, running python setup.py bdist gives the output:

[alex@lancelot tryp]$ python setup.py bdist
running bdist
running bdist_dumb
running build
running build_py
creating build/lib
copying hello.py -> build/lib
installing to build/bdist.linux-i686/dumb
running install
running install_lib
creating build/bdist.linux-i686/dumb
creating build/bdist.linux-i686/dumb/usr
creating build/bdist.linux-i686/dumb/usr/local
creating build/bdist.linux-i686/dumb/usr/local/lib
creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3
creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
copying build/lib/hello.py ->
build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
byte-compiling
build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages/hello.py
to hello.pyc
tar -cf /x/tryp/dist/onemodule-0.0.0.linux-i686.tar .
gzip -f9 /x/tryp/dist/onemodule-0.0.0.linux-i686.tar
removing 'build/bdist.linux-i686/dumb' (and everything under it)

Notice that hello.py is copied into the directory to be packaged, as
well as hello,pyc. And indeed we can double check what's in the tarball:

[alex@lancelot tryp]$ tar tvzf dist/onemodule-0.0.0.linux-i686.tar.gz
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/lib/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41
../usr/local/lib/python2.3/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:42
../usr/local/lib/python2.3/site-packages/
-rw-rw-r-- alex/alex 56 2003-10-01 14:30:10
../usr/local/lib/python2.3/site-packages/hello.py
-rw-rw-r-- alex/alex 335 2003-10-01 14:30:42
../usr/local/lib/python2.3/site-packages/hello.pyc
[alex@lancelot tryp]$

See? Both the .py AND the .pyc, as expected.

Should you want to REMOVE the .py files, for example, that can
easily be achieved by operating directly on the tarball or zipfile,
after distutils is finished (customizing distutils for such
purposes, on the other hand, is harder).


Alex
 
C

Christian Seberino

Alex

Your email motivated me to buy your book. I think Nutshell
was a VERY good purchase. Why? Lots of other docs for newbies
don't go into too much detail. Your book is great for the completeness.
Intermediate users need that.... especially if they don't like reading
python.org docs online.

CS


Alex Martelli said:
Christian said:
I really want to learn distutils to professionally package some open
source software.

I had some distutils questions after reading python.org distutil docs
on packaging code
if you don't mind. I would really appreciate some help.....


0. I'm confused about use of the word "package" in setup.py.
packages = ["<package1>", "<package2>", ...]
Does package here just mean "a new directory of source code to
consider"???
Click to expand...

No, it means a Python package. The concept of "package" (a module
that may contain other modules, and corresponds to a directory with
an __init__.py file) exists in Python quite independently from
distutils -- the distutils just SUPPORT the concept, if the code
you want to package-up uses Python packages.

1. What exactly goes into the __init__.py files? I know what to put in
setup.py
and I know __init__.py is used for packages but I don't know what
to put IN this file.
Click to expand...

Again, not a distutils-related question. A Python package's __init__.py
has the module-body of the package (remember, any package IS a module).
If you need no Python statements whatsoever in that body, __init__.py
may be empty (it must still exist, or else Python will refuse to
consider that directory as a package). Normally you do want, at least,
to assign the appropriate list to __all__ .

In p. 124 of "Python in a Nutshell", i.e. the section named "Packages",
I summarize the issue as follows:
"""
A package named P resides in a subdirectory, also called P, of some
directory in sys.path. The module body of P is in the file P/__init__.py.
You must have a file named P/__init__.py, even if it's empty (representing
an empty module body), in order to indicate to Python that directory P
is indeed a package.
"""

Do you find this unclear, or was it hard for you to find -- or didn't
you even considering checking out the Nutshell book before setting
yourself the task to "professionally package" some Python software?
Feedback on this (private or public as you wish) may help me do a
better job next time. [Note that if the issue is with the purchase
price of the Nutshell book, there IS a perfectly legal way to read it
for free: the "Safari" online book service of O'Reilly offers the
first two weeks free, so you can join it, and read Python in a Nutshell,
which is among the over 1000 books on Safari, paying nothing as long
as you cancel before two weeks are up].

2. About the data_files option for setup.py....

data_files = [ "<installation directory>", [ "<file1>",
"<file2>", ...] ]

Is there a way to AVOID having to type EVERY single data file?? Can
I specify just the directory or use wild cards??
Click to expand...

You can form the list of file names in any way you wish, including,
for example, using the glob module from Python's standard library.
setup.py is a Python script, and any Python construct is available
to you as the author of that script.

For example, suppose we wanted to package up (for installation under
directory foodir) all the files in our directory named datadir EXCEPT
those whose file-extension is '.not' . That's easy...:

"""
from distutils.core import setup

import glob

thefiles = []
for afile in glob.glob('datadir/*'):
if afile.endswith('.not'): continue
thefiles.append(afile)

print "Packaging up", thefiles

setup(name='justdata', data_files = [ [ 'foodir', thefiles ] ] )
"""

Here's an example of how this works:

[alex@lancelot tryp]$ ls datadir
aa.yes bb.not cc.yes
[alex@lancelot tryp]$ python setup.py bdist
Packaging up ['datadir/aa.yes', 'datadir/cc.yes']
running bdist
running bdist_dumb
running build
installing to build/bdist.linux-i686/dumb
running install
running install_data
creating build/bdist.linux-i686/dumb
creating build/bdist.linux-i686/dumb/usr
creating build/bdist.linux-i686/dumb/usr/local
creating build/bdist.linux-i686/dumb/usr/local/foodir
copying datadir/aa.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
copying datadir/cc.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
tar -cf /x/tryp/dist/justdata-0.0.0.linux-i686.tar .
gzip -f9 /x/tryp/dist/justdata-0.0.0.linux-i686.tar
removing 'build/bdist.linux-i686/dumb' (and everything under it)
[alex@lancelot tryp]$ tar tvzf dist/justdata-0.0.0.linux-i686.tar.gz
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/
drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/foodir/
-rw-rw-r-- alex/alex 0 2003-10-01 14:13:48 ./usr/local/foodir/aa.yes
-rw-rw-r-- alex/alex 0 2003-10-01 14:13:55 ./usr/local/foodir/cc.yes
[alex@lancelot tryp]$

3. python setup.py sdist will make a package of SOURCE code and
using "bdist" will put BINARIES in package. What does this
mean???
Will bdist packages unpack to pyc (bytecode) files and sdist will
unpack
py (noncompiled) files???? What if I wanted py and pyc files
installed
from the package?? (i.e. source and compiled versions)
Click to expand...

Hmmm, if the issue was so unclear (and important) to you, was it
SO hard to try things out...? Say that setup.py is:

from distutils.core import setup

setup(name='onemodule', py_modules=['hello'])

and hello.py in the same directory is:

def greet(name='World'):
return 'Hello, %s!' % name

Now, running python setup.py bdist gives the output:

[alex@lancelot tryp]$ python setup.py bdist
running bdist
running bdist_dumb
running build
running build_py
creating build/lib
copying hello.py -> build/lib
installing to build/bdist.linux-i686/dumb
running install
running install_lib
creating build/bdist.linux-i686/dumb
creating build/bdist.linux-i686/dumb/usr
creating build/bdist.linux-i686/dumb/usr/local
creating build/bdist.linux-i686/dumb/usr/local/lib
creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3
creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
copying build/lib/hello.py ->
build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
byte-compiling
build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages/hello.py
to hello.pyc
tar -cf /x/tryp/dist/onemodule-0.0.0.linux-i686.tar .
gzip -f9 /x/tryp/dist/onemodule-0.0.0.linux-i686.tar
removing 'build/bdist.linux-i686/dumb' (and everything under it)

Notice that hello.py is copied into the directory to be packaged, as
well as hello,pyc. And indeed we can double check what's in the tarball:

[alex@lancelot tryp]$ tar tvzf dist/onemodule-0.0.0.linux-i686.tar.gz
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/lib/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:41
./usr/local/lib/python2.3/
drwxrwxr-x alex/alex 0 2003-10-01 14:30:42
./usr/local/lib/python2.3/site-packages/
-rw-rw-r-- alex/alex 56 2003-10-01 14:30:10
./usr/local/lib/python2.3/site-packages/hello.py
-rw-rw-r-- alex/alex 335 2003-10-01 14:30:42
./usr/local/lib/python2.3/site-packages/hello.pyc
[alex@lancelot tryp]$

See? Both the .py AND the .pyc, as expected.

Should you want to REMOVE the .py files, for example, that can
easily be achieved by operating directly on the tarball or zipfile,
after distutils is finished (customizing distutils for such
purposes, on the other hand, is harder).


Alex
Click to expand...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

distutils: "build" command 1
generating .zip including __main__.py file in top root using distutils 1
Python packaging usabilty (distutils) - automatic downloadingrequired packages 1
Packaging a private project 1
distutils examples? 1
Advice needed for Python packaging - can't find required libraryduring installation 2
Distutils, bdist_XXX, rpmbuild issues 0
Distutils questions. 0

Members online

No members online now.
Total: 21 (members: 0, guests: 21)
Robots: 328

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,013
Latest member
KatriceSwa

Latest Threads

Top