distutils (packaging) newbie questions please....

Discussion in 'Python' started by Christian Seberino, Oct 1, 2003.

  1. I really want to learn distutils to professionally package some open
    source software.

    I had some distutils questions after reading python.org distutil docs
    on packaging code
    if you don't mind. I would really appreciate some help.....


    0. I'm confused about use of the word "package" in setup.py.
    packages = ["<package1>", "<package2>", ...]
    Does package here just mean "a new directory of source code to
    consider"???

    1. What exactly goes into the __init__.py files? I know what to put in
    setup.py
    and I know __init__.py is used for packages but I don't know what
    to put IN this file.

    2. About the data_files option for setup.py....

    data_files = [ "<installation directory>", [ "<file1>",
    "<file2>", ...] ]

    Is there a way to AVOID having to type EVERY single data file?? Can
    I specify
    just the directory or use wild cards??

    3. python setup.py sdist will make a package of SOURCE code and
    using "bdist" will put BINARIES in package. What does this
    mean???
    Will bdist packages unpack to pyc (bytecode) files and sdist will
    unpack
    py (noncompiled) files???? What if I wanted py and pyc files
    installed
    from the package?? (i.e. source and compiled versions)

    Thanks!

    Chris
     
    Christian Seberino, Oct 1, 2003
    #1
    1. Advertising

  2. Christian Seberino

    Mark Hahn Guest

    I just went through this with a python shareware app
    (http://EzPicMailer.com) and after a bit of wandering around lost, I came up
    with a packaging combination that worked great for me. I can highly
    recommend using:

    1) McMillan utility instead of distutils
    http://www.mcmillan-inc.com/install1.html

    2) Inno setup for actual installer http://www.jrsoftware.org/isinfo.php

    After setting up a few .bat files, I can now release in a matter of a minute
    or so. Oh, and I didn't have to set up any __init__.py files or anything
    like that.

    "Christian Seberino" <> wrote in message
    news:...
    > I really want to learn distutils to professionally package some open
    > source software.
    >
    > I had some distutils questions after reading python.org distutil docs
    > on packaging code
    > if you don't mind. I would really appreciate some help.....
    >
    >
    > 0. I'm confused about use of the word "package" in setup.py.
    > packages = ["<package1>", "<package2>", ...]
    > Does package here just mean "a new directory of source code to
    > consider"???
    >
    > 1. What exactly goes into the __init__.py files? I know what to put in
    > setup.py
    > and I know __init__.py is used for packages but I don't know what
    > to put IN this file.
    >
    > 2. About the data_files option for setup.py....
    >
    > data_files = [ "<installation directory>", [ "<file1>",
    > "<file2>", ...] ]
    >
    > Is there a way to AVOID having to type EVERY single data file?? Can
    > I specify
    > just the directory or use wild cards??
    >
    > 3. python setup.py sdist will make a package of SOURCE code and
    > using "bdist" will put BINARIES in package. What does this
    > mean???
    > Will bdist packages unpack to pyc (bytecode) files and sdist will
    > unpack
    > py (noncompiled) files???? What if I wanted py and pyc files
    > installed
    > from the package?? (i.e. source and compiled versions)
    >
    > Thanks!
    >
    > Chris
     
    Mark Hahn, Oct 1, 2003
    #2
    1. Advertising

  3. Christian Seberino

    Greg Krohn Guest

    "Christian Seberino" <> wrote in message
    news:...
    > I really want to learn distutils to professionally package some open
    > source software.
    >
    > I had some distutils questions after reading python.org distutil docs
    > on packaging code
    > if you don't mind. I would really appreciate some help.....


    I've never used distutil, so I can onlt help a little bit here.

    > 1. What exactly goes into the __init__.py files? I know what to put in
    > setup.py
    > and I know __init__.py is used for packages but I don't know what
    > to put IN this file.


    You don't NEED to put anything in it, it works fine as an empty file. You
    CAN put code in there to initialize your package, but personally I've never
    had to.

    > 2. About the data_files option for setup.py....
    >
    > data_files = [ "<installation directory>", [ "<file1>",
    > "<file2>", ...] ]
    >
    > Is there a way to AVOID having to type EVERY single data file?? Can
    > I specify
    > just the directory or use wild cards??


    Sure, it's just a list of filenames. You can use glob.glob('*.dat') like
    this:

    data_files = = ['./images', glob.glob('*.png')]


    Hopefully someone else can help you with the rest.


    greg
     
    Greg Krohn, Oct 1, 2003
    #3
  4. Christian Seberino wrote:

    > I really want to learn distutils to professionally package some open
    > source software.
    >
    > I had some distutils questions after reading python.org distutil docs
    > on packaging code
    > if you don't mind. I would really appreciate some help.....
    >
    >
    > 0. I'm confused about use of the word "package" in setup.py.
    > packages = ["<package1>", "<package2>", ...]
    > Does package here just mean "a new directory of source code to
    > consider"???


    No, it means a Python package. The concept of "package" (a module
    that may contain other modules, and corresponds to a directory with
    an __init__.py file) exists in Python quite independently from
    distutils -- the distutils just SUPPORT the concept, if the code
    you want to package-up uses Python packages.


    > 1. What exactly goes into the __init__.py files? I know what to put in
    > setup.py
    > and I know __init__.py is used for packages but I don't know what
    > to put IN this file.


    Again, not a distutils-related question. A Python package's __init__.py
    has the module-body of the package (remember, any package IS a module).
    If you need no Python statements whatsoever in that body, __init__.py
    may be empty (it must still exist, or else Python will refuse to
    consider that directory as a package). Normally you do want, at least,
    to assign the appropriate list to __all__ .

    In p. 124 of "Python in a Nutshell", i.e. the section named "Packages",
    I summarize the issue as follows:
    """
    A package named P resides in a subdirectory, also called P, of some
    directory in sys.path. The module body of P is in the file P/__init__.py.
    You must have a file named P/__init__.py, even if it's empty (representing
    an empty module body), in order to indicate to Python that directory P
    is indeed a package.
    """

    Do you find this unclear, or was it hard for you to find -- or didn't
    you even considering checking out the Nutshell book before setting
    yourself the task to "professionally package" some Python software?
    Feedback on this (private or public as you wish) may help me do a
    better job next time. [Note that if the issue is with the purchase
    price of the Nutshell book, there IS a perfectly legal way to read it
    for free: the "Safari" online book service of O'Reilly offers the
    first two weeks free, so you can join it, and read Python in a Nutshell,
    which is among the over 1000 books on Safari, paying nothing as long
    as you cancel before two weeks are up].


    > 2. About the data_files option for setup.py....
    >
    > data_files = [ "<installation directory>", [ "<file1>",
    > "<file2>", ...] ]
    >
    > Is there a way to AVOID having to type EVERY single data file?? Can
    > I specify just the directory or use wild cards??


    You can form the list of file names in any way you wish, including,
    for example, using the glob module from Python's standard library.
    setup.py is a Python script, and any Python construct is available
    to you as the author of that script.

    For example, suppose we wanted to package up (for installation under
    directory foodir) all the files in our directory named datadir EXCEPT
    those whose file-extension is '.not' . That's easy...:

    """
    from distutils.core import setup

    import glob

    thefiles = []
    for afile in glob.glob('datadir/*'):
    if afile.endswith('.not'): continue
    thefiles.append(afile)

    print "Packaging up", thefiles

    setup(name='justdata', data_files = [ [ 'foodir', thefiles ] ] )
    """

    Here's an example of how this works:

    [alex@lancelot tryp]$ ls datadir
    aa.yes bb.not cc.yes
    [alex@lancelot tryp]$ python setup.py bdist
    Packaging up ['datadir/aa.yes', 'datadir/cc.yes']
    running bdist
    running bdist_dumb
    running build
    installing to build/bdist.linux-i686/dumb
    running install
    running install_data
    creating build/bdist.linux-i686/dumb
    creating build/bdist.linux-i686/dumb/usr
    creating build/bdist.linux-i686/dumb/usr/local
    creating build/bdist.linux-i686/dumb/usr/local/foodir
    copying datadir/aa.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
    copying datadir/cc.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
    tar -cf /x/tryp/dist/justdata-0.0.0.linux-i686.tar .
    gzip -f9 /x/tryp/dist/justdata-0.0.0.linux-i686.tar
    removing 'build/bdist.linux-i686/dumb' (and everything under it)
    [alex@lancelot tryp]$ tar tvzf dist/justdata-0.0.0.linux-i686.tar.gz
    drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./
    drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/
    drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/
    drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/foodir/
    -rw-rw-r-- alex/alex 0 2003-10-01 14:13:48 ./usr/local/foodir/aa.yes
    -rw-rw-r-- alex/alex 0 2003-10-01 14:13:55 ./usr/local/foodir/cc.yes
    [alex@lancelot tryp]$


    > 3. python setup.py sdist will make a package of SOURCE code and
    > using "bdist" will put BINARIES in package. What does this
    > mean???
    > Will bdist packages unpack to pyc (bytecode) files and sdist will
    > unpack
    > py (noncompiled) files???? What if I wanted py and pyc files
    > installed
    > from the package?? (i.e. source and compiled versions)


    Hmmm, if the issue was so unclear (and important) to you, was it
    SO hard to try things out...? Say that setup.py is:

    from distutils.core import setup

    setup(name='onemodule', py_modules=['hello'])

    and hello.py in the same directory is:

    def greet(name='World'):
    return 'Hello, %s!' % name

    Now, running python setup.py bdist gives the output:

    [alex@lancelot tryp]$ python setup.py bdist
    running bdist
    running bdist_dumb
    running build
    running build_py
    creating build/lib
    copying hello.py -> build/lib
    installing to build/bdist.linux-i686/dumb
    running install
    running install_lib
    creating build/bdist.linux-i686/dumb
    creating build/bdist.linux-i686/dumb/usr
    creating build/bdist.linux-i686/dumb/usr/local
    creating build/bdist.linux-i686/dumb/usr/local/lib
    creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3
    creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
    copying build/lib/hello.py ->
    build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
    byte-compiling
    build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages/hello.py
    to hello.pyc
    tar -cf /x/tryp/dist/onemodule-0.0.0.linux-i686.tar .
    gzip -f9 /x/tryp/dist/onemodule-0.0.0.linux-i686.tar
    removing 'build/bdist.linux-i686/dumb' (and everything under it)

    Notice that hello.py is copied into the directory to be packaged, as
    well as hello,pyc. And indeed we can double check what's in the tarball:

    [alex@lancelot tryp]$ tar tvzf dist/onemodule-0.0.0.linux-i686.tar.gz
    drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./
    drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/
    drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/
    drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/lib/
    drwxrwxr-x alex/alex 0 2003-10-01 14:30:41
    ../usr/local/lib/python2.3/
    drwxrwxr-x alex/alex 0 2003-10-01 14:30:42
    ../usr/local/lib/python2.3/site-packages/
    -rw-rw-r-- alex/alex 56 2003-10-01 14:30:10
    ../usr/local/lib/python2.3/site-packages/hello.py
    -rw-rw-r-- alex/alex 335 2003-10-01 14:30:42
    ../usr/local/lib/python2.3/site-packages/hello.pyc
    [alex@lancelot tryp]$

    See? Both the .py AND the .pyc, as expected.

    Should you want to REMOVE the .py files, for example, that can
    easily be achieved by operating directly on the tarball or zipfile,
    after distutils is finished (customizing distutils for such
    purposes, on the other hand, is harder).


    Alex
     
    Alex Martelli, Oct 1, 2003
    #4
  5. Alex

    Your email motivated me to buy your book. I think Nutshell
    was a VERY good purchase. Why? Lots of other docs for newbies
    don't go into too much detail. Your book is great for the completeness.
    Intermediate users need that.... especially if they don't like reading
    python.org docs online.

    CS


    Alex Martelli <> wrote in message news:<SSzeb.148956$>...
    > Christian Seberino wrote:
    >
    > > I really want to learn distutils to professionally package some open
    > > source software.
    > >
    > > I had some distutils questions after reading python.org distutil docs
    > > on packaging code
    > > if you don't mind. I would really appreciate some help.....
    > >
    > >
    > > 0. I'm confused about use of the word "package" in setup.py.
    > > packages = ["<package1>", "<package2>", ...]
    > > Does package here just mean "a new directory of source code to
    > > consider"???

    >
    > No, it means a Python package. The concept of "package" (a module
    > that may contain other modules, and corresponds to a directory with
    > an __init__.py file) exists in Python quite independently from
    > distutils -- the distutils just SUPPORT the concept, if the code
    > you want to package-up uses Python packages.
    >
    >
    > > 1. What exactly goes into the __init__.py files? I know what to put in
    > > setup.py
    > > and I know __init__.py is used for packages but I don't know what
    > > to put IN this file.

    >
    > Again, not a distutils-related question. A Python package's __init__.py
    > has the module-body of the package (remember, any package IS a module).
    > If you need no Python statements whatsoever in that body, __init__.py
    > may be empty (it must still exist, or else Python will refuse to
    > consider that directory as a package). Normally you do want, at least,
    > to assign the appropriate list to __all__ .
    >
    > In p. 124 of "Python in a Nutshell", i.e. the section named "Packages",
    > I summarize the issue as follows:
    > """
    > A package named P resides in a subdirectory, also called P, of some
    > directory in sys.path. The module body of P is in the file P/__init__.py.
    > You must have a file named P/__init__.py, even if it's empty (representing
    > an empty module body), in order to indicate to Python that directory P
    > is indeed a package.
    > """
    >
    > Do you find this unclear, or was it hard for you to find -- or didn't
    > you even considering checking out the Nutshell book before setting
    > yourself the task to "professionally package" some Python software?
    > Feedback on this (private or public as you wish) may help me do a
    > better job next time. [Note that if the issue is with the purchase
    > price of the Nutshell book, there IS a perfectly legal way to read it
    > for free: the "Safari" online book service of O'Reilly offers the
    > first two weeks free, so you can join it, and read Python in a Nutshell,
    > which is among the over 1000 books on Safari, paying nothing as long
    > as you cancel before two weeks are up].
    >
    >
    > > 2. About the data_files option for setup.py....
    > >
    > > data_files = [ "<installation directory>", [ "<file1>",
    > > "<file2>", ...] ]
    > >
    > > Is there a way to AVOID having to type EVERY single data file?? Can
    > > I specify just the directory or use wild cards??

    >
    > You can form the list of file names in any way you wish, including,
    > for example, using the glob module from Python's standard library.
    > setup.py is a Python script, and any Python construct is available
    > to you as the author of that script.
    >
    > For example, suppose we wanted to package up (for installation under
    > directory foodir) all the files in our directory named datadir EXCEPT
    > those whose file-extension is '.not' . That's easy...:
    >
    > """
    > from distutils.core import setup
    >
    > import glob
    >
    > thefiles = []
    > for afile in glob.glob('datadir/*'):
    > if afile.endswith('.not'): continue
    > thefiles.append(afile)
    >
    > print "Packaging up", thefiles
    >
    > setup(name='justdata', data_files = [ [ 'foodir', thefiles ] ] )
    > """
    >
    > Here's an example of how this works:
    >
    > [alex@lancelot tryp]$ ls datadir
    > aa.yes bb.not cc.yes
    > [alex@lancelot tryp]$ python setup.py bdist
    > Packaging up ['datadir/aa.yes', 'datadir/cc.yes']
    > running bdist
    > running bdist_dumb
    > running build
    > installing to build/bdist.linux-i686/dumb
    > running install
    > running install_data
    > creating build/bdist.linux-i686/dumb
    > creating build/bdist.linux-i686/dumb/usr
    > creating build/bdist.linux-i686/dumb/usr/local
    > creating build/bdist.linux-i686/dumb/usr/local/foodir
    > copying datadir/aa.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
    > copying datadir/cc.yes -> build/bdist.linux-i686/dumb/usr/local/foodir
    > tar -cf /x/tryp/dist/justdata-0.0.0.linux-i686.tar .
    > gzip -f9 /x/tryp/dist/justdata-0.0.0.linux-i686.tar
    > removing 'build/bdist.linux-i686/dumb' (and everything under it)
    > [alex@lancelot tryp]$ tar tvzf dist/justdata-0.0.0.linux-i686.tar.gz
    > drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./
    > drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/
    > drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/
    > drwxrwxr-x alex/alex 0 2003-10-01 14:25:40 ./usr/local/foodir/
    > -rw-rw-r-- alex/alex 0 2003-10-01 14:13:48 ./usr/local/foodir/aa.yes
    > -rw-rw-r-- alex/alex 0 2003-10-01 14:13:55 ./usr/local/foodir/cc.yes
    > [alex@lancelot tryp]$
    >
    >
    > > 3. python setup.py sdist will make a package of SOURCE code and
    > > using "bdist" will put BINARIES in package. What does this
    > > mean???
    > > Will bdist packages unpack to pyc (bytecode) files and sdist will
    > > unpack
    > > py (noncompiled) files???? What if I wanted py and pyc files
    > > installed
    > > from the package?? (i.e. source and compiled versions)

    >
    > Hmmm, if the issue was so unclear (and important) to you, was it
    > SO hard to try things out...? Say that setup.py is:
    >
    > from distutils.core import setup
    >
    > setup(name='onemodule', py_modules=['hello'])
    >
    > and hello.py in the same directory is:
    >
    > def greet(name='World'):
    > return 'Hello, %s!' % name
    >
    > Now, running python setup.py bdist gives the output:
    >
    > [alex@lancelot tryp]$ python setup.py bdist
    > running bdist
    > running bdist_dumb
    > running build
    > running build_py
    > creating build/lib
    > copying hello.py -> build/lib
    > installing to build/bdist.linux-i686/dumb
    > running install
    > running install_lib
    > creating build/bdist.linux-i686/dumb
    > creating build/bdist.linux-i686/dumb/usr
    > creating build/bdist.linux-i686/dumb/usr/local
    > creating build/bdist.linux-i686/dumb/usr/local/lib
    > creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3
    > creating build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
    > copying build/lib/hello.py ->
    > build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages
    > byte-compiling
    > build/bdist.linux-i686/dumb/usr/local/lib/python2.3/site-packages/hello.py
    > to hello.pyc
    > tar -cf /x/tryp/dist/onemodule-0.0.0.linux-i686.tar .
    > gzip -f9 /x/tryp/dist/onemodule-0.0.0.linux-i686.tar
    > removing 'build/bdist.linux-i686/dumb' (and everything under it)
    >
    > Notice that hello.py is copied into the directory to be packaged, as
    > well as hello,pyc. And indeed we can double check what's in the tarball:
    >
    > [alex@lancelot tryp]$ tar tvzf dist/onemodule-0.0.0.linux-i686.tar.gz
    > drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./
    > drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/
    > drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/
    > drwxrwxr-x alex/alex 0 2003-10-01 14:30:41 ./usr/local/lib/
    > drwxrwxr-x alex/alex 0 2003-10-01 14:30:41
    > ./usr/local/lib/python2.3/
    > drwxrwxr-x alex/alex 0 2003-10-01 14:30:42
    > ./usr/local/lib/python2.3/site-packages/
    > -rw-rw-r-- alex/alex 56 2003-10-01 14:30:10
    > ./usr/local/lib/python2.3/site-packages/hello.py
    > -rw-rw-r-- alex/alex 335 2003-10-01 14:30:42
    > ./usr/local/lib/python2.3/site-packages/hello.pyc
    > [alex@lancelot tryp]$
    >
    > See? Both the .py AND the .pyc, as expected.
    >
    > Should you want to REMOVE the .py files, for example, that can
    > easily be achieved by operating directly on the tarball or zipfile,
    > after distutils is finished (customizing distutils for such
    > purposes, on the other hand, is harder).
    >
    >
    > Alex
     
    Christian Seberino, Oct 7, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Christian Seberino
    Replies:
    0
    Views:
    271
    Christian Seberino
    Oct 11, 2003
  2. Christian Seberino
    Replies:
    0
    Views:
    275
    Christian Seberino
    Oct 11, 2003
  3. timw.google
    Replies:
    1
    Views:
    563
    Serge Orlov
    May 11, 2006
  4. Luis Alberto Zarrabeitia Gomez

    distutils, No module named numpy.distutils.fcompiler.conv_template

    Luis Alberto Zarrabeitia Gomez, Mar 30, 2009, in forum: Python
    Replies:
    0
    Views:
    1,362
    Luis Alberto Zarrabeitia Gomez
    Mar 30, 2009
  5. XLiIV
    Replies:
    1
    Views:
    132
    Stefan Behnel
    Feb 25, 2012
Loading...

Share This Page