distutils, sdist and tests

S

Steven Bethard

How do I get distutils to include my testing module in just the "sdist"
distribution? My current call to setup() looks like::

distutils.core.setup(
...
py_modules=['argparse'],
)

If change this to::

distutils.core.setup(
...
py_modules=['argparse', 'test_argparse'],
)

then test_argparse.py gets included in the source distribution, but it
also gets installed to site-packages like a normal module. I don't think
I want that. If I change it to::

distutils.core.setup(
...
py_modules=['argparse'],
scripts=['test_argparse.py'],
)

then test_argparse.py gets included in the source distribution, but it
also gets installed in the Python scripts directory. I don't think I
want that either.

I want test_argparse.py to be available in the source distribution, but
I don't think it should be included in the binary distributions.

Or am I just being to picky? Do folks normally include their tests in
the binary distributions?

STeVe
 
R

Robert Kern

Steven said:
How do I get distutils to include my testing module in just the "sdist"
distribution?
Click to expand...

Use a MANIFEST.

http://docs.python.org/dist/source-dist.html
I want test_argparse.py to be available in the source distribution, but
I don't think it should be included in the binary distributions.

Or am I just being to picky? Do folks normally include their tests in
the binary distributions?
Click to expand...

Some do, but only when they are packages and can thus hide.the.tests .

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
 
R

Robert Kern

Also, I just noted this tidbit:

"""If you don't supply an explicit list of files (or instructions on how to
generate one), the sdist command puts a minimal default set into the source
distribution:
...
* anything that looks like a test script: test/test*.py (currently, the
Distutils don't do anything with test scripts except include them in source
distributions, but in the future there will be a standard for testing Python
module distributions)
...
"""

So you can just stick test_argparse.py into a test/ directory. Having tested
this, it appears to work (see below).
Using a MANIFEST appears to do the same thing as putting "test_argparse"
into py_modules -- that is, it puts "test_argparse.py" into both "sdist"
and "bdist" distributions. In "bdist" distributions it gets installed to
the site-packages directory like any other module.
Click to expand...

Are you sure that you don't have changes left over in your setup.py when you
tested that? Also check that you clear out your build/ directory every time. I
think the bdist_* commands tend to just copy whatever is in the appropriate
build/lib.*/ directory even though you've changed how those files should be
generated.

This is what I get with setup.py unmodified. The only thing I added was
MANIFEST.in and an empty test_argparse.py file.

[argparse]$ ls
MANIFEST.in PKG-INFO README.txt argparse.py setup.py
test_argparse.py
[argparse]$ cat MANIFEST.in
include PKG-INFO
include *.txt
include *.py
[argparse]$ python setup.py bdist_dumb
running bdist_dumb
running build
running build_py
creating build
creating build/lib
copying argparse.py -> build/lib
installing to build/bdist.macosx-10.3-fat/dumb
running install
running install_lib
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/dumb
creating build/bdist.macosx-10.3-fat/dumb/Library
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
copying build/lib/argparse.py ->
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
byte-compiling
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
to argparse.pyc
running install_egg_info
Writing
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
creating /Users/rkern/src/argparse/dist
tar -cf /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar .
gzip -f9 /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar
removing 'build/bdist.macosx-10.3-fat/dumb' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.macosx-10.3-fat.tar.gz
../
../Library/
../Library/Frameworks/
../Library/Frameworks/Python.framework/
../Library/Frameworks/Python.framework/Versions/
../Library/Frameworks/Python.framework/Versions/2.5/
../Library/Frameworks/Python.framework/Versions/2.5/lib/
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.pyc
[argparse]$ python setup.py sdist
running sdist
reading manifest template 'MANIFEST.in'
writing manifest file 'MANIFEST'
creating argparse-0.5.0
making hard links in argparse-0.5.0...
hard linking PKG-INFO -> argparse-0.5.0
hard linking README.txt -> argparse-0.5.0
hard linking argparse.py -> argparse-0.5.0
hard linking setup.py -> argparse-0.5.0
hard linking test_argparse.py -> argparse-0.5.0
tar -cf dist/argparse-0.5.0.tar argparse-0.5.0
gzip -f9 dist/argparse-0.5.0.tar
removing 'argparse-0.5.0' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.tar.gz
argparse-0.5.0/
argparse-0.5.0/argparse.py
argparse-0.5.0/PKG-INFO
argparse-0.5.0/README.txt
argparse-0.5.0/setup.py
argparse-0.5.0/test_argparse.py


And this is what I get when I put test_argparse.py into a test/ directory:

[argparse]$ ls
PKG-INFO README.txt argparse.py setup.py test
[argparse]$ python setup.py bdist_dumb
running bdist_dumb
running build
running build_py
creating build
creating build/lib
copying argparse.py -> build/lib
installing to build/bdist.macosx-10.3-fat/dumb
running install
running install_lib
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/dumb
creating build/bdist.macosx-10.3-fat/dumb/Library
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks
creating build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
creating
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
copying build/lib/argparse.py ->
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
byte-compiling
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
to argparse.pyc
running install_egg_info
Writing
build/bdist.macosx-10.3-fat/dumb/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
creating /Users/rkern/src/argparse/dist
tar -cf /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar .
gzip -f9 /Users/rkern/src/argparse/dist/argparse-0.5.0.macosx-10.3-fat.tar
removing 'build/bdist.macosx-10.3-fat/dumb' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.macosx-10.3-fat.tar.gz
../
../Library/
../Library/Frameworks/
../Library/Frameworks/Python.framework/
../Library/Frameworks/Python.framework/Versions/
../Library/Frameworks/Python.framework/Versions/2.5/
../Library/Frameworks/Python.framework/Versions/2.5/lib/
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse-0.5.0-py2.5.egg-info
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.py
../Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/argparse.pyc
[argparse]$ python setup.py sdist
running sdist
warning: sdist: manifest template 'MANIFEST.in' does not exist (using default
file list)
writing manifest file 'MANIFEST'
creating argparse-0.5.0
creating argparse-0.5.0/test
making hard links in argparse-0.5.0...
hard linking README.txt -> argparse-0.5.0
hard linking argparse.py -> argparse-0.5.0
hard linking setup.py -> argparse-0.5.0
hard linking test/test_argparse.py -> argparse-0.5.0/test
tar -cf dist/argparse-0.5.0.tar argparse-0.5.0
gzip -f9 dist/argparse-0.5.0.tar
removing 'argparse-0.5.0' (and everything under it)
[argparse]$ tar ztf dist/argparse-0.5.0.tar.gz
argparse-0.5.0/
argparse-0.5.0/argparse.py
argparse-0.5.0/PKG-INFO
argparse-0.5.0/README.txt
argparse-0.5.0/setup.py
argparse-0.5.0/test/
argparse-0.5.0/test/test_argparse.py

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
 
S

Steven Bethard

Robert said:
Also, I just noted this tidbit:

"""If you don't supply an explicit list of files (or instructions on how to
generate one), the sdist command puts a minimal default set into the source
distribution:
...
* anything that looks like a test script: test/test*.py (currently, the
Distutils don't do anything with test scripts except include them in source
distributions, but in the future there will be a standard for testing Python
module distributions)
...
"""

So you can just stick test_argparse.py into a test/ directory. Having tested
this, it appears to work (see below).
Click to expand...

Yeah, I was trying to avoid having to put it in a test/ directory. When
they're in the same directory, I can just run test_argparse.py and it
tests the local one instead of the one I have installed in
site-packages. When it's in a test/ directory, running
test/test_argparse.py tests the one in site-packages.

I did try putting test_argparse.py in a test/ directory though, and it
doesn't seem to solve the problem -- test_argparse.py gets put into
site-packages with the "bdist" command:
> python setup.py sdist
Click to expand...
running sdist
warning: sdist: manifest template 'MANIFEST.in' does not exist (using
default file list)
writing manifest file 'MANIFEST'
creating argparse-0.6.0
creating argparse-0.6.0\test
copying files to argparse-0.6.0...
copying README.txt -> argparse-0.6.0
copying argparse.py -> argparse-0.6.0
copying setup.py -> argparse-0.6.0
copying test\test_argparse.py -> argparse-0.6.0\test
creating 'dist\argparse-0.6.0.zip' and adding 'argparse-0.6.0' to it
adding 'argparse-0.6.0\argparse.py'
adding 'argparse-0.6.0\PKG-INFO'
adding 'argparse-0.6.0\README.txt'
adding 'argparse-0.6.0\setup.py'
adding 'argparse-0.6.0\test\test_argparse.py'
removing 'argparse-0.6.0' (and everything under it)
> python setup.py bdist_dumb
Click to expand...
running bdist_dumb
running build
running build_py
installing to build\bdist.win32\dumb
running install
running install_lib
creating build\bdist.win32\dumb
creating build\bdist.win32\dumb\Program Files
creating build\bdist.win32\dumb\Program Files\Python
creating build\bdist.win32\dumb\Program Files\Python\Lib
creating build\bdist.win32\dumb\Program Files\Python\Lib\site-packages
copying build\lib\argparse.py -> build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages
copying build\lib\test_argparse.py -> build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages
byte-compiling build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages\argparse.py to argparse.pyc
byte-compiling build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages\test_argparse.py to test_argparse.pyc
running install_egg_info
Writing build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages\argparse-0.6.0-py2.5.egg-info
creating 'dist\argparse-0.6.0.win32.zip' and adding '.' to it
adding 'Program
Files\Python\Lib\site-packages\argparse-0.6.0-py2.5.egg-info'
adding 'Program Files\Python\Lib\site-packages\argparse.py'
adding 'Program Files\Python\Lib\site-packages\argparse.pyc'
adding 'Program Files\Python\Lib\site-packages\test_argparse.py'
adding 'Program Files\Python\Lib\site-packages\test_argparse.pyc'
removing 'build\bdist.win32\dumb' (and everything under it)

Are you sure that you don't have changes left over in your setup.py when you
tested that?
Click to expand...

Yep. (Though I still cleared everything out and tried it again.)
Here's what I got using an unmodified setup.py and the MANIFEST.in you
suggested. Note that the "bdist" version is putting test_argparse.py
into site-packages.
> type MANIFEST.in
Click to expand...
include PKG-INFO
include *.txt
include *.py
> python setup.py sdist
Click to expand...
running sdist
reading manifest template 'MANIFEST.in'
warning: no files found matching 'PKG-INFO'
writing manifest file 'MANIFEST'
creating argparse-0.6.0
copying files to argparse-0.6.0...
copying LICENSE.txt -> argparse-0.6.0
copying README.txt -> argparse-0.6.0
copying argparse.py -> argparse-0.6.0
copying setup.py -> argparse-0.6.0
copying test_argparse.py -> argparse-0.6.0
creating 'dist\argparse-0.6.0.zip' and adding 'argparse-0.6.0' to it
adding 'argparse-0.6.0\argparse.py'
adding 'argparse-0.6.0\LICENSE.txt'
adding 'argparse-0.6.0\PKG-INFO'
adding 'argparse-0.6.0\README.txt'
adding 'argparse-0.6.0\setup.py'
adding 'argparse-0.6.0\test_argparse.py'
removing 'argparse-0.6.0' (and everything under it)
> python setup.py bdist_dumb
Click to expand...
running bdist_dumb
running build
running build_py
installing to build\bdist.win32\dumb
running install
running install_lib
creating build\bdist.win32\dumb
creating build\bdist.win32\dumb\Program Files
creating build\bdist.win32\dumb\Program Files\Python
creating build\bdist.win32\dumb\Program Files\Python\Lib
creating build\bdist.win32\dumb\Program Files\Python\Lib\site-packages
copying build\lib\argparse.py -> build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages
copying build\lib\test_argparse.py -> build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages
byte-compiling build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages\argparse.py to argparse.pyc
byte-compiling build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages\test_argparse.py to test_argparse.pyc
running install_egg_info
Writing build\bdist.win32\dumb\Program
Files\Python\Lib\site-packages\argparse-0.6.0-py2.5.egg-info
creating 'dist\argparse-0.6.0.win32.zip' and adding '.' to it
adding 'Program
Files\Python\Lib\site-packages\argparse-0.6.0-py2.5.egg-info'
adding 'Program Files\Python\Lib\site-packages\argparse.py'
adding 'Program Files\Python\Lib\site-packages\argparse.pyc'
adding 'Program Files\Python\Lib\site-packages\test_argparse.py'
adding 'Program Files\Python\Lib\site-packages\test_argparse.pyc'
removing 'build\bdist.win32\dumb' (and everything under it)


STeVe
 
R

Robert Kern

Steven said:
Robert Kern wrote:

Yep. (Though I still cleared everything out and tried it again.)
Here's what I got using an unmodified setup.py and the MANIFEST.in you
suggested. Note that the "bdist" version is putting test_argparse.py
into site-packages.
running bdist_dumb
running build
running build_py
installing to build\bdist.win32\dumb
running install
running install_lib
Click to expand...

Are you sure that you removed the build/ directory? Because if it wasn't there,
distutils would say that it was creating one:

[argparse]$ python setup.py bdist_dumb
running bdist_dumb
running build
running build_py
creating build
creating build/lib
copying argparse.py -> build/lib
installing to build/bdist.macosx-10.3-fat/dumb
running install
running install_lib

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
 
S

Steven Bethard

Robert said:
Are you sure that you removed the build/ directory?
Click to expand...

Ahh thanks. Sorry I thought I was supposed to remove the old stuff from
dist/ not from build/. Yes, I get the same behavior as you do when I
remove build/. Sorry for the confusion!

Steve
 
S

Steven Bethard

Robert said:
Use a MANIFEST.
Click to expand...

Thanks again to Robert Kern for all the help. For the record, in the
end all I did was add a MANIFEST.in file with the single line:

include test_argparse.py

and "test_argparse.py" got included in the source distribution and left
out of the binary distributions.

It's a pity there's no way to specify something this simple in the
setup.py file, but since I'm not offering to maintain distutils, I guess
I don't really have a right to complain. ;-)

STeVe
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Distutils error 2
distutils and libraries 0
pip and distutils 0
distutils bdist_wininst failure on Linux 4
How to include docs using distutils? 0
Distutils, bdist_XXX, rpmbuild issues 0
distutils: "build" command 1
How to install/uninstall manpages with distutils/setuptools? 0

Members online

Total: 38 (members: 2, guests: 36)
Robots: 112

Forum statistics

Threads
473,773
Messages
2,569,594
Members
45,117
Latest member
Matilda564

Latest Threads

Top