py2exe/distutils: how to include a tree of files?

[Grant Edwards]

I'm packaging up a python program and need to include an entire
directory tree in the distribution. the data_files=[] option to
setup() is fine for individual files, but what do I do when I
need to include an entire directory tree?

On a more general note, I'm having trouble finding much
documentation on py2exe at all. There's a wiki page at
py2exe.org, but it's pretty superficial. It mentions looking
at the distutils documentation for details, but I've no idea
how distutils relates to py2exe or where in the distutils
documentation to look for py2exe stuff (I don't see anything
that looks familiar in the distutils docs).

There must be some py2exe documentation somewhere...

[http://sourceforge.net/projects/py2exe/ hasn't been updated in
5 years, is there any reason why it's still around?]

 

[Grant Edwards]

I'm packaging up a python program and need to include an entire
directory tree in the distribution. the data_files=[] option to
setup() is fine for individual files, but what do I do when I
need to include an entire directory tree?
[...]

There must be some py2exe documentation somewhere...

I can't even find anywhere that documents the 'data_files=[]'
option to setup(). It certainly doesn't show up when I do
help(setup) after importing setup from distutils.core.

 

[Grant Edwards]

I'm packaging up a python program and need to include an entire
directory tree in the distribution.

I've cobbled together something using package_dir/package_data
and os.path.walk(), but it seems like a bit of a kludge...

 

[Carl Banks]

I've cobbled together something using package_dir/package_data
and os.path.walk(), but it seems like a bit of a kludge...

"data_files" is documented in the official distutils documentation.
(Yes, in regular distutils you do have to pretty much walk the data
directories to get your data.)

I guess the idea was that you would use the same setup.py for
building, installing, and packaging (source and py2exe). But it
doesn't work all that well in practice, because using "data_files" is
rather useless for non-py2exe packaging since you could never be sure
where distutils actually installed the data files. (You could only
guess, based on the location of the script files, and even that was
shaky because of a bug if the user installed it with --prefix.)

The packages I use all seem to avoid this difficulty by placing data
in the package directories, alongside the py files.

I suppose because "data_files" wasn't too helpful, there was never
much incentive to improve it. Which (getting back to py2exe) is
unfortunate since you CAN rely on the location when using py2exe, but
it is stuck with the unwieldy usage. Oh well.


Carl Banks

 

[Grant Edwards]

"data_files" is documented in the official distutils
documentation.

It took quite a while, but I eventually found it. Having the
documentation split up into dozens and dozens of small pages
sure makes it hard to find things. I don't really understand
what the advantage for splitting up documentation into
screen-sized chunks is supposed to be -- all the web browsers
I've ever seen can scroll. One page per module or chapter
sure would make it easier to search...

(Yes, in regular distutils you do have to pretty much walk the data
directories to get your data.)

That's what I ended up doing. Fortunately os.walk() makes it
fairly easy.

 

[Basilisk96]

I'm packaging up a python program and need to include an entire
directory tree in the distribution. the data_files=[] option to
setup() is fine for individual files, but what do I do when I
need to include an entire directory tree?

On a more general note, I'm having trouble finding much
documentation on py2exe at all. There's a wiki page at
py2exe.org, but it's pretty superficial. It mentions looking
at the distutils documentation for details, but I've no idea
how distutils relates to py2exe or where in the distutils
documentation to look for py2exe stuff (I don't see anything
that looks familiar in the distutils docs).

There must be some py2exe documentation somewhere...

[http://sourceforge.net/projects/py2exe/hasn't been updated in
5 years, is there any reason why it's still around?]

Grant,

Any os.walk() loop that you can conceive which will generate a list of
the kind [(dir1, pathList1), (dir2, pathList2), etc...] will work.
Each subdir of your top-level tree has to be listed this way with the
accompanying pathList of the subdir's files. I did this once for a
'samples' directory tree that I packaged with an app:

#sample data files
sampleDir = r'.\\samples'
for root, dirs, files in os.walk(sampleDir):
sampleList = []
if files:
for filename in files:
#ignore SVN dirs
if ".svn" not in root:
sampleList.append(os.path.join(root, filename))
if sampleList:
dataFiles.append((root, sampleList))

On a side note, I've discovered that matplotlib since 0.90 or maybe
earlier has specifically added a matplotlib.get_py2exe_datafiles()
function to add its necessary data files in this way. Wouldn't it be
nice if py2exe had something simple like that built in...

Have you tried GUI2Exe? I've found its interface quite easy to use.
It doesn't recurse into a dir tree (feature request?), but it makes
the selection process at least half as painful.

The py2exe docs leave something more to be desired. I found more tips
and help via Google, ASPN, and on this list.

Good luck,
Basilisk96

 

[Ben Finney]

I don't really understand what the advantage for splitting up
documentation into screen-sized chunks is supposed to be -- all the
web browsers I've ever seen can scroll.

The issue is the time to download the page. Most of the world does not
have fast internet access.

One page per module or chapter sure would make it easier to
search...

Fortunately online documentation in multiple HTML pages is indexable
by the major search engines; most of those can search within one
domain, or even within one path prefix of a domain.

 

[Peter Otten]

The issue is the time to download the page. Most of the world does not
have fast internet access.


Fortunately online documentation in multiple HTML pages is indexable
by the major search engines; most of those can search within one
domain, or even within one path prefix of a domain.

The new and improved layout uses fewer pages, but indeed feels less snappy.
See for yourself:

http://docs.python.org/dev/distutils/setupscript.html
http://docs.python.org/dist/setup-script.html

Peter