Imports (in Py3), please help a novice

[John Ladasky]

I'm currently running Python 3.3 on Ubuntu 13.04.

Up until now, I have gotten away with writing modules whose namespace is completely defined within a single Python script. That has allowed me to usethe following, simple setup script for installation by distutils:

## setup.py ##

from distutils.core import setup
setup(name = "foo", version = "1.0", author = "John", py_modules = ["foo"])

##

As long as all the names I want to import are defined in foo.py, which is located in the same folder as my setup.py, this works. I can execute "import foo" from any Python3 program, and I get my names.

But now this approach is frustrating me. I'm looking at a 1000-line foo.pyfile, with five different functions I would like to make importable, and several functions which I do not need to import. Keeping track of the import statements from other modules which various parts of foo.py uses is getting confusing. I would like to make things more modular, to simplify my editing.

I don't necessarily need to break it down all the way to a single class perfile, but I would like to get closer to that goal than what I have. I also haven't reached the point where I would need subpackages within my parentpackage. Files in a flat folder will do. And it would be nice that, oncethose files are copied to site-packages, they would appear in a single sub-folder thereof.

I've just spent the past few hours trying to read the distutils documentation (which is opaque, and has not been revised for Py3, even though I can see articles on-line which indicate that several PEPs concerning imports havebeen implemented), looking at some existing packages (numpy and wx, both perhaps too complex), and performing some experiments with __init__.py scripts.

I have some __init__.py scripts which can import names when they are executed from within their own directory, but which fail when they are called by another program outside of that directory, indicating that I do not understand how to control the scope. I've tried to make use of __all__, but apparently that only affects statements of the "from foo import *" variety? I've installed things with distutils which work -- but which somehow install my modules twice, once in my site-packages folder and then also in a sub-folder of site-packages, creating clutter (which I then delete). I've tried looking at sys.modules, and I have found that imports will sometimes work even when modules are not listed there.

In short, I'm getting myself lost.

Is there an import / distutils tutorial out there? I'm looking for it, butperhaps one of you already knows where to find it. Thanks!

 

[John Ladasky]

Followup to my own post:

I am sticking pretty closely to this example from Mike Driscoll which, admittedly, is based on Python 2.6:

http://www.blog.pythonlibrary.org/2012/07/08/python-201-creating-modules-and-packages/

I'm trying to do this one step at a time. First try a local import, then install the module.

When I execute ./foo/__init__.py directly, from within its own directory, all the import statements work. Suppose that one of those import statementsreads, "from bar import baz". And, suppose that there are others, which refer to other names we want to import from other files.

Next, I execute another program from the parent directory of foo which says"from foo import bar". It finds ./foo/__init__.py. But when it tries to execute "from bar import baz", an ImportError is raised. That strikes me as odd. Since the foo directory is clearly already found, why isn't foo (atleast, temporarily) searched for modules?

Now, Mike suggests that, once you have built up the desired file hierarchy,"you can copy the folder into your Python’s site-packages folder." Really? The safe, recommended way to do this is to use distutils. Five years ago, I did it the same way that Mike shows.

http://www.gossamer-threads.com/lists/python/python/653838

Windows would let you cheat that way, if you had admin privileges. Linux stopped you, and made you use sudo. That red flag is what made me ask what the recommended method of installing to site-packages was, and got me started with distutils.

So, why don't I just do that now? As my first post indicated, I have tried.. There must be something wrong with the setup.py files I have written. Ikeep getting files installed in places I don't want them to be.

 

[Miki Tebeka]

Is there an import / distutils tutorial out there? I'm looking for it, but perhaps one of you already knows where to find it. Thanks!
Did you have a look at http://docs.python.org/3.3/distutils/examples.html?

Another thing to do is to look at what other packages on PyPi are doing.

 

[John Ladasky]

Thanks for your reply, Miki.

Did you have a look at http://docs.python.org/3.3/distutils/examples.html?

I wrote: "I've just spent the past few hours trying to read the distutils documentation..." but I didn't quite get as far as Section 7, the Examples. I was trying to RTFM linearly, beginning (naturally) at the beginning:

http://docs.python.org/3.3/distutils/introduction.html

I see some new things in the Examples section that I might try.

Another thing to do is to look at what other packages on PyPi are doing.

That's what I was trying to do by looking at numpy, matplotlib, and wx on my own system -- but they are all too complex. I'll have a look at PyPI, and with luck I will find some simpler examples. Thanks!

 

[Chris Angelico]

I was trying to RTFM linearly, beginning (naturally) at the beginning

Alas, the King of Hearts's good advice [1] doesn't work so well with
large documentation. It seems distutils is rather more complicated
than could be desired; but wasn't there a project a while ago to
simplify all that mess? Or was distutils the result of that, and it
used to be worse?

ChrisA

[1] To the White Rabbit -
http://en.wikisource.org/wiki/Alice's_Adventures_in_Wonderland/Chapter_12#182