Import Semantics, or 'What's the scope of an import?', and classattribute instantiation

[Andrew James]

All,
I'm having some trouble with understanding python's importing behaviour
in my application. I'm using psyco to optimise part of my code, but I'm
not sure whether it inherits throughout the rest of my application (read
this as for any imported module) if I import in in a 'higher-level'
module. For example:

A.py
====

import psyco
from B import BClass
class AClass():
...
...
b = BClass()

B.py
====

class BClass():
...
...

In general, I've noticed that if import X and B in A.py and want to
reference X.* from B.py, I need to import X again in B. Is this a hard
and fast rule, or is there a way I can import the common libs, etc. in
the starting .py file and have those inherited by other classes I
import/instantiate? How would I do this?

It seems to be the general consensus that it's best to keep a Python app
in fewer files in the same directory rather than spreading them out, a
few (or one) classes to a file in a directory hierarchy as in Java. I
understand this is due to Python's, self.* and import <path> operations
having a relatively high cost (traversing directories, etc. etc.)

What I don't see mentioned is that when I step through a Python script
(say, in Eric3), if the structure of the file is like this:

X.py
====

class myX():
att1 = 'Test'
att2 = []
att3 = MemoryHungryClass()

class myY():
a = 'Another test'

....

if __name__ == 'main':
x = myX()

Python loads all the class attributes into memory (or, at least, cycles
through them) at runtime *even if I never instantiate the class*. This
has lead me to write code like:

class myX():
att3 = None
def __init__(self):
att3 = MemoryHungryClass()

Which seems to work a little better, but it seems rather ugly. I guess
the reason Python does this is because it's interpreted, not statically
compiled (and so needs to know whether myX.attr3 exists when called),
but I don't understand why the interpreter can't parse/instantiate the
attributes on the first call to instantiate the class. Surely this would
be a reason *for* splitting your code up into multiple files?

Being relatively new to Python, I'm trying to avoid coding things in an
un-Python (read C++/Java) manner, and am looking for some tutorials on
python-specific advanced features I can use (things like closures,
lambda forms, map(), etc. etc.). Could anyone point me towards some good
resources?

I would much appreciate some assistance in finding some answers to these
questions, as the research I've done seems to be inconclusive, if not
downright confusing.

Many thanks,
Andrew

 

[Diez B. Roggisch]

Hi,

I'm having some trouble with understanding python's importing behaviour
in my application. I'm using psyco to optimise part of my code, but I'm
not sure whether it inherits throughout the rest of my application (read
this as for any imported module) if I import in in a 'higher-level'
module. For example:

In the psyco doc it says that you can do full() - but it will bloat the
memory consumption, so its better to use either explicit or profile-based
optimization. I suggest reading the docs for psyco on thate.

A.py
====

import psyco
from B import BClass
class AClass():
...
...
b = BClass()

B.py
====

class BClass():
...
...

In general, I've noticed that if import X and B in A.py and want to
reference X.* from B.py, I need to import X again in B. Is this a hard
and fast rule, or is there a way I can import the common libs, etc. in
the starting .py file and have those inherited by other classes I
import/instantiate? How would I do this?

No, you can't - and as you say later on that you come from java: That's not
possible there, either.

Generally speaking, for each unit/file for interpretation or compilation (in
java/c++), you have to import all names that should be known there.

It seems to be the general consensus that it's best to keep a Python app
in fewer files in the same directory rather than spreading them out, a
few (or one) classes to a file in a directory hierarchy as in Java. I
understand this is due to Python's, self.* and import <path> operations
having a relatively high cost (traversing directories, etc. etc.)

No - the cost for importing are not so high, and occur only once. A second
import will make python recognize that this module is already known, so it
won't be imported again.

So they don't add much to your runtime - only startup time. Which is still
way faster than java's....

Java simply limits you to one class per file so the can maintain a
bijektive .java <-> .class mapping. I guess for make-like dependency
checking.

And as in java the class is the only unit of code you can write, there is no
way to declare functions outside of classes. Which you can do in python.

If you really want to, you can go the way way java does it. But then you
have to manually update the __init__.py for a module to make all declared
names visible, like this:

foo/__init__.py
from A import A

foo/A.py
class A:
pass

This is of course somewhat tedious. Instead putting all classes and
functions directly into a file called foo.py will rid you of these
complications, and keep belonging code in one file.

Python loads all the class attributes into memory (or, at least, cycles
through them) at runtime *even if I never instantiate the class*. This
has lead me to write code like:

class myX():
att3 = None
def __init__(self):
att3 = MemoryHungryClass()

Which seems to work a little better, but it seems rather ugly. I guess
the reason Python does this is because it's interpreted, not statically
compiled (and so needs to know whether myX.attr3 exists when called),
but I don't understand why the interpreter can't parse/instantiate the
attributes on the first call to instantiate the class. Surely this would
be a reason *for* splitting your code up into multiple files?

I'm not totally sure that I understand what you are doing here - it seems to
me that you confuse class attributes with instance attributes. The latter
are (usually) created in the __init__-method, like this:

class Foo:
def __init__(self):
self.bar = 1


a = Foo()
b = Foo()
a.bar += 1

print a.bar, b.bar

yield 2 for a.bar and 1 for b.bar

The former are attributes created while importing (so far you're right), but
they are created only once - for the _class_, not the objects of that
class. So you can compare them to static properties in java. Which will
also be created at the first import, and consume whatever resources they
need - time- and memorywise.

You can of course instaniate them lazily - like this:

class Foo:
bar = None
def __init__(self):
if Foo.bar is None:
Foo.bar = SomeMemoryConsumingObject()

So bar gets filled only when you actually instantiate a Foo. But this is no
different from java:

class Foo {

static Bar bar = null;

public Foo() {
if (bar == null) {
bar = new SomeMemoryConsumingObject();
}
}
}

Being relatively new to Python, I'm trying to avoid coding things in an
un-Python (read C++/Java) manner, and am looking for some tutorials on
python-specific advanced features I can use (things like closures,
lambda forms, map(), etc. etc.). Could anyone point me towards some good
resources?

http://docs.python.org/tut/tut.html

Especially

http://docs.python.org/tut/node8.html