del an imported Class at EOF... why?

[Ryan]

Good day all!

I've just inherited a large amount of python code. After spending some
time pour through the code, I've noticed that the original developer
(who is no longer w/ the company) constantly deletes the imported
classes at the end of the .py file. Why would one want to do such a
thing?

Ryan

 

[Carl Banks]

Good day all!

I've just inherited a large amount of python code. After spending some
time pour through the code, I've noticed that the original developer
(who is no longer w/ the company) constantly deletes the imported
classes at the end of the .py file. Why would one want to do such a
thing?

Sometimes an object is used only temporarily by a modules, while it's
importing.

In such cases, there are two reasons you might delete the object. If
it uses a lot of memory you could free up the memory for other
objects. Also you might want to clean up the module's namespace.

I del objects for these reasons sparingly, usually only when the
object uses a whole lot of memory, or the namespace is very messy.

However, I'm guessing that the person who wrote your code is just
overly paranoid.



Carl Banks

 

[Dave Angel]

Sometimes an object is used only temporarily by a modules, while it's
importing.

In such cases, there are two reasons you might delete the object. If
it uses a lot of memory you could free up the memory for other
objects. Also you might want to clean up the module's namespace.

I del objects for these reasons sparingly, usually only when the
object uses a whole lot of memory, or the namespace is very messy.

However, I'm guessing that the person who wrote your code is just
overly paranoid.



Carl Banks

There are several things you might have meant by "the end of the .py
file" Presumably you mean in top-level code, and since you said module
and not script, I'm presuming this is the code that runs outside of the
if __name__ == logic.

An example would be very good. I'd assume you meant something like:

//start module
from extern_module import MyClass

some definitions and classes

some common initialization code

if __name__ == "__main__":
some testing code

del MyClass
//end module

In other words, these classes are being deleted before the module is
made visible to other modules that imported it. This seems to me just
prudent management. He/she is making sure that the importer of your
code doesn't use your code as a longcut to get at MyClass. He's forcing
them to import them explicitly (from extern_module), not just get your
copy. Presumably these classes are not a part of your public interface,
so they shouldn't be visible, except with a leading underscore.
Naturally, if you try to use them directly in your own definitions and
classes, you'll also have trouble. So presumably these are classes that
are used only in the initialization code, or as base classes for new
classes defined in your code, or in default value expressions of your
function definitions.

I tend to always use the plain "import xmodule" form of statement, so
just one symbol gets into my space. And each use of a class from there
is of the form

xmodule.MyClass

Clearly there you would not delete MyClass in your own code, though you
could delete xmodule.

DaveA

 

[Steven D'Aprano]

Good day all!

I've just inherited a large amount of python code. After spending some
time pour through the code, I've noticed that the original developer
(who is no longer w/ the company) constantly deletes the imported
classes at the end of the .py file. Why would one want to do such a
thing?

Too much lead paint in his milk as a small child? *grin*


Possibly out of a (misplaced?) sense of "keeping the namespace clean", or
a desire to prevent people importing his module and then using the
classes he imports from elsewhere.

Personally, I think it's just a quirk. There's nothing wrong with doing
so, but nor is there anything right with it either. If he's worried about
the presence of an alien class messing up his beautifully designed API,
that's an aesthetic judgment, and can be (partially) managed by using
__all__ (a global list of names). But in general, I would prefer to
manage such namespace issues by saying:


import alienmodule

class MyClass(alienmodule.AlienClass):
do_stuff()



rather than:


from alienmodule import AlienClass

class MyClass(AlienClass):
do_stuff()

del AlienClass

 

[alex23]

import alienmodule

class MyClass(alienmodule.AlienClass):
    do_stuff()

rather than:

from alienmodule import AlienClass

class MyClass(AlienClass):
    do_stuff()

del AlienClass

The original developer may also have been unaware of the ability to
limit a * import through the use of __all__.

from alienmodule import AlienClass

__all__ = ['MyClass']

class MyClass(AlienClass):
do_stuff()

Of course, this is really only useful if you're doing "from module
import *" which is generally discouraged inside actual code.

 

[Ryan]

Thanks everyone for your insight. I'm going to have to agree with the
paranoid desire to prevent people importing his module and then using
the
classes he imports from elsewhere (I'm not ruling out the lead paint
theory until I can gather more evidence). It does beg the question for
me. Consider the example from his code below

from PyQt4 import QtGui

class LauncherWidget( QtGui.QWidget ):
# A Specialization of QWidget

del QtGui

Next time python comes across

from PyQt4 import QtGui

it would have to re-import the class, which seems a waste of cycles
that could accumulate. In this situation, the use of __all__ is
better. Plus, by using __all__ instead of del you do not have to worry
about forgetting to del a class.


Ryan

 

[Hans Mulder]

[....] It does beg the question for
me. Consider the example from his code below

from PyQt4 import QtGui

class LauncherWidget( QtGui.QWidget ):
# A Specialization of QWidget

del QtGui

Next time python comes across

from PyQt4 import QtGui

it would have to re-import the class, which seems a waste of cycles
that could accumulate.

Errrhm, no. He is not deleting the PyQt4 module from sys.modules;
he's only deleting the name QtGui from his own namespace. Next
time Python comes across

from PyQt4 import QtGui

, it finds that the module PyQt4 already exists in sys.modules, so
Python does not have to load the module again. All it has to do is
bind name QtGui in the importing module to the class with the same
name in the PyQt4 module. That does not take many cycles.


-- HansM

 

[Carl Banks]

Thanks everyone for your insight. I'm going to have to agree with the
paranoid desire to prevent people importing his module and then using
the
classes he imports from elsewhere (I'm not ruling out the lead paint
theory until I can gather more evidence).

It's wasted effort. Python isn't really designed to have totally
clean namespaces. A base class can be deleted at the end of a module
because it's only used while the module is being imported, but lots of
other symbols can't. You can't delete any global that's used in a
function, that means most imported modules or functions have to remain
in the module's namespace, and usually there are a lot more of them
than there are deletable base classes.

Given that you can't generally have an externally clean namespace, why
put forth such effort just to delete one or two names?

The right thing to do is to advertise which symbols are external with
__all__ (or, as judgment call, to use leading underscore on all
internal symbols).


There are some tricks you could use to keep __all__ up-to-date, for
instance you can use this idiom to automatically add anything defined
between two points to __all__ (though it can be subject to name
clashing):

## do all your imports here

_internals = set(globals())

## define all your functions/classes/constants here

_allnames = set(globals())
__all__ = list(x for x in _allnames-_internals
if not x.startswith('_'))
del _internals, _allnames
# del ok here because this does free memory


Carl Banks

 

[Steven D'Aprano]

Thanks everyone for your insight. I'm going to have to agree with the
paranoid desire to prevent people importing his module and then using
the
classes he imports from elsewhere (I'm not ruling out the lead paint
theory until I can gather more evidence). It does beg the question for
me.

No it doesn't, it raises the question.

http://en.wikipedia.org/wiki/Begging_the_question

Consider the example from his code below

from PyQt4 import QtGui

class LauncherWidget( QtGui.QWidget ):
# A Specialization of QWidget

del QtGui

Next time python comes across

from PyQt4 import QtGui

it would have to re-import the class, which seems a waste of cycles that
could accumulate.

As Hans explained, not quite. When you call "from PyQt4 import QtGui",
Python loads the module PyQt4, caches the module in sys.modules, and then
adds PyQt4.QtGui into the current namespace (your module). When you
delete QtGui, that only removes it from your namespace. The module is
still in the cache, so the next time you call the import, it's much, much
faster.

Multiple imports aren't entirely free, but they're much less expensive
than you might think.

 

[Terry Reedy]

Errrhm, no. He is not deleting the PyQt4 module from sys.modules;
he's only deleting the name QtGui from his own namespace. Next
time Python comes across

from PyQt4 import QtGui

, it finds that the module PyQt4 already exists in sys.modules, so
Python does not have to load the module again. All it has to do is
bind name QtGui in the importing module to the class with the same
name in the PyQt4 module. That does not take many cycles.

I recently discovered that when IDLE restarts (as when one runs a file
from an edit window), it clears the global namespace but does not clear
sys.modules. Hence re-running a script that does time-consuming imports
is much faster, as only the global name binding is done after the first run.

 

[Simon Forman]

I recently discovered that when IDLE restarts (as when one runs a file from
an edit window), it clears the global namespace but does not clear
sys.modules. Hence re-running a script that does time-consuming imports is
much faster, as only the global name binding is done after the first run.

Is that with or without '-n'?

 

[Terry Reedy]

Is that with or without '-n'?

The Windows start menu IDLE entry, which I gather is without -n, though
the property sheet does not actually give the command-line equivalent.

 

[Terry Reedy]

I recently discovered that when IDLE restarts (as when one runs a file
from an edit window), it clears the global namespace but does not clear
sys.modules. Hence re-running a script that does time-consuming imports
is much faster, as only the global name binding is done after the first
run.

Whoops, on rechecking, above does not seem to be true. The faster rerun
time I was seeing was due to some other cause. Sorry for the noise.