del an imported Class at EOF... why?

Discussion in 'Python' started by Ryan, Oct 6, 2009.

  1. Ryan

    Ryan Guest

    Good day all!

    I've just inherited a large amount of python code. After spending some
    time pour through the code, I've noticed that the original developer
    (who is no longer w/ the company) constantly deletes the imported
    classes at the end of the .py file. Why would one want to do such a
    thing?

    Ryan
     
    Ryan, Oct 6, 2009
    #1
    1. Advertising

  2. Ryan

    Carl Banks Guest

    On Oct 6, 10:56 am, Ryan <> wrote:
    > Good day all!
    >
    > I've just inherited a large amount of python code. After spending some
    > time pour through the code, I've noticed that the original developer
    > (who is no longer w/ the company) constantly deletes the imported
    > classes at the end of the .py file. Why would one want to do such a
    > thing?



    Sometimes an object is used only temporarily by a modules, while it's
    importing.

    In such cases, there are two reasons you might delete the object. If
    it uses a lot of memory you could free up the memory for other
    objects. Also you might want to clean up the module's namespace.

    I del objects for these reasons sparingly, usually only when the
    object uses a whole lot of memory, or the namespace is very messy.

    However, I'm guessing that the person who wrote your code is just
    overly paranoid.



    Carl Banks
     
    Carl Banks, Oct 6, 2009
    #2
    1. Advertising

  3. Ryan

    Dave Angel Guest

    Carl Banks wrote:
    > On Oct 6, 10:56 am, Ryan <> wrote:
    >
    >> Good day all!
    >>
    >> I've just inherited a large amount of python code. After spending some
    >> time pour through the code, I've noticed that the original developer
    >> (who is no longer w/ the company) constantly deletes the imported
    >> classes at the end of the .py file. Why would one want to do such a
    >> thing?
    >>

    >
    >
    > Sometimes an object is used only temporarily by a modules, while it's
    > importing.
    >
    > In such cases, there are two reasons you might delete the object. If
    > it uses a lot of memory you could free up the memory for other
    > objects. Also you might want to clean up the module's namespace.
    >
    > I del objects for these reasons sparingly, usually only when the
    > object uses a whole lot of memory, or the namespace is very messy.
    >
    > However, I'm guessing that the person who wrote your code is just
    > overly paranoid.
    >
    >
    >
    > Carl Banks
    >
    >

    There are several things you might have meant by "the end of the .py
    file" Presumably you mean in top-level code, and since you said module
    and not script, I'm presuming this is the code that runs outside of the
    if __name__ == logic.

    An example would be very good. I'd assume you meant something like:

    //start module
    from extern_module import MyClass

    some definitions and classes

    some common initialization code

    if __name__ == "__main__":
    some testing code

    del MyClass
    //end module

    In other words, these classes are being deleted before the module is
    made visible to other modules that imported it. This seems to me just
    prudent management. He/she is making sure that the importer of your
    code doesn't use your code as a longcut to get at MyClass. He's forcing
    them to import them explicitly (from extern_module), not just get your
    copy. Presumably these classes are not a part of your public interface,
    so they shouldn't be visible, except with a leading underscore.
    Naturally, if you try to use them directly in your own definitions and
    classes, you'll also have trouble. So presumably these are classes that
    are used only in the initialization code, or as base classes for new
    classes defined in your code, or in default value expressions of your
    function definitions.

    I tend to always use the plain "import xmodule" form of statement, so
    just one symbol gets into my space. And each use of a class from there
    is of the form

    xmodule.MyClass

    Clearly there you would not delete MyClass in your own code, though you
    could delete xmodule.

    DaveA
     
    Dave Angel, Oct 6, 2009
    #3
  4. On Tue, 06 Oct 2009 10:56:26 -0700, Ryan wrote:

    > Good day all!
    >
    > I've just inherited a large amount of python code. After spending some
    > time pour through the code, I've noticed that the original developer
    > (who is no longer w/ the company) constantly deletes the imported
    > classes at the end of the .py file. Why would one want to do such a
    > thing?



    Too much lead paint in his milk as a small child? *grin*


    Possibly out of a (misplaced?) sense of "keeping the namespace clean", or
    a desire to prevent people importing his module and then using the
    classes he imports from elsewhere.

    Personally, I think it's just a quirk. There's nothing wrong with doing
    so, but nor is there anything right with it either. If he's worried about
    the presence of an alien class messing up his beautifully designed API,
    that's an aesthetic judgment, and can be (partially) managed by using
    __all__ (a global list of names). But in general, I would prefer to
    manage such namespace issues by saying:


    import alienmodule

    class MyClass(alienmodule.AlienClass):
    do_stuff()



    rather than:


    from alienmodule import AlienClass

    class MyClass(AlienClass):
    do_stuff()

    del AlienClass





    --
    Steven
     
    Steven D'Aprano, Oct 6, 2009
    #4
  5. Ryan

    alex23 Guest

    Steven D'Aprano <> wrote:
    > import alienmodule
    >
    > class MyClass(alienmodule.AlienClass):
    >     do_stuff()
    >
    > rather than:
    >
    > from alienmodule import AlienClass
    >
    > class MyClass(AlienClass):
    >     do_stuff()
    >
    > del AlienClass


    The original developer may also have been unaware of the ability to
    limit a * import through the use of __all__.

    from alienmodule import AlienClass

    __all__ = ['MyClass']

    class MyClass(AlienClass):
    do_stuff()

    Of course, this is really only useful if you're doing "from module
    import *" which is generally discouraged inside actual code.
     
    alex23, Oct 7, 2009
    #5
  6. Ryan

    Ryan Guest

    Thanks everyone for your insight. I'm going to have to agree with the
    paranoid desire to prevent people importing his module and then using
    the
    classes he imports from elsewhere (I'm not ruling out the lead paint
    theory until I can gather more evidence). It does beg the question for
    me. Consider the example from his code below

    from PyQt4 import QtGui

    class LauncherWidget( QtGui.QWidget ):
    # A Specialization of QWidget

    del QtGui

    Next time python comes across

    from PyQt4 import QtGui

    it would have to re-import the class, which seems a waste of cycles
    that could accumulate. In this situation, the use of __all__ is
    better. Plus, by using __all__ instead of del you do not have to worry
    about forgetting to del a class.


    Ryan
     
    Ryan, Oct 7, 2009
    #6
  7. Ryan

    Hans Mulder Guest

    Ryan wrote:
    > [....] It does beg the question for
    > me. Consider the example from his code below
    >
    > from PyQt4 import QtGui
    >
    > class LauncherWidget( QtGui.QWidget ):
    > # A Specialization of QWidget
    >
    > del QtGui
    >
    > Next time python comes across
    >
    > from PyQt4 import QtGui
    >
    > it would have to re-import the class, which seems a waste of cycles
    > that could accumulate.


    Errrhm, no. He is not deleting the PyQt4 module from sys.modules;
    he's only deleting the name QtGui from his own namespace. Next
    time Python comes across

    from PyQt4 import QtGui

    , it finds that the module PyQt4 already exists in sys.modules, so
    Python does not have to load the module again. All it has to do is
    bind name QtGui in the importing module to the class with the same
    name in the PyQt4 module. That does not take many cycles.


    -- HansM
     
    Hans Mulder, Oct 7, 2009
    #7
  8. Ryan

    Carl Banks Guest

    On Oct 7, 2:31 am, Ryan <> wrote:
    > Thanks everyone for your insight. I'm going to have to agree with the
    > paranoid desire to prevent people importing his module and then using
    > the
    > classes he imports from elsewhere (I'm not ruling out the lead paint
    > theory until I can gather more evidence).


    It's wasted effort. Python isn't really designed to have totally
    clean namespaces. A base class can be deleted at the end of a module
    because it's only used while the module is being imported, but lots of
    other symbols can't. You can't delete any global that's used in a
    function, that means most imported modules or functions have to remain
    in the module's namespace, and usually there are a lot more of them
    than there are deletable base classes.

    Given that you can't generally have an externally clean namespace, why
    put forth such effort just to delete one or two names?

    The right thing to do is to advertise which symbols are external with
    __all__ (or, as judgment call, to use leading underscore on all
    internal symbols).


    There are some tricks you could use to keep __all__ up-to-date, for
    instance you can use this idiom to automatically add anything defined
    between two points to __all__ (though it can be subject to name
    clashing):

    ## do all your imports here

    _internals = set(globals())

    ## define all your functions/classes/constants here

    _allnames = set(globals())
    __all__ = list(x for x in _allnames-_internals
    if not x.startswith('_'))
    del _internals, _allnames
    # del ok here because this does free memory


    Carl Banks
     
    Carl Banks, Oct 7, 2009
    #8
  9. On Wed, 07 Oct 2009 02:31:00 -0700, Ryan wrote:

    > Thanks everyone for your insight. I'm going to have to agree with the
    > paranoid desire to prevent people importing his module and then using
    > the
    > classes he imports from elsewhere (I'm not ruling out the lead paint
    > theory until I can gather more evidence). It does beg the question for
    > me.


    No it doesn't, it raises the question.

    http://en.wikipedia.org/wiki/Begging_the_question



    > Consider the example from his code below
    >
    > from PyQt4 import QtGui
    >
    > class LauncherWidget( QtGui.QWidget ):
    > # A Specialization of QWidget
    >
    > del QtGui
    >
    > Next time python comes across
    >
    > from PyQt4 import QtGui
    >
    > it would have to re-import the class, which seems a waste of cycles that
    > could accumulate.


    As Hans explained, not quite. When you call "from PyQt4 import QtGui",
    Python loads the module PyQt4, caches the module in sys.modules, and then
    adds PyQt4.QtGui into the current namespace (your module). When you
    delete QtGui, that only removes it from your namespace. The module is
    still in the cache, so the next time you call the import, it's much, much
    faster.

    Multiple imports aren't entirely free, but they're much less expensive
    than you might think.



    --
    Steven
     
    Steven D'Aprano, Oct 7, 2009
    #9
  10. Ryan

    Terry Reedy Guest

    Hans Mulder wrote:

    > Errrhm, no. He is not deleting the PyQt4 module from sys.modules;
    > he's only deleting the name QtGui from his own namespace. Next
    > time Python comes across
    >
    > from PyQt4 import QtGui
    >
    > , it finds that the module PyQt4 already exists in sys.modules, so
    > Python does not have to load the module again. All it has to do is
    > bind name QtGui in the importing module to the class with the same
    > name in the PyQt4 module. That does not take many cycles.


    I recently discovered that when IDLE restarts (as when one runs a file
    from an edit window), it clears the global namespace but does not clear
    sys.modules. Hence re-running a script that does time-consuming imports
    is much faster, as only the global name binding is done after the first run.
     
    Terry Reedy, Oct 8, 2009
    #10
  11. Ryan

    Simon Forman Guest

    On Thu, Oct 8, 2009 at 1:42 AM, Terry Reedy <> wrote:
    > Hans Mulder wrote:
    >
    >> Errrhm, no.  He is not deleting the PyQt4 module from sys.modules;
    >> he's only deleting the name QtGui from his own namespace.  Next
    >> time Python comes across
    >>
    >>    from PyQt4 import QtGui
    >>
    >> , it finds that the module PyQt4 already exists in sys.modules, so
    >> Python does not have to load the module again.  All it has to do is
    >> bind name QtGui in the importing module to the class with the same
    >> name in the PyQt4 module.  That does not take many cycles.

    >
    > I recently discovered that when IDLE restarts (as when one runs a file from
    > an edit window), it clears the global namespace but does not clear
    > sys.modules. Hence re-running a script that does time-consuming imports is
    > much faster, as only the global name binding is done after the first run.


    Is that with or without '-n'?
     
    Simon Forman, Oct 8, 2009
    #11
  12. Ryan

    Terry Reedy Guest

    Simon Forman wrote:
    > On Thu, Oct 8, 2009 at 1:42 AM, Terry Reedy <> wrote:
    >> Hans Mulder wrote:
    >>
    >>> Errrhm, no. He is not deleting the PyQt4 module from sys.modules;
    >>> he's only deleting the name QtGui from his own namespace. Next
    >>> time Python comes across
    >>>
    >>> from PyQt4 import QtGui
    >>>
    >>> , it finds that the module PyQt4 already exists in sys.modules, so
    >>> Python does not have to load the module again. All it has to do is
    >>> bind name QtGui in the importing module to the class with the same
    >>> name in the PyQt4 module. That does not take many cycles.

    >> I recently discovered that when IDLE restarts (as when one runs a file from
    >> an edit window), it clears the global namespace but does not clear
    >> sys.modules. Hence re-running a script that does time-consuming imports is
    >> much faster, as only the global name binding is done after the first run.

    >
    > Is that with or without '-n'?


    The Windows start menu IDLE entry, which I gather is without -n, though
    the property sheet does not actually give the command-line equivalent.
     
    Terry Reedy, Oct 8, 2009
    #12
  13. Ryan

    Terry Reedy Guest

    Terry Reedy wrote:
    > Hans Mulder wrote:
    >
    >> Errrhm, no. He is not deleting the PyQt4 module from sys.modules;
    >> he's only deleting the name QtGui from his own namespace. Next
    >> time Python comes across
    >>
    >> from PyQt4 import QtGui
    >>
    >> , it finds that the module PyQt4 already exists in sys.modules, so
    >> Python does not have to load the module again. All it has to do is
    >> bind name QtGui in the importing module to the class with the same
    >> name in the PyQt4 module. That does not take many cycles.

    >
    > I recently discovered that when IDLE restarts (as when one runs a file
    > from an edit window), it clears the global namespace but does not clear
    > sys.modules. Hence re-running a script that does time-consuming imports
    > is much faster, as only the global name binding is done after the first
    > run.


    Whoops, on rechecking, above does not seem to be true. The faster rerun
    time I was seeing was due to some other cause. Sorry for the noise.
     
    Terry Reedy, Oct 8, 2009
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. dw
    Replies:
    1
    Views:
    321
    Peter Hansen
    Feb 18, 2004
  2. Mr. SweatyFinger
    Replies:
    2
    Views:
    2,129
    Smokey Grindel
    Dec 2, 2006
  3. Kobu
    Replies:
    10
    Views:
    657
    Keith Thompson
    Mar 4, 2005
  4. Dun Peal
    Replies:
    10
    Views:
    468
    Chris Rebert
    May 3, 2011
  5. Volker Nicolai
    Replies:
    9
    Views:
    1,072
    Fabian Pilkowski
    Jul 4, 2005
Loading...

Share This Page