Re: super not working in __del__ ?

Discussion in 'Python' started by Christopher J. Bottaro, Feb 16, 2005.

  1. Fredrik Lundh wrote:

    > "Warning: Due to the precarious circumstances under which __del__()
    > methods are invoked, exceptions that occur during their execution are
    > ignored, and a warning is printed to sys.stderr instead. Also, when
    > __del__() is invoked in response to a module being deleted (e.g.,
    > when execution of the program is done), other globals referenced by
    > the __del__() method may already have been deleted. For this
    > reason, __del__() methods should do the absolute minimum needed
    > to maintain external invariants."


    Jeff Epler wrote:

    > *       Bugs and caveats: The destruction of modules and objects in
    > modules is *       done in random order; this may cause destructors
    > (__del__() methods) to *       fail when they depend on other objects
    > (even functions) or modules.


    2 Questions...
    1) Why does this never happen in C++? Or does it, its just never happened
    to me?
    2) I can understand random destruction of instantiated objects, but I find
    it weird that class definitions (sorry, bad terminology) are destroyed at
    the same time. So __del__ can't safely instantiate any classes if its
    being called as a result of interpreter shutdown? Boo...

    Oh well, thanks for the help...now I know.
     
    Christopher J. Bottaro, Feb 16, 2005
    #1
    1. Advertising

  2. Christopher J. Bottaro

    Jeff Shannon Guest

    Christopher J. Bottaro wrote:

    > 2 Questions...
    > 1) Why does this never happen in C++? Or does it, its just never happened
    > to me?
    > 2) I can understand random destruction of instantiated objects, but I find
    > it weird that class definitions (sorry, bad terminology) are destroyed at
    > the same time. So __del__ can't safely instantiate any classes if its
    > being called as a result of interpreter shutdown? Boo...


    Keep in mind that in Python, everything is a (dynamically created)
    object, including class objects. My recall of C/C++ memory
    organization is pretty weak, but IIRC it gives completely different
    behavior to code, stack objects, and heap objects. Code never needs
    to be cleaned up. In Python, everything (including functions and
    classes) is effectively a heap object, and thus functions and classes
    can (and indeed must) be cleaned up. Refcounting means that they
    won't ever (normally) be cleaned up while they're still in use, but
    during program shutdown refcounting necessarily ceases to apply.

    The closest that would happen in C++, I believe, would manifest itself
    as memory leaks and/or access of already-freed memory.

    Jeff Shannon
    Technician/Programmer
    Credit International
     
    Jeff Shannon, Feb 16, 2005
    #2
    1. Advertising

  3. Jeff Shannon wrote:

    > Christopher J. Bottaro wrote:
    >
    >> 2 Questions...
    >> 1) Why does this never happen in C++? Or does it, its just never
    >> happened to me?
    >> 2) I can understand random destruction of instantiated objects, but I
    >> find it weird that class definitions (sorry, bad terminology) are
    >> destroyed at
    >> the same time. So __del__ can't safely instantiate any classes if its
    >> being called as a result of interpreter shutdown? Boo...

    >
    > Keep in mind that in Python, everything is a (dynamically created)
    > object, including class objects. My recall of C/C++ memory
    > organization is pretty weak, but IIRC it gives completely different
    > behavior to code, stack objects, and heap objects. Code never needs
    > to be cleaned up. In Python, everything (including functions and
    > classes) is effectively a heap object, and thus functions and classes
    > can (and indeed must) be cleaned up. Refcounting means that they
    > won't ever (normally) be cleaned up while they're still in use, but
    > during program shutdown refcounting necessarily ceases to apply.
    >
    > The closest that would happen in C++, I believe, would manifest itself
    > as memory leaks and/or access of already-freed memory.


    So encapsulating your script in a main function fixes the problem:

    def main():
    blah blah

    if __name__ == "__main__":
    main()

    because all the objects instantiated in main() will be deleted when main
    ends, but before the interpreter shuts down, thus the objects will have
    access to all the symbols in module's __dict__ (or however that works).

    Why can't Python kind of do this step for us? I'm just curious, this stuff
    is interesting to me. Is it because it wouldn't know what parts of the
    module should be automatically put in the main function and what parts
    shouldn't?

    class blah:
    pass

    class blah1:
    pass

    # following goes in automatically generated main function
    obj = blah()
    obj2 = blah2()
    obj.doStuff()
    # main ends
    # interpreter shutdown

    I guess it wouldn't know to leave the class objects outside of the
    automatically generated main function.

    Just tell if my ramblings are idiotic and I'll shutup.

    Thanks,
    -- C
     
    Christopher J. Bottaro, Feb 16, 2005
    #3
  4. Christopher J. Bottaro

    Jeff Shannon Guest

    Christopher J. Bottaro wrote:


    > So encapsulating your script in a main function fixes the problem:


    Not necessarily.

    > because all the objects instantiated in main() will be deleted when main
    > ends, but before the interpreter shuts down, thus the objects will have
    > access to all the symbols in module's __dict__ (or however that works).


    What about default function parameters? Those can't be cleaned up
    until the function is deleted. What about class attributes, which
    can't be cleaned until the class is deleted? What about objects which
    have had references passed to other modules? What about sets of
    objects with cyclical references?

    There's too many corner cases and special cases for this to be
    reliable. Cutting down on module-global variables will help (and is a
    good idea anyhow), but it's not perfect.

    I'm just guessing, here, but I'd imagine that it might be possible to
    modify the interpreter so that, at shutdown, it carefully builds
    dependency trees and then walks through them in reverse, deleting
    objects in the "proper" order, and trying to handle cycles as sanely
    as possible. You could probably get the __del__ of almost every
    object to be fairly reliable. But that's a lot of work to go to when
    99% of the time all you need to do is flush the entire block of
    memory. (By 'a lot of work', I mean both in execution time causing
    the shutdown of Python to be notably slower, and in developer time
    writing such a fancy shutdown scheme.) It's much more practical to
    just say that __del__() is not reliable (especially given some of the
    other issues with it, such as cyclic references, etc.) and suggest
    that people write their code in such a way that it isn't required.

    Python's __del__() is not a C++/Java destructor. Trying to make it
    into one is unlikely to give an overal benefit.

    Jeff Shannon
    Technician/Programmer
    Credit International
     
    Jeff Shannon, Feb 16, 2005
    #4
  5. Jeff Shannon wrote:

    > Python's __del__() is not a C++/Java destructor.


    Learn something new everyday... What is it then? Excuse my ignorance, but
    what are you suppose to do if your object needs to clean up when its no
    longer used (like close open file handles, etc)? Are you use supposed to
    make a method called Destroy() or something and require users to call it
    when the object is about to be deleted? That seems to put the burdon of
    ref counting on the user. I read in the programming FAQ about making an
    explicit close() method that can be called multiple times. It just seems
    like kinda a pain when a C++/Java style destructor would nicely do what is
    desired. Should I just stop digging and chalk it up to a limitation of
    Python?

    I'm just trying to understand all this...

    Thanks,
    -- C
     
    Christopher J. Bottaro, Feb 17, 2005
    #5
  6. Christopher J. Bottaro

    Jeff Shannon Guest

    Christopher J. Bottaro wrote:

    > Jeff Shannon wrote:
    >
    >>Python's __del__() is not a C++/Java destructor.

    >
    > Learn something new everyday... What is it then? Excuse my ignorance, but
    > what are you suppose to do if your object needs to clean up when its no
    > longer used (like close open file handles, etc)?


    Well, those "open file handles" are presumably wrapped by file
    objects. Those file objects will close the underlying file handle as
    they are deallocated. This means that the only time you really need
    to worry about closing a file handle is if you want to reopen that
    same file immediately -- and in that case, __del__() can't help you
    because you can't know whether or not another reference to the file
    object exists somewhere else, so you'll have to explicitly close() the
    file anyhow. The same goes for sockets.

    > Are you use supposed to
    > make a method called Destroy() or something and require users to call it
    > when the object is about to be deleted? That seems to put the burdon of
    > ref counting on the user.


    Python's refcounting/GC scheme is such that you rarely actually need
    to explicitly destroy an object. Unlike C++, you're not actually
    freeing memory, and most of the resources that you might be using are
    already wrapped by an object that will finalize itself properly
    without being explicitly destroyed. Thanks to GC and the possibility
    of needing to break cycles, there's not much that you can guarantee
    about __del__() ... but there's not much that you *need* to do in
    __del__().

    > It just seems
    > like kinda a pain when a C++/Java style destructor would nicely do what is
    > desired. Should I just stop digging and chalk it up to a limitation of
    > Python?


    Well, I guess you could look at it as a trade-off. In C++, you can
    count on your destructor getting called, but you have to do your own
    memory management. (Note that you still can't count on heap objects
    that your destructor might want to use still being there -- it's just
    that you can *never* count on such things in C++, and are always
    expected to ensure these things yourself.) Python will take care of
    your memory for you, and it will safely handle most OS resources for
    you, so that you don't have to worry about them... and in return, if
    you have some type of resource that Python doesn't automatically
    handle, you need to explicitly save it yourself.

    Now, while I haven't yet done anything terribly complicated nor tried
    to use extensive persistence, I've essentially never felt a need to
    use __del__() in Python, nor missed the "proper" destructor of C++.
    In general, cleanup takes care of itself, and in those cases where I
    have more demanding needs, well, it's not *that* hard to hook into
    application shutdown and explicitly save my data.

    So, I don't think it's so much a "limitation" of Python, as it is
    simply a different way of handling things.

    Jeff Shannon
     
    Jeff Shannon, Feb 18, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Guest

    super.super.super how?

    Guest, Feb 19, 2005, in forum: Java
    Replies:
    24
    Views:
    10,872
    Darryl Pierce
    Feb 24, 2005
  2. Christopher J. Bottaro

    super not working in __del__ ?

    Christopher J. Bottaro, Feb 15, 2005, in forum: Python
    Replies:
    0
    Views:
    343
    Christopher J. Bottaro
    Feb 15, 2005
  3. Fredrik Lundh

    Re: super not working in __del__ ?

    Fredrik Lundh, Feb 16, 2005, in forum: Python
    Replies:
    5
    Views:
    627
    Duncan Booth
    Feb 16, 2005
  4. Replies:
    7
    Views:
    450
    Patricia Shanahan
    Apr 6, 2008
  5. Hugh Sasse Staff Elec Eng

    Inheriting variables, super, and "not super"?

    Hugh Sasse Staff Elec Eng, Dec 9, 2003, in forum: Ruby
    Replies:
    7
    Views:
    144
    Hugh Sasse Staff Elec Eng
    Dec 11, 2003
Loading...

Share This Page