RE: does python have useless destructors?

Discussion in 'Python' started by Tim Peters, Jun 14, 2004.

  1. Tim Peters

    Tim Peters Guest

    [David Turner]
    ....
    > The D programming language somehow contrives to have both garbage
    > collection and working destructors.


    Copied from the D docs:

    The garbage collector is not guaranteed to run the destructor for all
    unreferenced objects. Furthermore, the order in which the garbage
    collector calls destructors for unreference [sic] objects is not
    specified.

    ...

    Allocating class instances on the stack is useful for temporary objects
    that are to be automatically deallocated when the function is exited.
    No special handling is needed to account for function termination via
    stack unwinding from an exception. To work, they must not have
    destructors.

    ...

    When the garbage collector calls a destructor for an object of a class
    that has members that are references to garbage collected objects, those
    references are no longer valid. This means that destructors cannot
    reference sub objects. This rule does not apply to auto objects or
    objects deleted with the DeleteExpression.


    > So why can't Python?


    It's too hard to come up with an implementation so full of subtle but
    crucial distinctions <wink>.
    Tim Peters, Jun 14, 2004
    #1
    1. Advertising

  2. Tim Peters

    David Turner Guest

    "Tim Peters" <> wrote in message news:<>...
    > [David Turner]
    > ...
    > > The D programming language somehow contrives to have both garbage
    > > collection and working destructors.

    >

    [snip D docs]
    >
    > > So why can't Python?

    >
    > It's too hard to come up with an implementation so full of subtle but
    > crucial distinctions <wink>.


    Heh. You're quite right of course. In fact, the D implementation is
    bad in many ways, not least of which is the "auto" keyword. But
    still, this is a feature I'd very much like to see in Python. I
    *want* to spend more time in Python, but as things stand my C++ code
    is more robust.

    Regards
    David Turner
    David Turner, Jun 14, 2004
    #2
    1. Advertising

  3. Tim Peters

    Peter Hansen Guest

    David Turner wrote:

    > I *want* to spend more time in Python, but as things stand my C++ code
    > is more robust.


    Is this a theoretical definition of "robust", or a practical one?
    In other words, do you mean you Python code actually fails more
    often than your C++ code, or do you just mean that, theoretically,
    the uncertainty with Python destructors makes you feel that your
    Python code is less reliable?

    In my experience, and I know also the experience of *many* others,
    our Python programs are very much *more* reliable than our C++
    programs. In fact, I've rarely if ever had a Python program that
    failed in any way related to Python's destructor behaviour. On
    the other hand, I've rarely if ever had a C++ program which did
    not crash my entire system at least once during development...

    -Peter
    Peter Hansen, Jun 14, 2004
    #3
  4. Tim Peters

    David Turner Guest

    Peter Hansen <> wrote in message news:<>...
    > David Turner wrote:
    >
    > > I *want* to spend more time in Python, but as things stand my C++ code
    > > is more robust.

    >
    > Is this a theoretical definition of "robust", or a practical one?
    > In other words, do you mean you Python code actually fails more
    > often than your C++ code, or do you just mean that, theoretically,
    > the uncertainty with Python destructors makes you feel that your
    > Python code is less reliable?


    I mean that it's very much easier for me to write provably correct
    code in C++. Trying to do the same in Python is an uphill struggle.
    Furthermore, the C++ version is far more maintainable; the coding
    style is *intentional*. For big systems with multiple programmers
    using each other's objects, there's no question that I'd use C++.

    Of course, for small projects, Python is miles ahead of C++. I'd like
    to see that advantage extended to large projects as well. I hope that
    the complications involved in writing C++ code are accidental rather
    than inherent qualities of a big-system language.


    > In my experience, and I know also the experience of *many* others,
    > our Python programs are very much *more* reliable than our C++
    > programs. In fact, I've rarely if ever had a Python program that
    > failed in any way related to Python's destructor behaviour. On
    > the other hand, I've rarely if ever had a C++ program which did
    > not crash my entire system at least once during development...


    "...in any way related to Python's destructor behaviour" is the key
    part here. Python programs don't fail so much as misbehave. If they
    don't misbehave (and they're doing anything complicated), the code is
    often nested several layers deep in try/finally blocks. It's ugly,
    and hard to maintain.

    In my experience, the main reason for this is that the onus is on the
    user of the API to use it correctly. The most trivial examples of
    this are acquire/release cycles in things like mutexes. There are too
    many ways in which a user can fail to release a mutex, either through
    forgetfulness, or through not considering all possible paths of
    execution (which is difficult to do!). Now, interface contracts can
    remind the user when he needs to do something, and help to trace a
    fault, but they don't necessarily make the user's job any easier (or
    quicker). The C++ community, on the other hand, is working towards
    ways of *enforcing* API contracts, by reworking the API in such a way
    that it can't be abused. Again, the mutex is a handy example of this.
    Consider the following C++ code:

    class mutex {
    private:
    void acquire();
    void release();
    public:
    class lock {
    mutex& m_;
    public:
    lock(mutex& m): m_(m) { m_.acquire(); }
    ~lock() { m_.release(); }
    };
    };

    The acquire() and release() methods are private, so the only way to
    get at them is through the lock class. Ownership of the mutex is thus
    represented by the lifetime of the lock object. This makes it very
    difficult for the user of the mutex to abuse the acquire/release
    cycle.

    Critics will point out that a devious user could in fact circumvent
    these guarantees by allocating the lock on the heap. If that be the
    case:

    class mutex {
    private:
    void acquire();
    void release();
    class lock_ { /* as before */ };
    public:
    typedef std::auto_ptr<lock_> lock;
    lock get_lock() { return lock(new lock_(*this)); }
    };

    Were I to meet anybody who could circumvent such a design after its
    declaration, I'd probably hire them - after making them promise never
    to do it again.

    Regards
    David Turner
    David Turner, Jun 15, 2004
    #4
  5. Tim Peters

    Peter Hansen Guest

    David Turner wrote:

    > Peter Hansen <> wrote in message news:<>...
    >>David Turner wrote:
    >>>I *want* to spend more time in Python, but as things stand my C++ code
    >>>is more robust.

    >>
    >>Is this a theoretical definition of "robust", or a practical one?

    >
    > I mean that it's very much easier for me to write provably correct
    > code in C++. Trying to do the same in Python is an uphill struggle.


    Interesting... when was the last time you proved that your C++
    program was correct? Honestly. I've *never* seen anyone prove
    a program was correct, especially not C++, though I know that
    some folks do seem to want to do that, and they apparently
    actually do in some environments. I'm just surprised to find
    anyone here who actually writes the kind of code that must have
    formal proofs done for it before it can be used...

    > Furthermore, the C++ version is far more maintainable; the coding
    > style is *intentional*. For big systems with multiple programmers
    > using each other's objects, there's no question that I'd use C++.


    Also interesting. Have you actually tried using Python for big
    systems with multiple programmers, and encountered serious problems?
    I have just recently left a company where we had at least twenty
    man-years invested in one large Python system, with as many as about
    six Python programmers working on it simultaneously. I can say
    uncategorically that the software is the most stable and robust that
    I've ever been involved in writing. Any C, C++, Delphi, or Java
    system in which I was previously involved was definitely less robust.

    Since it's therefore obviously not a property of these languages
    that they are robust or not, it must be something more to do with
    the programmers, or the process, or something. I guess... ;-)

    > Of course, for small projects, Python is miles ahead of C++. I'd like
    > to see that advantage extended to large projects as well. I hope that
    > the complications involved in writing C++ code are accidental rather
    > than inherent qualities of a big-system language.


    I would say those complications are not inherent, and my experience
    with Python (where we had few such complications) bears out that idea.

    > "...in any way related to Python's destructor behaviour" is the key
    > part here. Python programs don't fail so much as misbehave. If they
    > don't misbehave (and they're doing anything complicated), the code is
    > often nested several layers deep in try/finally blocks. It's ugly,
    > and hard to maintain.


    I don't think code with multiple layers of nested try/finally and
    such is well-written. It should likely be refactored so that more
    methods are used, so that the code has greater cohesion and lower
    coupling. Two levels deep is probably the max I would consider
    without examining the structure very closely and considering
    refactoring.

    > In my experience, the main reason for this is that the onus is on the
    > user of the API to use it correctly. The most trivial examples of
    > this are acquire/release cycles in things like mutexes. There are too
    > many ways in which a user can fail to release a mutex, either through
    > forgetfulness, or through not considering all possible paths of
    > execution (which is difficult to do!). Now, interface contracts can
    > remind the user when he needs to do something, and help to trace a
    > fault, but they don't necessarily make the user's job any easier (or
    > quicker). The C++ community, on the other hand, is working towards
    > ways of *enforcing* API contracts, by reworking the API in such a way
    > that it can't be abused.


    Sounds like two worlds here. We didn't need contracts or enforcement
    of this kind. Our developers were working test-first, with as wide
    coverage of unit tests as possible, so we already knew the code was
    going to be pretty good without getting into B&D like that. I can
    see you have not adopted the XP mentality and test-driven development,
    so our thoughts on this whole matter are inevitably going to be
    contrary...

    -Peter
    Peter Hansen, Jun 15, 2004
    #5
  6. Tim Peters

    David Turner Guest

    Peter Hansen <> wrote in message news:<>...
    > David Turner wrote:
    > >
    > > I mean that it's very much easier for me to write provably correct
    > > code in C++. Trying to do the same in Python is an uphill struggle.

    >
    > Interesting... when was the last time you proved that your C++
    > program was correct? Honestly. I've *never* seen anyone prove
    > a program was correct, especially not C++, though I know that


    Very rarely do I do whole-program analysis. I don't know anyone who
    does that either. What I'm talking about are small chunks of code
    with critical behaviour. The mutex example I gave is a case in point.
    I can prove, under a reasonable set of assumptions, that there is no
    use-case that will lead to incorrect behaviour. This is far more
    difficult in Python as I have to do the proof each time I use the
    object. This is why formal proofs are rare in the industry - they
    just become too complicated.


    > some folks do seem to want to do that, and they apparently
    > actually do in some environments. I'm just surprised to find
    > anyone here who actually writes the kind of code that must have
    > formal proofs done for it before it can be used...


    I don't have a gun to my head. But I do have direct financial
    liability, which is just as good :).


    > Also interesting. Have you actually tried using Python for big
    > systems with multiple programmers, and encountered serious problems?


    Admittedly I have not. I mostly use it to glue things together, in
    which area it excels.


    > I have just recently left a company where we had at least twenty
    > man-years invested in one large Python system, with as many as about
    > six Python programmers working on it simultaneously. I can say
    > uncategorically that the software is the most stable and robust that
    > I've ever been involved in writing. Any C, C++, Delphi, or Java
    > system in which I was previously involved was definitely less robust.


    Interesting. I must say I'm a little envious :).


    > Since it's therefore obviously not a property of these languages
    > that they are robust or not, it must be something more to do with
    > the programmers, or the process, or something. I guess... ;-)


    I don't see how the "obviously" bit follows. But I will say that I
    know of almost no software makers out there that use C++ "properly".
    99% of the code I've seen out there is "C with classes". I'd be happy
    to discuss my views on what constitutes "proper" use of C++, but this
    is not the forum for that :).


    > Sounds like two worlds here. We didn't need contracts or enforcement
    > of this kind. Our developers were working test-first, with as wide
    > coverage of unit tests as possible, so we already knew the code was
    > going to be pretty good without getting into B&D like that. I can
    > see you have not adopted the XP mentality and test-driven development,
    > so our thoughts on this whole matter are inevitably going to be
    > contrary...


    I agree. Not much point in getting into an argument over that :).
    But the two approaches are not necessarily mutually exclusive:- some
    bits of the program (read libraries) require careful thought and
    design before a line of code is written; other bits (read purposive
    code) do better in a test-driven environment.

    My "other bits" tend to be a tiny fraction of the whole, so please
    excuse me if I exhibit some bias towards top-down development ;-).

    Regards
    David Turner
    David Turner, Jun 17, 2004
    #6
  7. Tim Peters

    Peter Hansen Guest

    David Turner wrote:

    > Peter Hansen <> wrote in message news:<>...
    >>Since it's therefore obviously not a property of these languages
    >>that they are robust or not, it must be something more to do with
    >>the programmers, or the process, or something. I guess... ;-)

    >
    > I don't see how the "obviously" bit follows.


    It was just that you've had non-robust code in Python and
    robust code in C++, while I've had the other way around.
    Therefore obviously it's not a property of the language,
    but more about how we're using them... something like that.

    -Peter
    Peter Hansen, Jun 17, 2004
    #7
  8. Tim Peters

    David Turner Guest

    Peter Hansen <> wrote in message news:<>...
    > David Turner wrote:
    >
    > > Peter Hansen <> wrote in message news:<>...
    > >>Since it's therefore obviously not a property of these languages
    > >>that they are robust or not, it must be something more to do with
    > >>the programmers, or the process, or something. I guess... ;-)

    > >
    > > I don't see how the "obviously" bit follows.

    >
    > It was just that you've had non-robust code in Python and
    > robust code in C++, while I've had the other way around.
    > Therefore obviously it's not a property of the language,
    > but more about how we're using them... something like that.
    >


    I see what you mean. Yes, I agree that the idioms used make all the
    difference, and that there are "safe" idioms in every language which
    should be promoted.

    I would still dearly love to see the RAII idiom available in Python,
    however, as I don't think there can be any doubt that it's safer than
    the try/finally idiom. It's another tool in the toolbox...

    Regards
    David Turner
    David Turner, Jun 18, 2004
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael P. Soulier

    does python have useless destructors?

    Michael P. Soulier, Jun 9, 2004, in forum: Python
    Replies:
    138
    Views:
    2,215
    Christos TZOTZIOY Georgiou
    Jun 22, 2004
  2. Delaney, Timothy C (Timothy)

    RE: does python have useless destructors?

    Delaney, Timothy C (Timothy), Jun 10, 2004, in forum: Python
    Replies:
    4
    Views:
    252
    Tim Bradshaw
    Jun 14, 2004
  3. Robert Brewer

    RE: does python have useless destructors?

    Robert Brewer, Jun 10, 2004, in forum: Python
    Replies:
    1
    Views:
    543
    Duncan Booth
    Jun 10, 2004
  4. Tim Peters
    Replies:
    2
    Views:
    307
    Paul Rubin
    Jun 12, 2004
  5. Delaney, Timothy C (Timothy)

    RE: does python have useless destructors?

    Delaney, Timothy C (Timothy), Jun 15, 2004, in forum: Python
    Replies:
    2
    Views:
    410
    Ype Kingma
    Jun 15, 2004
Loading...

Share This Page