status of Programming by Contract (PEP 316)?

[Steve Holden]

You quoted what I wrote in reply to a personal attack against me, but
you conveniently
neglected to quote the original insult that I was replying to. OK,
I'll concede that I shouldn't
have replied to a personal insult with another insult, but why am I
the only one at fault here
rather than the guy who started it?

You aren't, and if I wasn't being even-handed I apologize.

Frankly, Mr. Holden, I'm getting a bit tired of the clannish behavior
here, where
"outsiders" like me are held to a higher standard than your "insider"
friends. I don't know
who you are, nor do I care what you and your little group think about
me.

Well that's a healthy attitude, but I am concerned that the Python
community should be as welcoming as possible, so I don't like the fact
that you feel you are being treated differently from anyone else.

As for DbC or what I call "self-testing code," I have come to the
(tentative) realization that
it is easy to implement in current Python -- without resorting to the
"decorators" hack.
OK, maybe this should have been obvious to me from the
start, but here goes:

All you really need to test the pre-conditions of a function
is a call at the top of the function to another function that checks
the inputs. To test the
post-conditions, you just need a call at the bottom of the function,
just before the return,
that checks the return values. Those functions can also check the
invariants. Then you
define a global variable to switch all the self-test functions on or
off at once.

It does constrain functions to return only from the bottom of their
code, though, which not all Python functions currently do. Though
there's nothing to stop you putting the calls before every return.

An advantage of this approach is that all the self tests can be put at
the bottom of the file
(or perhaps in another file) to reduce "clutter" in the primary code.

I'd still prefer PEP 316, but this seems like a reasonable
alternative.

One suggestion I have for PEP 316 is to provide a way to place the
self-test
checks in a separate file to reduce clutter.

That would be a sensible suggestion. It would also be possible to guard
the lengthier tests with "if debug" to allow them to be omitted
(presumably along with their imports with a little further effort in
code organization) in (optimized) production mode where timings were
critical.

This could allow the benefits of DbC and unit testing at the same time.

Putting the verification code into external functions does assume that
global variables aren't used in the pre- and post-conditions, but I
presume that's a part of the regime anyway.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Russ]

Who watches the watchmen? The contracts are composed by the
programmers writing the code. Is it likely that the same person
who wrote a buggy function will know the right contract?

The idea here is that errors in the self-testing code are unlikely to
be correlated with
errors in the primary code. Hence, you get a sort of multiplying
effect on reliability. For
example, if the chance of error in the primary code and the self-test
code are each 0.01,
the chance of an undetected error is approximately 0.01^2 or 0.0001.

 

[Russ]

Well that's a healthy attitude, but I am concerned that the Python
community should be as welcoming as possible, so I don't like the fact
that you feel you are being treated differently from anyone else.

I certainly appreciate that. And I will try my best to refrain from
the little personal zingers (and
the big ones too).

It does constrain functions to return only from the bottom of their
code, though, which not all Python functions currently do. Though
there's nothing to stop you putting the calls before every return.

Oops! I didn't think of that. The idea of putting one before every
return certainly doesn't
appeal to me. So much for that idea.

 

[=?ISO-8859-1?Q?Ricardo_Ar=E1oz?=]

Who watches the watchmen? The contracts are composed by the
programmers writing the code. Is it likely that the same person
who wrote a buggy function will know the right contract?

Actually my point was that if a program is to be trusted in a critical
situation (critical as in catastrophe if it goes wrong) then the OS, the
compiler/interpreter etc should abide by the same rules. That is
obviously not possible, so there's not much case in making the time
investment necessary for correctness proof of a little program (or
usually a little function inside a program) when the possibilities for
failure are all around it and even in the code that will run that
function. And we should resort to other more sensible answers to the
safety problem.

 

[Jorge Godoy]

Alex, I think you are missing the point. Yes, I'm sure that web
searches are critical to
Google's mission and commercial success. But the point is that a few
subtle bugs cannot
destroy Google. If your search engines and associated systems have
bugs, you fix them
(or simply tolerate them) and continue on. And if a user does not get
the results he wants,
he isn't likely to die over it -- or even care much.

But if this pattern of not getting wanted results is common, then the user
will migrate to alternative search engines and this will *kill* the
business. Wrong results won't impact ONE search, but many will impact the
company business and will be part of the recipe to take it out of business.

Online financial transactions are another matter altogether, of
course. User won't die, but
they will get very irate if they lose money. But I don't think that's
what you are talking about
here.

Lets make someone loose his job and have all his money commitments
compromised because of this money lost and we might be talking about people
taking their lives.

Again, this isn't 100% sure to happen, but it *can* happen.

As it happens with a peacemaker: the user won't die if his heart skips one
beat, but start skipping a series of them and you're incurring in serious
problems.

Just because the result isn't immediate it doesn't mean it isn't critical.

 

[Russ]

Actually my point was that if a program is to be trusted in a critical
situation (critical as in catastrophe if it goes wrong) then the OS, the
compiler/interpreter etc should abide by the same rules. That is
obviously not possible, so there's not much case in making the time
investment necessary for correctness proof of a little program (or
usually a little function inside a program) when the possibilities for
failure are all around it and even in the code that will run that
function. And we should resort to other more sensible answers to the
safety problem.

I don't quite see it that way.

I would agree that if your OS and compiler are unreliable, then it
doesn't make much sense to bend over backwards worrying about the
reliability of your
application. But for real safety-critical applications, you have no
excuse for not using a
highly reliable OS and compiler. For really critical stuff, I think
the real-time OSs are usually
stripped down to
the bare basics. And if you are using something like SPARK Ada, the
language itself is
stripped of many of the fancier features in Ada itself. (There's also
something called the
Ada Ravenscar profile, which I believe is geared for safety-critical
use but is not quite as
restrictive as SPARK.)

Keep in mind that the OS and compiler are typically also
used for many other applications, so they tend to get tested fairly
thoroughly. And remember
also that you won't have extraneous applications running -- like a web
browser
or a video game, so the OS will probably not be heavily stressed. The
most likely source
of failure is likely to be your application, so bending over backwards
to get it right makes
sense.

Then again, if you are running C on Windows, you might as well just
give up on reliability
from the start. You don't have a prayer.

 

[Steve Holden]

But if this pattern of not getting wanted results is common, then the user
will migrate to alternative search engines and this will *kill* the
business. Wrong results won't impact ONE search, but many will impact the
company business and will be part of the recipe to take it out of business.


Lets make someone loose his job and have all his money commitments
compromised because of this money lost and we might be talking about people
taking their lives.

Again, this isn't 100% sure to happen, but it *can* happen.

As it happens with a peacemaker: the user won't die if his heart skips one
beat, but start skipping a series of them and you're incurring in serious
problems.

Just because the result isn't immediate it doesn't mean it isn't critical.

We probably need to distinguish between "mission-critical", where a
program has to work reliably for an organization to meet its goals, and
"safety-critical" where people die or get hurt if the program misbehaves.

The latter are the ones where you need to employ all possible techniques
to avoid all possible failure modes.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Carl Banks]

Actually my point was that if a program is to be trusted in a critical
situation (critical as in catastrophe if it goes wrong) then the OS, the
compiler/interpreter etc should abide by the same rules. That is
obviously not possible, so there's not much case in making the time
investment necessary for correctness proof of a little program (or
usually a little function inside a program) when the possibilities for
failure are all around it and even in the code that will run that
function. And we should resort to other more sensible answers to the
safety problem.

On the systems I work on, the OS is unit tested same as the application
software. Not sure about the compiler; I'm pretty sure we're not using
the latest GCC beta, though.


Carl Banks

 

[Carl Banks]

But if this pattern of not getting wanted results is common, then the
user will migrate to alternative search engines and this will *kill* the
business. Wrong results won't impact ONE search, but many will impact
the company business and will be part of the recipe to take it out of
business.


Lets make someone loose his job and have all his money commitments
compromised because of this money lost and we might be talking about
people taking their lives.

Again, this isn't 100% sure to happen, but it *can* happen.

As it happens with a peacemaker: the user won't die if his heart skips
one beat, but start skipping a series of them and you're incurring in
serious problems.

Just because the result isn't immediate it doesn't mean it isn't
critical.

This is starting to sound silly, people. Critical is a relative term,
and one project's critical may be anothers mundane. Sure a flaw in your
flagship product is a critical problem *for your company*, but are you
really trying to say that the criticalness of a bad web search is even
comparable to the most important systems on airplanes, nuclear reactors,
dams, and so on? Come on.

BTW, I'm not really agreeing with Russ here. His suggestion (that
because Python is not used in highly critical systems, it is not suitable
for them) is logically flawed. And Alex's point, that Python has a good
track record of reliabilty (look at Google's 99.9% uptime) is valid
whether Google is a critical system or not.

So please leave the laughable comparisons between flight systems and web
searches out of it. It's unnecessary and makes Pythoners look bad.


Carl Banks

(P.S. 99.9% uptime would be a critical flaw in the systems I work on.)

 

[Russ]

On Aug 31, 6:45 pm, Steve Holden

We probably need to distinguish between "mission-critical", where a
program has to work reliably for an organization to meet its goals, and
"safety-critical" where people die or get hurt if the program misbehaves.

The term "mission critical" itself can have a wide range of
connotations.

If a software failure would force a military pilot to abort his
mission and hobble back home with a partially disabled aircraft,
that's what I think of as "mission critical" software.

If Google needs reliable software on its servers to maintain its
revenue stream, that's another kind of "mission critical" software,
but the criticality is certainly less immediate in that case.

In the first case, the software glitch definitely causes mission
failure. In the Google case, the software problems *may* ultimately
cause mission failure, but probably only if nothing is done for quite
some time to rectify the situation. If that is the case, then the
software itself is not the critical factor unless it cannot be
corrected and made to function properly in a reasonable amount of time.

 

[Paul Rubin]

The idea here is that errors in the self-testing code are unlikely
to be correlated with errors in the primary code. Hence, you get a
sort of multiplying effect on reliability. For example, if the
chance of error in the primary code and the self-test code are each
0.01, the chance of an undetected error is approximately 0.01^2 or
0.0001.

But I think you give a lot of that back when you turn the checks off.
The errors you detect when the checks are enabled are only the ones
that result from your test data. Turn off the checks and expose the
application to data from a potentially different distribution, and
you're back where you started.

 

[Hendrik van Rooyen]

This is starting to sound silly, people. Critical is a relative term,
and one project's critical may be anothers mundane. Sure a flaw in your
flagship product is a critical problem *for your company*, but are you
really trying to say that the criticalness of a bad web search is even
comparable to the most important systems on airplanes, nuclear reactors,
dams, and so on? Come on.

This really intrigues me - how do you program a dam? - and why is it
critical?

Most dams just hold water back.

Dam design software - well yes that I would agree is critical.
Is that what you mean?

- Hendrik

 

[Marc 'BlackJack' Rintsch]

This really intrigues me - how do you program a dam? - and why is it
critical?

Most dams just hold water back.

And some produce electricity. And most if not all can regulate how many
water is let through. If something goes wrong the valley behind the dam
gets flooded. If this is controlled by a computer you have the need for
reliable software.

Ciao,
Marc 'BlackJack' Rintsch

 

[llothar]

I have not yet personally used it, but I am interested in anything
that can help to make my programs more reliable. If you are
programming something that doesn't really need to be correct, than you
probably don't need it. But if you really need (or want) your software

I'm one of the few (thousand) hard core Eiffel programmers on this
world
and i can tell you that this would not add to much to python. To get
the
benefits of it you need to use it together with a runtime that is
designed
from ground with DBC and a language that is fast enough to be able to
check the contracts, if you don't have the latter all you get is a
better
specification language (which you can write as comments in python).

Learn the Eiffel design way and then add assert statements whereever
you
need them. Works well when i do C/C++ programming and maybe even for
script languages - but i never used it for scripts as i don't see a
real
value here.

 

[Bryan Olson]

Steve Holden wrote:
[...]

If I can blow my own trumpet briefly, two customers (each using over 25
kLOC I have delivered over the years) ran for two years while I was away
in the UK without having to make a single support call. One of the
systems was actually locked in a cupboard all that time (though I have
since advised that client to at least apply OS patches to bring it up to
date).

This was achieved by defensive programming, understanding the user
requirements and just generally knowing what I was doing.

On the one hand, nice work. Made your customers happy and kept
them happy. Can't argue with success. On the other hand, 25K lines
is tiny by software engineering standards. If a support call would
have to go to you, then the project must be small. Software
engineering is only a little about an engineer knowing or not
knowing what he or she is doing; the bigger problem is that
hundreds or thousand of engineers cannot possibly all know what
all the others are doing.

I work on large and complex systems. If I screw up -- O.K., face
facts: *when* I screw up -- the chance of the issue being assigned
to me is small. Other engineers will own the problems I cause,
while I work on defects in code I've never touched. I wish I could
own all my own bugs, but that's not how large and complex systems
work. Root-cause analysis is the hard part. By the time we know
what went wrong, 99.99% of the work is done.


Design-by-contract (or programming-by-contract) shines in large
and complex projects, though it is not a radical new idea in
software engineering. We pretty much generally agree that we want
strong interfaces to encapsulate implementation complexity.
That's what design-by-contract is really about.

There is no strong case for adding new features to Python
specifically for design-by-contract. The language is flexible
enough to support optionally-executed pre-condition and
post-condition checks, without any extension. The good and bad
features of Python for realizing reliable abstraction are set
and unlikely to change. Python's consistency and flexibility
are laudable, while duck-typing is a convenience that will
always make it less reliable than more type-strict languages.

 

[Jorge Godoy]

This is starting to sound silly, people. Critical is a relative term,
and one project's critical may be anothers mundane. Sure a flaw in your
flagship product is a critical problem *for your company*, but are you
really trying to say that the criticalness of a bad web search is even
comparable to the most important systems on airplanes, nuclear reactors,
dams, and so on? Come on.

Who said they were the same? I said that just because it doesn't take lives
it doesn't mean it isn't important. I wasn't going to reply to not extend
this, but this misunderstanding of your was bugging me.

I use Python on systems that deal with human health and wrong calculations
may have severe impact on a good sized population. Using Python.

As with nuclear reactors, dams, airplanes and so on we have a lot of
redundancy and a lot of checkpoints. No one is crazy to take them out or
even to remove some kind of dispositive to allow manual intervention at
critical points.

 

[Steve Holden]

Steve Holden wrote:
[...]

If I can blow my own trumpet briefly, two customers (each using over 25
kLOC I have delivered over the years) ran for two years while I was away
in the UK without having to make a single support call. One of the
systems was actually locked in a cupboard all that time (though I have
since advised that client to at least apply OS patches to bring it up to
date).

This was achieved by defensive programming, understanding the user
requirements and just generally knowing what I was doing.

On the one hand, nice work. Made your customers happy and kept
them happy. Can't argue with success. On the other hand, 25K lines
is tiny by software engineering standards. If a support call would
have to go to you, then the project must be small. Software
engineering is only a little about an engineer knowing or not
knowing what he or she is doing; the bigger problem is that
hundreds or thousand of engineers cannot possibly all know what
all the others are doing.

I agree that programming-in-the-large brings with it problems that
aren't experienced in smaller scale projects such as the ones I mentioned.

I work on large and complex systems. If I screw up -- O.K., face
facts: *when* I screw up -- the chance of the issue being assigned
to me is small. Other engineers will own the problems I cause,
while I work on defects in code I've never touched. I wish I could
own all my own bugs, but that's not how large and complex systems
work. Root-cause analysis is the hard part. By the time we know
what went wrong, 99.99% of the work is done.

This is the kind of realism I like to see in engineering. I am always
suspicious when any method is promoted as capable of reducing the error
rate to zero.

Design-by-contract (or programming-by-contract) shines in large
and complex projects, though it is not a radical new idea in
software engineering. We pretty much generally agree that we want
strong interfaces to encapsulate implementation complexity.
That's what design-by-contract is really about.

There is no strong case for adding new features to Python
specifically for design-by-contract. The language is flexible
enough to support optionally-executed pre-condition and
post-condition checks, without any extension. The good and bad
features of Python for realizing reliable abstraction are set
and unlikely to change. Python's consistency and flexibility
are laudable, while duck-typing is a convenience that will
always make it less reliable than more type-strict languages.

Python's dynamic nature certainly makes it more difficult to reason
about programs in any formal sense. I've always thought of it as a
pragmatist's language, and we need to be pragmatic about the downside as
well as the upside.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Steve Holden]

Steve Holden wrote:
[...]

If I can blow my own trumpet briefly, two customers (each using over 25
kLOC I have delivered over the years) ran for two years while I was away
in the UK without having to make a single support call. One of the
systems was actually locked in a cupboard all that time (though I have
since advised that client to at least apply OS patches to bring it up to
date).

This was achieved by defensive programming, understanding the user
requirements and just generally knowing what I was doing.

On the one hand, nice work. Made your customers happy and kept
them happy. Can't argue with success. On the other hand, 25K lines
is tiny by software engineering standards. If a support call would
have to go to you, then the project must be small. Software
engineering is only a little about an engineer knowing or not
knowing what he or she is doing; the bigger problem is that
hundreds or thousand of engineers cannot possibly all know what
all the others are doing.

I agree that programming-in-the-large brings with it problems that
aren't experienced in smaller scale projects such as the ones I mentioned.

I work on large and complex systems. If I screw up -- O.K., face
facts: *when* I screw up -- the chance of the issue being assigned
to me is small. Other engineers will own the problems I cause,
while I work on defects in code I've never touched. I wish I could
own all my own bugs, but that's not how large and complex systems
work. Root-cause analysis is the hard part. By the time we know
what went wrong, 99.99% of the work is done.

This is the kind of realism I like to see in engineering. I am always
suspicious when any method is promoted as capable of reducing the error
rate to zero.

Design-by-contract (or programming-by-contract) shines in large
and complex projects, though it is not a radical new idea in
software engineering. We pretty much generally agree that we want
strong interfaces to encapsulate implementation complexity.
That's what design-by-contract is really about.

There is no strong case for adding new features to Python
specifically for design-by-contract. The language is flexible
enough to support optionally-executed pre-condition and
post-condition checks, without any extension. The good and bad
features of Python for realizing reliable abstraction are set
and unlikely to change. Python's consistency and flexibility
are laudable, while duck-typing is a convenience that will
always make it less reliable than more type-strict languages.

Python's dynamic nature certainly makes it more difficult to reason
about programs in any formal sense. I've always thought of it as a
pragmatist's language, and we need to be pragmatic about the downside as
well as the upside.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[=?ISO-8859-1?Q?Ricardo_Ar=E1oz?=]

This really intrigues me - how do you program a dam? - and why is it
critical?

Most dams just hold water back.

Dam design software - well yes that I would agree is critical.
Is that what you mean?

- Hendrik

Yup! He was referring to that Damn design software. Just as almost
everyone has at one time or another. ;c)

 

[Pierre Hanser]

Carl Banks a écrit :

This is starting to sound silly, people. Critical is a relative term,
and one project's critical may be anothers mundane. Sure a flaw in your
flagship product is a critical problem *for your company*, but are you
really trying to say that the criticalness of a bad web search is even
comparable to the most important systems on airplanes, nuclear reactors,
dams, and so on? Come on.

20 years ago, there was *no* computer at all in nuclear reactors.