Does Python really follow its philosophy of "Readability counts"?

[mk]

1) Parallelism. Commodity desktop computers now have 8 effective cpu
cores or maybe even 16 soon (Mac Pro, Intel Core i7) but Python still
has the evil GIL that forces all threads to run on one core. Java,
Erlang, and Haskell (GHC) all beat Python in this area. By the time
Python 4 comes out, we will probably all be using PC's with 32 or more
cores, so the current limitations will be intolerable. Even today,
since no one doing anything serious uses single core machines any
more, the GIL is a huge pain in the neck which the multiprocessing
module helps only slightly. (While we are at it, lightweight threads
like Erlang's or GHC's would be very useful.)

+100 for this one

2) Native-code compilation. Per the Alioth shootouts, Python is much
slower (even on single cores) than Java, Haskell, ML, or even Scheme.
PyPy is addressing this but it will be a while before it replaces
CPython.

The lack of this already causes some pains at my company.

I was flabbergasted to read that optional static typing was dropped by
Guido due to "lack of interest in community" IIRC.

Why!! Among other reasons, this could have provided for very easy
performance optimizations of the small portions of the code! This could
have been a huge gain acquired for little effort!

Regards,
mk

 

[mk]

But, if something is done by convention, then departing from the
convention is by definition unconventional. If you do something
unconventional in a program, it could be on purpose for a reason, or
it could be by accident indicating a bug.

I for one would love to see at least compiler warning (optionally,
error) for situation where attributes are added to self outside of
__init__. I consider it generally evil and hate to see code like that.


Regards,
mk

 

[Paul Rubin]

I was flabbergasted to read that optional static typing was dropped by
Guido due to "lack of interest in community" IIRC.

I don't remember that happening. PEP 3107 still lists type checking
as a use case for Python 3.0 function annotations.

 

[Russ P.]

Russ P. wrote:

[...]

I spent *way* too much time on that post. I really need to quit
spending my time refuting the baloney that passes for wisdom here.

He who cannot ignore baloney is doomed to refute it.

regards
Steve

Yeah, and I should really learn to leave off those little zingers. If
Mr. D weren't obviously an intelligent person and a Python expert, I
wouldn't care what he writes. I just think that he, along with a few
others here, love Python so much that they refuse to recognize its
limitations. It's an easy trap to fall into.

 

[Bruno Desthuilliers]

Russ P. a écrit :

If you "*shouldn't* mess with the implementation", then what is wrong
with enforcing that "shouldn't" in the language itself?

Because sometimes you have a legitimate reason to do so and are ok to
deal with the possible issues.

Why leave to
coding standards and company policy what can be encoded right into the
language?

Because human are smarter than computers.

Why leave to humans (who are known to err) what can be
automated relatively easily? Isn't that what computers are for?

Error is human. For a real catastrophic failure, it requires a computer.

Hey, I share your distaste for Java and C++.

I'm not talking about the languages by themselves, but about the gap
between what they pretend to promote and how they are usually used.

All those "setters" and
"getters" are a kludge. I think Python "properties" are a major step
forward here. Just for fun, I checked to see if Scala has properties.
Guess what? Not only does it have them, but they are generated
automatically for all member data. That's even better than Python
properties!

Oh yes ? "Better", really ? So it's better to have a language that
automagically breaks encapsulation (requiring an additionnal level of
indirection) than a language that do the right thing by default ? I'm
afraid I missed the point ???

Let me try to be very clear here. We are dealing with two separate but
related issues. The first is whether data hiding should be added to
Python.

No need to add it, it's already there : every name that starts with an
underscore is hidden !-)

The second is whether data hiding provides any net benefit in
*any* language.

If it doesn't provide any benefit in Python, then it doesn't provide any
benefit in any language.

As for whether data hiding should be added to Python, I am not arguing
one way or the other. I am merely saying that it is worth considering.

It is not.

Whether it can be added without screwing up the language, I don't
know. If not, then so be it. That just limits the range of domains
where Python is suitable.

That's just plain stupid.

As for whether data hiding provides a net benefit in any language, it
certainly does for large programs and for safety-critical programs.
For large, safety-critical systems, it's a no-brainer.

Only if you fail to use your brain. Now, except for regurgitating the
official OMG prose, do you have *anything* to back these claims ? Python
is older than Java, and there are quite enough man/years of experience
and Python success stories to prove that it *just work*.

I think data
hiding can also provide net benefits for medium-size and even smaller
programs, but lets just consider large programs, so I can shoot down
your argument that data hiding provides no benefit.

C'mon, make my day...

I like to use the example of the flight software for a large
commercial transport aircraft, but many other examples could be given.
How about medical systems that control radiation therapy or
chemotherapy? How about financial systems that could take away your
retirement account in 1.5 milliseconds. Or how about the control
software for the strategic nuclear arsenals of the US or Russia? When
you consider the sea, air, and land-based components, I'm sure that's
one hell of a lot of code!

And ? Such systems have been written (and quite a lot are still running)
with languages way more permissive than Python. You know, languages like
C or assembly. Until you understand that *no technology is idiot-proof*,
you'll get nowhere in "software engineering".

So let's take the airplane example. Boeing and its contractors
probably has hundreds of programmers working on millions of lines of
code.

And ?

If they believed you, they would abandon enforced data hiding,
and programmers would have much more discretion to screw around with
code that they don't work on directly.

Yes. And ?

An FMS programmer could perhaps
decide to change the parameters of the engine controls, for example.

Why on earth would he do something so stupid ?

To prevent that sort of thing from happening, the management could
decree that henceforth all "private" variable names will start with an
underscore. Problem solved, eh?

Certainly not. The only way to solve such a problem is to fire this cretin.

Wait a minute. You yourself have
stated several times that those stupid library developers

Where did I say that library developpers where "stupid" ? Chapter and
verse, please.

can never
predict what the client will actually need.

Now this is a well known fact - applied to "generic" libraries. Nothing
to do with project-specific libraries. Straw man.

So some FMS guy named
Bruno

Fine, personal attack now. Man, beware, this is not going to help you
make your point.

(snip remaining braindead fantasy).

Now, let's talk about a data access violations in the nuclear missile
launch software. Oh, yes, I'm sure the leading underscore convention
is more than sufficient! Wake up, dude, and quit spouting bullsh*t!

Please educate yourself and learn about why Ariane 5 crashed on it's
first flight, due to an error in a module written in ADA (which is such
a psychorigid language that C++ and Java are even looser than Javascript
in comparison). Perhaps will it light a bulb for you.

Yeah, if you live in Nome, Alaska.

Brillant. You obviously failed to understand the differences between
"software engineering" and real life. When it comes to computers, the
only doors I care closing are those which would let someone crack my
computer.

But FWIW, I never close my car. And, in case you don't know, it's
perfectly possible to access "private" members in Java (and, if you do
have access to the source code, in C++ too).

Also, have you ever heard the expression "locks keep honest people
honest"?

Yes. You don't wan't to know what I'm thinking of it.

I spent *way* too much time on that post.

At least something sensible.

I really need to quit
spending my time refuting the baloney that passes for wisdom here.

I don't have any pretention to "wisdom" as far as I'm concerned, unless
you defined wisdom as thinking that experimental results have more
weight than theory.

 

[Bruno Desthuilliers]

Russ P. a écrit :

Russ P. wrote:

[...]

I spent *way* too much time on that post. I really need to quit
spending my time refuting the baloney that passes for wisdom here.

He who cannot ignore baloney is doomed to refute it.

Yeah, and I should really learn to leave off those little zingers. If
Mr. D weren't obviously an intelligent person and a Python expert, I
wouldn't care what he writes. I just think that he, along with a few
others here, love Python so much that they refuse to recognize its
limitations. It's an easy trap to fall into.

Python surely has lot of limitations (and quirks and whatever). The
point is that not having language-enforced access restrictions is *not*
a limitation - because it's *not* the right way to solve the problems
you are (very rightfully) concerned with. No technology can solve
*human* problems. Anyone trying to make you believe otherwise is selling
you snake oil. Once again, there's quite a lot to learn from the story
of Ariane 5.

 

[Paul Rubin]

Once again, there's quite a lot to learn from
the story of Ariane 5.

Do you know what actually happened with Ariane 5? The failure was
because "smart" humans overrode the language enforced protection by
casting a floating point number down to a 16-bit integer, which worked
ok in Ariane 4, but failed with an overflow on Ariane 5 where bigger
numbers were involved. The actual code fragment is here, and you can
see where the error is suppressed:

http://www-aix.gsi.de/~giese/swr/ariane5.html

This is one thing that Python gets right, automatically using bignums
rather than allowing int overflow. In that sense, Python has more
enforced protection than Ada.

See also: http://en.wikipedia.org/wiki/Ariane_5_Flight_501

 

[Russ P.]

No one ever claimed that a programming language, no matter how
rigorous, can eliminate all bugs. All a language can do is to reduce
their rate of occurrence.

The Ariane fiasco was not a failure of Ada per se but rather a failure
of people using Ada. They attempted to re-use software written for one
rocket for another without proper testing. No language can prevent
that sort of error.

We can argue forever about the usefulness of language-enforced
restriction of access to private data and methods. I have no doubt
whatsoever that it is very useful at least for the extreme cases of
very large, safety-critical systems. If you don't think access to
private data needs to be restricted for control of strategic nuclear
arsenals, for example, I think you're crazy, but that's just my
opinion.

The only reasonable question in my mind is where the line should be
drawn between systems that should have enforced restrictions and those
that can rely on coding standards and voluntary cooperation among
programmers.

A while back, I read something about the final integration of the
flight software on the Boeing 777, which was written mainly in Ada.
The claim was made that this integration took only three days, whereas
normally it would be expected to take more like three months with a
less rigorous language such as C++. The reason for the simplified
integration is that Ada enforces interfaces and prevents access to
private data and methods. Apparently such enforcement can improve the
efficiency of software production -- and that's not just in "theory."

 

[Russ P.]

Please educate yourself and learn about why Ariane 5 crashed on it's
first flight, due to an error in a module written in ADA (which is such
a psychorigid language that C++ and Java are even looser than Javascript
in comparison). Perhaps will it light a bulb for you.

The claim here regarding the Ariane 5 failure is one of those urban
myths that refuses to die. It has been refuted over and over
(including on this thread already), but I would just like to add
something in defense of Ada.

Studies have found that Ada reduces both bugs and long-term
development costs by something like a factor of two compared to C. For
example, see

http://www.adaic.com/whyada/ada-vs-c/cada_art.html

Also, for a good discussion of the success of Ada in the 777, see

http://www.adaic.org/atwork/boeing.html

 

[Russ P.]

Do you know what actually happened with Ariane 5?  The failure was
because "smart" humans overrode the language enforced protection by
casting a floating point number down to a 16-bit integer, which worked
ok in Ariane 4, but failed with an overflow on Ariane 5 where bigger

So this turns out to be an example of a failure due, not to the
*rigidity* of Ada, but to its *permissiveness* in allowing such a
cast. Had such a cast not been allowed, the people who complain about
the "rigidity" of Ada would have complained that much more.

I don't know which variant of Ada was used here, but something called
the "Ravenscar Profile" is a reduced subset of Ada that might have
prevented this error (though I haven't verified this). Then there is
Spark Ada, which supposed to be much safer than even Ada.

numbers were involved.  The actual code fragment is here, and you can
see where the error is suppressed:

 http://www-aix.gsi.de/~giese/swr/ariane5.html

This is one thing that Python gets right, automatically using bignums
rather than allowing int overflow.  In that sense, Python has more
enforced protection than Ada.

True, but Ada does not have the luxury of just using doubles and
"bignums" everywhere, because it needs to work on cheap processors
too. But perhaps it could somehow be configured to do so by the user
if sufficiently powerful computers are being used.

 

[Simon Forman]

  I am sorry all I am not here to just blame Python. This is just an
introspection of whether
what I believe is right. Being a devotee of Python from past 2 years I
have been writing only
small apps and singing praises about Python where ever I go. I now got
a chance to read
Django's code for some reason. I have now strongly started feeling if
Python really follows its
"Readability Counts" philosophy. For example,

    class A:
    a = 10
    b = "Madhu"

    def somemethod(self, arg1):
        self.c = 20.22
        d = "some local variable"
        # do something
        ....
    ...
    def somemethod2 (self, arg2):
        self.c = "Changed the variable"
        # do something 2
        ...

In such situations, where the Instance variables come into existence
only when they are used
it is very difficult to track the flow of code. Its obviously not
possible to remember what
instance variable was defined where, when reading some substantial
amount of code and where
it was manipulated for that matter. It becomes so very frustrating
even when reading a Class's
code with just 6-8 methods and not more than 100-150 lines of code.

I am interested in knowing if I am reading this kind of code in the
wrong way mostly because
of C++/Java hangover since most other languages follow the same
approach as them? If there
is a Pythonic way reading this code for better readability? What made
Python developers to
adopt this strange strategy keeping "Readibility Counts" in mind?

-- Python Rocks!
   Madhusudan.C.S

Python doesn't follow philosophies, people follow philosophies.

;-)
~S

 

[Steven D'Aprano]

Russ P. a écrit :

Russ: There are SHOULD NOTs and there are MUST NOTs.

In Python, direct access to pointers is a MUST NOT. So the language
itself prohibits arbitrary access to memory, and pure Python programs
aren't subject to the same security holes and crashes that C programs are
subject to because of their use of pointers.

Messing with the implementation is a SHOULD NOT. There are times
(possibly rare) where you are allowed to mess with the implementation.
Hence the language doesn't prohibit it, only discourage it. The language
designers choose how many hoops you have to jump through (and what
performance penalty you suffer) in order to mess with the implementation.

Because sometimes you have a legitimate reason to do so and are ok to
deal with the possible issues.

Bruno: Yes, but most of the time you don't.

The consequence of this dynamism is that the Python VM can't do many
optimizations at all, because *at any time* somebody might mess with the
implementation. But 90% of the time nobody does, so Python is needlessly
slow 90% of the time. Wouldn't it be nice if there was a way to speed up
that 90% of the time while still allowing the 10% to take place?

The current solution to this problem is to try to push as much as
possible into functions written in C: built-ins and custom C extensions.
It's not a bad solution: most Python programs rely on many C built-ins,
which enforces real encapsulation and data hiding. With the possible
exception of mucking about with ctypes, you simply can't access mess with
the internals of (say) lists *at all*. Is this a bad thing?

Would it be so terrible if we could do the same thing in pure Python? Why
should I have to write in C if I want the same protection?

Because human are smarter than computers.

That's an awfully naive statement. It's a sound-byte instead of a
reasoned argument. We're smarter than computers? Then why are we
programming in languages like Python instead of directly in machine code?
Why can optimizing C compilers make more efficient code than the best
human assembly language programmers?

Humans and computers are smart at different things. Human beings are not
terribly good at keeping track of more than about seven things at once,
on average, and consequently we easily forget what scope variables are
in. It's probably been at least 15 years since any released version of
Python has been buggy enough to "forget" whether a name was in one scope
or another, and yet human programmers still generate NameError and
AttributeError exceptions *all the time*. I bet even Guido still makes
them occasionally.

Error is human. For a real catastrophic failure, it requires a computer.

Oh rubbish. That's a sound-byte invented by nervous technophobes scared
of computers. I expected better from a programmer. When computers fail,
it's is almost certainly because of human error: some human being wrote
buggy code, some human being turned off a test because it was generating
too many warnings, some human being failed to prove their code was
correct.

Chernobyl was a catastrophic failure that happened when *humans* turned
off their safety systems to do a test, then couldn't turn them back on.

Oh yes ? "Better", really ? So it's better to have a language that
automagically breaks encapsulation (requiring an additionnal level of
indirection) than a language that do the right thing by default ? I'm
afraid I missed the point ???

You certainly do. How do properties "break" encapsulation rather than
enforcing it?

No need to add it, it's already there : every name that starts with an
underscore is hidden !-)

That's not hidden. It's there in plain sight.

In Unix, file names with a leading dot are hidden in the shell: when you
do a file listing, you don't see them. It's not difficult to get to see
them: you just pass -a to the ls command. As data hiding goes, it's
pretty lame, but Python doesn't even suppress _ names when you call dir.
Frankly, I wish that by default it would -- 99% of the time when I call
dir, I *don't* want to see _ names. They just get in the way.


[...]

That's just plain stupid.

No it's not. It's *practical*. There are domains where *by law* code
needs to meet all sorts of strict standards to prove safety and security,
and Python *simply cannot meet those standards*.

Only if you fail to use your brain. Now, except for regurgitating the
official OMG prose, do you have *anything* to back these claims ? Python
is older than Java, and there are quite enough man/years of experience
and Python success stories to prove that it *just work*.

Nobody doubts that Python works for many applications. But can you point
to any large, safety-critical system programmed in Python?

And ? Such systems have been written (and quite a lot are still running)
with languages way more permissive than Python. You know, languages like
C or assembly.

Yes, and it is *hard* because the programmer has to worry about data
hiding *on his own*. That's why people no longer write large systems in
assembly and use high-level languages that deal with all those data
hiding issues for you.

One of my friends has worked for many years programming some pretty high-
powered banking software. Their approach is to move data-hiding into the
database, or the operating system. Cobol doesn't enforce encapsulation,
but the database and OS certainly do, with a permissions-based approach.

Speaking of banking software, consider a typical banking application. It
may have dozens or hundreds of programmers working on it. It's too big
for any one person to understand all of it. Once deployed it may
potentially have access to hundreds of billions of dollars of other
people's money. Don't you imagine that one or two of these programmers
might be tempted to skim a little off the top?

Data hiding is a good way of making sure that the guy writing the front
end can't just turn of the audit trail and transfer $60,000,000 into his
bank account. Why don't you approach your bank and suggest that it would
be a Good Thing if he could? Think of the programming time they would
save with the added dynamism! Why, it might shave off *weeks* from a six
year project!

Until you understand that *no technology is idiot-proof*,
you'll get nowhere in "software engineering".

I suspect that Russ has got a lot further in software engineering than
you have. I suspect your attitude is part of the reason why, as they say,
"If engineers built bridges the way programmers build software, the first
woodpecker than came along would destroy civilization".

No technology is failure proof. But there's no reason in the world why
most technology can't be idiot-proof. Televisions are idiot-proof,
because they protect people from casual mistakes. If televisions were
built according to the Python model, the internals of the TV would be
exposed, without even a cover. All the major parts would be plug-in
rather than soldered in, and there would be no cover over the parts that
were live. Every year, tens of thousands of people would electrocute
themselves fatally (because parts of the TV holds a massive charge for
days after you unplug them from the mains) but that would be okay,
because you never know when somebody might want to pull out the fly-back
transformer and replace it with a six ohm resistor. That sort of dynamism
is important!

Well, maybe so, but not in a television. Consequently televisions are
built to hide the internals from the user. If you *really* want to, then
you can get a screwdriver and remove the back and unsolder the fly-back
transformer and replace it with a six ohm resistor. It's your TV, do what
you want. But nobody is going to die because the picture was fuzzy and
they heard from some chat forum that the way to fix that was to poke the
back of the picture tube with a screw driver "to let out the excess ohms".



[...]

Why on earth would he do something so stupid ?

I'm sure he'd think he had a good reason. As you said,

"Because sometimes you have a legitimate reason to do so and are ok to
deal with the possible issues."

Maybe other people would disagree whether or not it was a legitimate
reason, or if he was OK dealing with the possible issues.

Perhaps he needed the extra optimization of skipping the getter/setters.
Perhaps he needed it for testing, and somehow one thing led to another.
Who knows?

It is strange that on the one hand you should insist that programmers
sometimes need to mess with internals, and on the other dismiss those who
do as "stupid". Your position is inconsistent.

Certainly not. The only way to solve such a problem is to fire this
cretin.

Again, we shouldn't enforce encapsulation and data hiding because there
are legitimate reasons for breaking it, but anyone who does break it is a
cretin. You have a very strange attitude.

Besides, it's not very practical. When you fire "the cretin", it has
consequences. Everyone else in the project has to work harder, which has
costs, or you have to replace him, which also has costs. Maybe you can't
fire him, because he's the only one who can debug problems in the auto-
pilot. Perhaps the rest of the team downs tools and walks off the job in
sympathy. Perhaps he sues you for unfair dismissal. Expenses rise. Time-
lines slip. Uncertainty increases.

It's also bad for moral when you fire somebody for messing with the
internals when you have a policy that it is allowed to mess with the
internals. That's why you picked a dynamic language like Python in the
first place, because it doesn't prevent you from messing with the
internals. And now when somebody does, you sack him? If you ask me, it's
management who needs to be sacked, not the programmer who merely used the
tools given to him.

Perhaps you can't fire the cretin, because the first time you discover
the problem is eight years later when a place filled with nuns and
orphans flips upside down and flies straight into the ground.

Perhaps it would have been better to prevent him from messing with the
internals in the first place, even at some extra cost. When you're in
business, you have to make decisions like:

* do I write the software in Python, which will have a 99% chance of
costing $100,000 and a 1% chance of costing $100,000,000?

* or do I write it in a B&D language like Java, which will have a 100%
chance of costing $2,000,000?


[...]

Please educate yourself and learn about why Ariane 5 crashed on it's
first flight, due to an error in a module written in ADA (which is such
a psychorigid language that C++ and Java are even looser than Javascript
in comparison). Perhaps will it light a bulb for you.

Others have already pointed out that the error in the Ariane 5 rocket was
*human* error due to somebody messing with the internals, namely
defeating the compiler's default type checking. I'd just like to ask:
Bruno, were you aware of the cause of the crash, and if not, why did you
raise the issue in the first place? Did you think it was a compiler bug
that caused the crash?

Brillant. You obviously failed to understand the differences between
"software engineering" and real life. When it comes to computers, the
only doors I care closing are those which would let someone crack my
computer.

If only there was some way to know which bugs could let people crack our
computer and which bugs couldn't.

Anyway, that's your choice. Personally, I'd much prefer my software not
to cause data loss, not to crash, not to DoS me, not to hang, and not to
generate bogus data, as well as not letting strangers crack into my
system. There are many, many program paths which could potentially lead
to these results. It would be nice if I could close those doors.

But FWIW, I never close my car. And, in case you don't know, it's
perfectly possible to access "private" members in Java (and, if you do
have access to the source code, in C++ too).

Yes, but it is more difficult. There's a larger psychological barrier.
It's easier to audit for such access, even in a large project. It
encourages a more careful attitude: "do I *really* need to mess with the
internals, or is there a safer way?". It forces the programmer to *think*
before doing something potentially dangerous.

This is why I wish eval and exec were in a module instead of built-ins.
They'd still be there, but you'd have to jump through one small hoop
before using them.

Suppose Python worked like this:

.... _private = 'spam'
....Traceback (most recent call last):
'ham'


I don't know if this scenario is even possible in Python, but pretend
that it is. Would it be so terrible? If a particular project wanted to
enforce encapsulation, all they need do is replace or remove the
protection module from their Python installations. (I assume the project
developers aren't *hostile*. If they are, then there's almost nothing you
can do to make the code safe. Encapsulation is about protecting from
accidents, not sabotage.) If you wanted to mess with the internals in
your own project, all you need do is import a module.


We could imagine the same scenario in reverse. Python allows getters and
setters, but they're more work, and so people just don't use them unless
they really need to. Suppose Python offered real data encapsulation, but
you had to work to get it:
.... _private = 'spam'
....Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ProtectionError: attribute is read-only from outside of class Parrot


Would that be so bad? I don't think so.

 

[Russ P.]

Wow! That was an instant classic! I just have a couple of points to
add.

The suggestion was made (not by you) that data hiding is worthless
because it can be defeated anyway. According to that kind of
reasoning, locks are worthless because they can be picked, cut, or
just blown off. I know that the lock on my front door cannot stop
someone who is determined to get into my house, but I think I'll keep
it anyway. I'm just irrational that way, I guess.

As I said before, I don't know if enforced data hiding can be added to
Python without ruining the language (or if it can be added at all, for
that matter). But if it can, I think the cleanest and most elegant
syntax would be to add the keyword "private" or "priv" and use it
essentially the same way it is used in Java, C++, and now Scala. That
would eliminate the need for leading underscores, which I personally
find tacky.

Since new attributes can be added outside of the constructor, the
private declaration would also have to be allowed outside the
constructor and for existing attributes.

One more thing. If an airplane is going to flip upside down and fly
straight into the ground, the passengers may as well be nuns. They are
better prepared for the result.

 

[Bruno Desthuilliers]

Paul Rubin a écrit :

Do you know what actually happened with Ariane 5?

*yes I do* - else I wouldn't mention it. Thanks.

The failure was
because "smart" humans overrode the language enforced protection by
casting a floating point number down to a 16-bit integer, which worked
ok in Ariane 4, but failed with an overflow on Ariane 5 where bigger
numbers were involved.

The failure was because a module tested, QA'd and certified within a
given context (in which it was ok to drop the builtin error handling)
was reused in a context where it was not ok. And the point is exactly
that : no *technology* can solve this kind of problem, because it is a
*human* problem (in that case, not taking time to repass the whole specs
/ tests / QA process given context change).

 

[Bruno Desthuilliers]

Russ P. a écrit :

So this turns out to be an example of a failure due, not to the
*rigidity* of Ada, but to its *permissiveness* in allowing such a
cast.

Nope. This is an example of failure due to the *human* part of the
process - this happened because of a lack of testing / QA, not because
of a language feature or misfeature.

(snip)

True, but Ada does not have the luxury of just using doubles and
"bignums" everywhere, because it needs to work on cheap processors
too. But perhaps it could somehow be configured to do so by the user
if sufficiently powerful computers are being used.

Here the error was *not* to disable the overflow error checking (in the
context of Ariane 4), but to reuse the module as-is in another context.

As I already stated, no technology can protect us from this kind of
error. Ask yourself why this module was reused as-is, instead of going
thru the whole specs / tests / QA process again, and *perhaps* you'll
start to understand why I say that language-enforced access restrictions
are the wrong solution to a real problem.

 

[Bruno Desthuilliers]

Russ P. a écrit :

No one ever claimed that a programming language, no matter how
rigorous, can eliminate all bugs. All a language can do is to reduce
their rate of occurrence.

The Ariane fiasco was not a failure of Ada per se but rather a failure
of people using Ada.

Almost right.

They attempted to re-use software written for one
rocket for another without proper testing. No language can prevent
that sort of error.

Now this is plain right.

We can argue forever about the usefulness of language-enforced
restriction of access to private data and methods. I have no doubt
whatsoever that it is very useful at least for the extreme cases of
very large, safety-critical systems.

And my POV is that it's just plain useless.

If you don't think access to
private data needs to be restricted for control of strategic nuclear
arsenals, for example, I think you're crazy, but that's just my
opinion.

If you think that this kind of access restriction makes softwares
controling strategic nuclear arsenal any safer, then *you* are totally
crazy. As *you* stated above, "no language can prevent this kind of error".

The only reasonable question in my mind is where the line should be
drawn between systems that should have enforced restrictions and those
that can rely on coding standards and voluntary cooperation among
programmers.

The only reasonable question in *my* mind is whether you think it's
better, specially for safety-critical stuff, to hire people you can
trust or to rely on technology.

A while back, I read something about the final integration of the
flight software on the Boeing 777, which was written mainly in Ada.
The claim was made that this integration took only three days, whereas
normally it would be expected to take more like three months with a
less rigorous language such as C++. The reason for the simplified
integration is that Ada enforces interfaces and prevents access to
private data and methods.

C++ does it too. Or at least, that's a very common "claim" about C++.

Apparently such enforcement can improve the
efficiency of software production -- and that's not just in "theory."

My own experience is that too much rigidity in a language only leads to
more boilerplate code and more workarounds (design patterns anyone ?),
IOW more accidental complexity, hence more space for bugs to creep in.

My very humble (no don't say it) opinion about this is that what's
important is how you handle and manage the whole project (including both
technical and non-technical aspects), not the technology used (which
might be relevant - I of course wouldn't use Python for anything
real-time - or not - most of the monstruogigantic "entreprise" software
written in Java would work just as well in Python or any other decent
language).

No technology will prevent human errors - I think this point is clear,
and that we both agree on it. OTHO, some technologies can at least help
reducing the opportunities for human errors - and I think we also agree
on this. Now the point is : *how* can a given techno helps wrt/ this
second goal. Here you have two philosophies. One is that you'll reduce
errors by improving simplicity and readability, and by promoting trust
(whithin the team), sense of responsabily, and communication. The other
one is that you'll reduce errors by not allowing those stupid
code-monkeys to do anything that *might* (according mostly to unproved
assertions) be "dangerous" - hence promoting distrust and
irresponsability, and adding quite a lot of accidental complexity. BTW,
do you know the by far most common Java idiom ? It's named the
"do-nothing catch-all exception handler". As an excercise, explain the
reasons behind this idiom, and it's practical results.

As a last point : I certainly don't think Python is perfect in any way,
*nor* that it's the appropriate tool for each and any project. I just
disagree with you on your assertion that Python is not ok for large or
critical projects because of it's "lack" of language-enforced access
restriction.

 

[Bruno Desthuilliers]

Russ P. a écrit :

The claim here regarding the Ariane 5 failure is one of those urban
myths that refuses to die.

You failed the test. Sorry.

It has been refuted over and over

What has been refuted exactly ?

(including on this thread already), but I would just like to add
something in defense of Ada.

Studies have found that Ada reduces both bugs and long-term
development costs by something like a factor of two compared to C.

This _might_ (or not - there are too many variables to get any
scientific certitudes here) be true. But that was not the point. See my
other posts here for more about it.

 

[Paul Rubin]

I don't know which variant of Ada was used here, but something called
the "Ravenscar Profile" is a reduced subset of Ada that might have
prevented this error (though I haven't verified this). Then there is
Spark Ada, which supposed to be much safer than even Ada.

I'm not sure, but I think Ravenscar and Spark would both have
permitted the cast. However, Spark is intended to be used with an
external tool called Spade (sort of a Pylint on steroids) that would
not have allowed the cast unless it was provable (starting from the
specification) that the number was in range before the cast.

I.e. the cast was wrong because of the failure of an unstated
assumption that a certain sensor reading was in a certain range.
Spark may still have allowed the cast only if the assumption was
stated explicitly in the specification. Requiring the assumption to
be stated may have made the error more likely to be spotted as part of
the surrounding systems engineering. Of course, the specification can
still be wrong, but that's hardly a problem with Ada or even with the
software. It's not too much different than if the specification says
the rocket is 10 feet wide and they build the launch pad for that
size, but then the rocket turns out to actually be 12 feet wide.

I have Barnes's book about Spark at the office and will try to check
next week what it says about these casts.

 

[Paul Rubin]

As I already stated, no technology can protect us from this kind of
error. Ask yourself why this module was reused as-is, instead of going
thru the whole specs / tests / QA process again, and *perhaps* you'll
start to understand why I say that language-enforced access
restrictions are the wrong solution to a real problem.

I have not seen anywhere that the specs stated that the sensor reading
that overflowed that variable was supposed to be in range. It looks
to me (but I don't know for sure) that the development process allowed
an assumption to creep into the code that wasn't stated in the specs.

 

[Bruno Desthuilliers]

Steven D'Aprano a écrit :

(snip)

That's an awfully naive statement. It's a sound-byte instead of a
reasoned argument. We're smarter than computers?

Obviously and definitively, yes. Not that we are that smart - it's just
that computers are totally stupids. The differences between a human and
a computer are that
1/ a human can eventually DWIM.
2/ a computer can do what is has been told way faster than any human

The first point is being smart. The second is being fast. Not quite the
same thing !-)

Then why are we
programming in languages like Python instead of directly in machine code?
Why can optimizing C compilers make more efficient code than the best
human assembly language programmers?

Because :
1/ this is something that can be solved by heavy computations
2/ these computations can be programmed
3/ these compuations would just take too long for a human being to do
manually

Humans and computers are smart at different things.
s/smart/good/g


Oh rubbish. That's a sound-byte invented by nervous technophobes scared
of computers.

Nope, that's a computer-user joke.

You certainly do. How do properties "break" encapsulation rather than
enforcing it?

Properties by themselves are not the problem, quite on the contrary - as
you say, they actually help wrt/ encapsulation. What breaks
encapsulation is *automatic generation* for properties for *each and
any* implementation attribute. Might as well just makes them all public
attribute then.

That's not hidden. It's there in plain sight.

Once again, I'm afraid you missed the joke - despite the smiley.

No it's not. It's *practical*. There are domains where *by law* code
needs to meet all sorts of strict standards to prove safety and security,
and Python *simply cannot meet those standards*.

Oh, sorry. I was talking about *technical* issues, not about legal ones.
IANAL...

(snip)

Yes, and it is *hard* because the programmer has to worry about data
hiding *on his own*.

Nope, it's hard because C and assembly are very low-level languages
where you have to micro-manage each and everything. This has nothing to
do with data hiding (unless you count memory management as data hiding ?)

One of my friends has worked for many years programming some pretty high-
powered banking software. Their approach is to move data-hiding into the
database, or the operating system. Cobol doesn't enforce encapsulation,
but the database and OS certainly do, with a permissions-based approach.

Guess what ? Whatever the language (Cobol, Java, Python, or even VB),
chances are such an application would be written using a RDBMS (not even
addressing the point about OS).

Speaking of banking software, consider a typical banking application. It
may have dozens or hundreds of programmers working on it. It's too big
for any one person to understand all of it. Once deployed it may
potentially have access to hundreds of billions of dollars of other
people's money. Don't you imagine that one or two of these programmers
might be tempted to skim a little off the top?

There have been some major failure on such applications written with
"serious" languages like Java. Know the sooo common "catch-all
do-nothing exception handler" idiom in Java ?

Data hiding is a good way of making sure that the guy writing the front
end can't just turn of the audit trail and transfer $60,000,000 into his
bank account.

Man, you can't be serious ? Tell me this is a joke ?

I suspect that Russ has got a lot further in software engineering than
you have.

Most of what is actually labelled as "software engineering" is snake
oil. They're trying to sell you silver bullets. Now where's the werewolf?

No technology is failure proof. But there's no reason in the world why
most technology can't be idiot-proof. Televisions are idiot-proof,
because they protect people from casual mistakes. If televisions were
built according to the Python model, the internals of the TV would be
exposed, without even a cover. All the major parts would be plug-in
rather than soldered in, and there would be no cover over the parts that
were live. Every year, tens of thousands of people would electrocute
themselves fatally (because parts of the TV holds a massive charge for
days after you unplug them from the mains) but that would be okay,
because you never know when somebody might want to pull out the fly-back
transformer and replace it with a six ohm resistor. That sort of dynamism
is important!

Really nice metaphor. Alas, there are limits to what you can do with
metaphores.

(snip remaining)

[...]

Why on earth would he do something so stupid ?

I'm sure he'd think he had a good reason. As you said,

"Because sometimes you have a legitimate reason to do so and are ok to
deal with the possible issues."

Maybe other people would disagree whether or not it was a legitimate
reason, or if he was OK dealing with the possible issues.

Perhaps he needed the extra optimization of skipping the getter/setters.
Perhaps he needed it for testing, and somehow one thing led to another.
Who knows?

It is strange that on the one hand you should insist that programmers
sometimes need to mess with internals, and on the other dismiss those who
do as "stupid". Your position is inconsistent.

Nope. My position is quite clear : just don't do something *stupid*. For
a definition of "stupid" that is indeed highly dependant on the context
(which you snipped).

Again, we shouldn't enforce encapsulation and data hiding

s/encapsulation//g

We were talking about language-enforced access restriction. Not about
encapsulation.

because there
are legitimate reasons for breaking it, but anyone who does break it is a
cretin.

Not "anyone". Just those who do it where they obviously should not. And
be sure that anyone cretin enough to mess with implementation *in the
context mentioned by Russ* (context which, once again, you snipped)
would find a "clever" way to do the same (or worse) in any language. You
don't want such a smartass in your team, *specially* for critical stuff.

You have a very strange attitude.

You have a very strange way of handling context.

Besides, it's not very practical. When you fire "the cretin", it has
consequences. Everyone else in the project has to work harder,

My own experience is that firing an obvious cretin from a team leads to
*less* work for the team - because then you don't have to re-read and
rewrite all his code.

[...]

Please educate yourself and learn about why Ariane 5 crashed on it's
first flight, due to an error in a module written in ADA (which is such
a psychorigid language that C++ and Java are even looser than Javascript
in comparison). Perhaps will it light a bulb for you.

Others have already pointed out that the error in the Ariane 5 rocket was
*human* error due to somebody messing with the internals, namely
defeating the compiler's default type checking.

This was *not* an error *in the context* this module was designed (and
tested, QAd etc) for.

I'd just like to ask:
Bruno, were you aware of the cause of the crash,

Yes I was. Else I wouldn't have mention it.

and if not, why did you
raise the issue in the first place?

Because the problem had nothing to do with technology. Only with not
being serious on specs / tests / QA. Hence my point : it's not because a
module has been written using a technology (rightly) known for it's
security and robustness that you can get by with non-technical issues.

Did you think it was a compiler bug
that caused the crash?

Do you think I am a newbie ?

If only there was some way to know which bugs could let people crack our
computer and which bugs couldn't.
>
Anyway, that's your choice. Personally, I'd much prefer my software not
to cause data loss, not to crash, not to DoS me, not to hang, and not to
generate bogus data,

This is a different problem. Please don't mix bananas and screwdrivers.

Yes, but it is more difficult. There's a larger psychological barrier.
It's easier to audit for such access, even in a large project. It
encourages a more careful attitude: "do I *really* need to mess with the
internals, or is there a safer way?". It forces the programmer to *think*
before doing something potentially dangerous.

This is a beautiful dream. Reality is that either the guy is wise enough
to not do something he know he shouldn't without due thinking and
consideration, or he is dumb enough to do something he shouldn't one way
or another.

This is why I wish eval and exec were in a module instead of built-ins.

I do agree they shouldn't be builtins, but mostly because there are too
few real use case for them to be builtins. (well, wrt/ exec, it's a
statement to the problem is a bit different).

Suppose Python worked like this:


... _private = 'spam'
...
Traceback (most recent call last):

'ham'
>

I don't know if this scenario is even possible in Python,

It actually isn't, at least not without way too much overhead messing
with call stack (and even then...).

"Methods" need to freely access implementation attributes. But Python's
"methods" are just wrappers around plain functions, which by themselves
don't know (and shouldn't need to know) if they where defined within a
class statement or not.

but pretend
that it is.

Ok, let's pretend !-)

Would it be so terrible?

No, I could live with it. Just a question now : how would this deal with
functions defined outside the class statement but used as methods ?

Oh, and please read until the end...

If a particular project wanted to
enforce encapsulation,

s/encapsulation/data hiding/

all they need do is replace or remove the
protection module from their Python installations. (I assume the project
developers aren't *hostile*. If they are, then there's almost nothing you
can do to make the code safe. Encapsulation is about protecting from
accidents, not sabotage.)

InMyArms(tm) !-)

If you wanted to mess with the internals in
your own project, all you need do is import a module.


We could imagine the same scenario in reverse. Python allows getters and
setters, but they're more work,

And more overhead.

and so people just don't use them unless
they really need to.

Which is fine. Well, MHO at least.

Suppose Python offered real data encapsulation,
s/encapsulation/hiding/

but
you had to work to get it:

... _private = 'spam'
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ProtectionError: attribute is read-only from outside of class Parrot


Would that be so bad? I don't think so.

So technically, this means that you'd have an optionel check for
_implementation attribute access outside methods ?

Good news my friend, Pylint already do this (and quite a lot of other
things too...). So wrt/ code reviews, QA etc, the "lack of access
protection" is not such an issue, is it ?


http://www.logilab.org/card/pylintfeatures
"""
W0212: Access to a protected member %s of a client class
Used when a protected member (i.e. class member with a name beginning
with an
underscore) is access outside the class or a descendant of the class where
it's defined.
"""