Does Python really follow its philosophy of "Readability counts"?

[Bruno Desthuilliers]

Russ P. a écrit :
(snip)

Wait a minute. Aren't the guy who just took me to task about the
definition of functional programming? So the definition of functional
programming is written in stone, but the definition of OO programming
is written in smoke?

Well, actually, the answer is mostly "yes". Functional programming is
based on a (mathematical) theoretical ground about calculation (google
for Alonzo Church and lambda calculus), while OO is originally nothing
more than a way to implement finite state machines with an imperative
language. OO *is* imperative programming (which is itself also well
defined).

Just for the record, I really don't care much about the definition of
OO programming. I brought it up only because someone tried to claim
that "enforced" encapsulation

data hiding.

is a terrible idea. Well, as far as I
can tell, the majority of OO "programmers" (and software engineers,
software architects,

playing buzzword bingo ?

etc.) seem to think otherwise. Maybe they are

wrong -- but I seriously doubt it.

The first OO languages (at least the second one - Smalltalk) used
data-hiding to clearly emphasize the "black box" nature of objects and
the use of messages as the main (only in the case of Smalltalk) support
for control flow. Remember than by that time, lost of programs where
still mostly relying on *global* state changes. IOW, it has a strong
educative value then...

Following "OO" languages - mostly C++ and Java - kept this "rule" like
it was a sacred cow (but mostly forgot about the more important points
of 'everything is an object' and message-passing as main control flow).
Then everyone started considering this as "fundamuntal", and here we are
years later with one more cargo cult, when years of experience prove
that it's not - at least from a practical POV.

Once again, the important point is that there's a *clear* distinction
between interface and implementation, and that you *shouldn't* mess with
implementation. But what, some people think programmers are stupid, and
so they hire stupid programmers, so they need b&d languages to protect
stupid programmers from themselves - which just doesn't work, since
*nothing* is idiot-proof. Heck, how many Java "OO" programs with dumb
getter/setter pairs for _each and any_ attribute ?

As I said before, enforced encapsulation may not be appropriate for
every application, but it is definitely appropriate for some.

No. It is appropriate for dummy managers hiring dummy programmers. The
project's size and domain have nothing to do with it.

Not
every door needs a lock, but certainly some do.

You only need locks when you don't trust your neighbours.

 

[Terry Reedy]

Usually it's "expected to find some value but got None", or got a
list, or expected some structure but got a different one, or some
field was missing, etc. It's not a single traceback, it's a recurring
theme in developing this stuff.

I do not see any connection, really, between what you describe above and
your desire for static type-checking expressed elsewhere. When I was
regularly doing analysis of empirical data files, I learned (sometimes
the hard way, as you describe above) to **ALWAYS** run preliminary
checks of all fields through the entire file so I would know the actual
range of values of each and whether the ranges comported with the
requirements of the intended analyses. The principle was/is the same
whether using BMDP, SAS, C, or Python.

Terry Jan Reedy

 

[Terry Reedy]

I would be fine having something like pylint built-in in the language
and running at every change of the source code (unless disabled with a
command line switch). I think this is the only reasonable solution to
get some additional protection we can hope for. A true change of the
language IMO is impossible, both technically, politically and for
legacy issue. Also, I am not convinced it would be a good idea, even
theoretically. It is easier to write a new Python-like language
from scratch than to add type checking to Python (I think you
were not proposing adding type checking in this post, right?).

Integrating checkers with the interpreter is something that could/should
be done by an IDE. It should not be too hard, for instance, to give
IDLE a configurable run-thru-PyLint option keyed, for instance, to
shift-F5 (given that F5 is run-with-CPython). Alt-F5 for instance could
be run-Pylint-and-if-no-errors-then-CPython.

 

[Terry Reedy]

Yes, and I use Python because it is a language that holds my hand,
otherwise I would use C and enjoy segmentation faults all the time.

So perhaps it is a matter of which handholding one wants and which one
hates. To avoid all, stick with assembler ;-).

tjr

 

[Paul Rubin]

I do not see any connection, really, between what you describe above
and your desire for static type-checking expressed elsewhere. When I
was regularly doing analysis of empirical data files, I learned
(sometimes the hard way, as you describe above) to **ALWAYS** run
preliminary checks of all fields through the entire file

Right. And if the file is large enough that even parsing all the
records to check the fields takes hours, well, that's where I'm at.

 

[Terry Reedy]

Right. And if the file is large enough that even parsing all the
records to check the fields takes hours, well, that's where I'm at.

So what is the problem? Let it run overnight. If you want more speed,
try numpy or Cython or C functions (swigged or ctyped) or...

To guarantee no crashes, make your top level something like

for line in open('humongous.dat', 'r'):
try:
<parse line>
<check fields and update summary>
except Exception as e:
print(line, e)

(How trivial in Python!)

If your total system (including electricity supply and busy fingers of
other people) is unstable, or you think you might want to interrupt it,
keep track of lines or bytes read and write a report every gigabyte or
so with enough info to restart from the last checkpoint -- just like
reloading your last save in a video game when your avatar dies.

Terry Jan Reedy

 

[Rhodri James]

The whole point is that would be possible if Python had data structure
definitions ("types") that were possible to copy and paste from some
single location, instead of building up structures dynamically, adding
fields on the fly in ways that have become obscure over the evolution
of the code.

mutter mutter __slots__ mutter mutter

It strikes me that what you actually want to write is the script that
scans your C data structure and spits out a Python class or module that
implements it with whatever degree of checking you want. That should
make writing your actual data munger safer and faster, if I'm
understanding you correctly.

 

[Rhodri James]

I would call a hammer evil if it were built in a way that made it
unnecessarily likely to hit your thumb.

But is that because the hammer is misdesigned, because you're holding
the nail all wrong, or because you're actually holding a screw?

Um. It wasn't designed for portability, or if it was, it wasn't
designed for portability very well. Then again, what makes for good
portability at the level C works best at -- macroassembly -- is not
an easy question to answer given how variable your environment can
be.

I don't think so: http://www.adaic.org/whyada/ada-vs-c/cada_art.html

I agree. I suspect that its more the case that once most
programmers have been taught how to use the hammer that is C,
all problems look like nails.

 

[Paul Rubin]

So what is the problem? Let it run overnight.

The idea of buying faster and faster computers is to not have to wait
overnight. The sooner you can get the results (preferably within
minutes not hours), the sooner they can be used in the next task,
which depends on them. Language implementation affects program speed
just as hardware does, thus the grumbling about Python being slow.

If you want more
speed, try numpy or Cython or C functions (swigged or ctyped) or...

I should try out Cython. Using plain C would slow down development an
awful lot. Either way would require putting most of the logic in C to
get the best speedup, since it would require building
Python-unfriendly data structures (the natural Pythonic structures
use a huge amount of dict lookups and storage allocation/release).

for line in open('humongous.dat', 'r'):
try:
<parse line>
<check fields and update summary>
except Exception as e:
print(line, e)

(How trivial in Python!)

It's a little more complicated than that since multiple threads are
involved, and "except Exception" is considered an antipattern, but
yeah, I'm already doing stuff like this.

 

[Paul Rubin]

It strikes me that what you actually want to write is the script that
scans your C data structure

There is not a C data structure at the moment. It's all Python.

 

[Michele Simionato]

I'd say there was a time Lisp worked the right way, and a time C
worked the right way, and maybe a time Python worked the right way,
and for a while, Algol 60 was perfection embodied.  But times have
changed more than those languages have, so they no longer work the
right way.

Or perhaps it is you who changed? I can only speak for myself, but
when I first met Python (coming from Basic, Pascal, Fortran,
Mathematica, Maple) it was the best language I could imagine.
Now Python is still the best language I can find, but it is no more
the best I can imagine, because I know much more about programming
than before. But I would say that's normal and even healthy.
The best language is the one yet to be invented!

Have you looked at Tim Sweeney's talk that I mentioned in another post?

http://www.st.cs.uni-saarland.de/edu/seminare/2005/advanced-fp/docs/s...

I will have a look now.

 

[Paul Rubin]

Or perhaps it is you who changed? I can only speak for myself, but
when I first met Python (coming from Basic, Pascal, Fortran,
Mathematica, Maple) it was the best language I could imagine.

I would say Algol 60 and Lisp were great languages the same way the
Burroughs B5500 or Dec PDP-10 were great computers. Today, though, We
would only describe their greatness in historical terms, since
technology has advanced so much and we know better ways to do things
now. All the languages were are using now, it seems to me, are pretty
far behind what PL researchers are working on. But the stuff the
researchers are doing isn't that immediately useful to us, since
they're concentrating on academic results and aren't really in the
practicality business. The gap between research and practice narrows
and widens over the general ebb and flow of advancing technology, and
I think it is wider than usual right now. An exception to that might
be Haskell, which makes Python feel almost juvenile, but (despite the
amazing work that its aficionados are doing) it lacks somewhat in
practicality, in my opinion.

Now Python is still the best language I can find, but it is no more
the best I can imagine, because I know much more about programming
than before. But I would say that's normal and even healthy.
The best language is the one yet to be invented!

Right. I think we are heading into a period of consolidation, where
the stuff currently in the ivory towers makes it into the hands of day
to day practictioners.

 

[Russ P.]

Once again, the important point is that there's a *clear* distinction
between interface and implementation, and that you *shouldn't* mess with
implementation.

If you "*shouldn't* mess with the implementation", then what is wrong
with enforcing that "shouldn't" in the language itself? Why leave to
coding standards and company policy what can be encoded right into the
language? Why leave to humans (who are known to err) what can be
automated relatively easily? Isn't that what computers are for?

But what, some people think programmers are stupid, and
so they hire stupid programmers, so they need b&d languages to protect
stupid programmers from themselves - which just doesn't work, since
*nothing* is idiot-proof. Heck, how many Java "OO" programs with dumb
getter/setter pairs for _each and any_ attribute ?

Hey, I share your distaste for Java and C++. All those "setters" and
"getters" are a kludge. I think Python "properties" are a major step
forward here. Just for fun, I checked to see if Scala has properties.
Guess what? Not only does it have them, but they are generated
automatically for all member data. That's even better than Python
properties!

No. It is appropriate for dummy managers hiring dummy programmers. The
project's size and domain have nothing to do with it.

Let me try to be very clear here. We are dealing with two separate but
related issues. The first is whether data hiding should be added to
Python. The second is whether data hiding provides any net benefit in
*any* language.

As for whether data hiding should be added to Python, I am not arguing
one way or the other. I am merely saying that it is worth considering.
Whether it can be added without screwing up the language, I don't
know. If not, then so be it. That just limits the range of domains
where Python is suitable.

As for whether data hiding provides a net benefit in any language, it
certainly does for large programs and for safety-critical programs.
For large, safety-critical systems, it's a no-brainer. I think data
hiding can also provide net benefits for medium-size and even smaller
programs, but lets just consider large programs, so I can shoot down
your argument that data hiding provides no benefit.

I like to use the example of the flight software for a large
commercial transport aircraft, but many other examples could be given.
How about medical systems that control radiation therapy or
chemotherapy? How about financial systems that could take away your
retirement account in 1.5 milliseconds. Or how about the control
software for the strategic nuclear arsenals of the US or Russia? When
you consider the sea, air, and land-based components, I'm sure that's
one hell of a lot of code!

So let's take the airplane example. Boeing and its contractors
probably has hundreds of programmers working on millions of lines of
code. If they believed you, they would abandon enforced data hiding,
and programmers would have much more discretion to screw around with
code that they don't work on directly. An FMS programmer could perhaps
decide to change the parameters of the engine controls, for example.

To prevent that sort of thing from happening, the management could
decree that henceforth all "private" variable names will start with an
underscore. Problem solved, eh? Wait a minute. You yourself have
stated several times that those stupid library developers can never
predict what the client will actually need. So some FMS guy named
Bruno decides that he knows better than the engine group what a
certain engine parameter should be. Or he decides that his code needs
to know the value of that parameter, so he just goes ahead and reads
it. No harm done, eh?

But wait a minute. Half the programmers did the same thing, and now we
have a spaghetti mess on our hands. So now management needs to
implement a costly, bureaucratic program to find all such accessing of
private data and correct it -- before a disaster occurs in flight. So
they have meetings and assemble a data-hiding enforcement team that
meets twice each week and constantly looks over every programmers
shoulder.

Wait a minute, wouldn't enforcement of data hiding by the programming
language be preferable to a huge bureaucratic program? I sure think
so. And thankfully, so does Boeing and anyone else who has a clue
about large-scale software production.

Now, let's talk about a data access violations in the nuclear missile
launch software. Oh, yes, I'm sure the leading underscore convention
is more than sufficient! Wake up, dude, and quit spouting bullsh*t!

You only need locks when you don't trust your neighbours.

Yeah, if you live in Nome, Alaska.

Also, have you ever heard the expression "locks keep honest people
honest"?

I spent *way* too much time on that post. I really need to quit
spending my time refuting the baloney that passes for wisdom here.

 

[Terry Reedy]

I did. He gives a really nice use case for Python's ability to
dynamically modify classes imported from a library. (Were not you
arguing against that? Or was is someone else?)

 

[Hendrik van Rooyen]

At the most basic level do you really think a machine
really cares about whether -you- the programmer
has illegally accessed something you shouldn't have ?

Yes it does - this is exactly why some chips have supervisor
and user modes - to keep the monkeys away from the typewriter.

- Hendrik

 

[Paul Rubin]

I did. He gives a really nice use case for Python's ability to
dynamically modify classes imported from a library. (Were not you
arguing against that? Or was is someone else?)

He is talking about extending classes by something like inheritance,
not modifying them dynamically. This is also in the context of an
extremely powerful static type system with existential and dependent
types, and totality proofs for just about everything. That is about a
billion light years away from anything anyone has ever proposed for
Python.

 

[Terry Reedy]

He is talking about extending classes by something like inheritance,
not modifying them dynamically. This is also in the context of an
extremely powerful static type system with existential and dependent
types, and totality proofs for just about everything. That is about a
billion light years away from anything anyone has ever proposed for
Python.

If the library framework were written in Python, one would have a choice
between creating a parallel class hierachy (through inheritance) that
dupicates *every* class in the original hierachy and dynamically
patching just those classes that need to be changed. He only discussed
the former option because that was the only one available. If, for
instance, one only needed to change the base class, patching just that
would be much easier. Hence my comment.

tjr

 

[Paul Rubin]

If the library framework were written in Python, one would have a
choice between creating a parallel class hierachy (through
inheritance) that dupicates *every* class in the original hierachy and
dynamically patching just those classes that need to be changed.

I think I understand what you're saying, but it could be handled with
generics.

 

[Steve Holden]

Russ P. wrote:
[...]

I spent *way* too much time on that post. I really need to quit
spending my time refuting the baloney that passes for wisdom here.

He who cannot ignore baloney is doomed to refute it.

regards
Steve

 

[Luis Zarrabeitia]

If you "*shouldn't* mess with the implementation", then what is wrong
with enforcing that "shouldn't" in the language itself?

Because, as a library user, it should be my power to chose when and how I
_should_ mess with the implementation, not the compiler, and definitely not you.

So let's take the airplane example. Boeing and its contractors
probably has hundreds of programmers working on millions of lines of
code. If they believed you, they would abandon enforced data hiding,
and programmers would have much more discretion to screw around with
code that they don't work on directly. An FMS programmer could perhaps
decide to change the parameters of the engine controls, for example.

I really hope that, at Boeing, they do a lot of unit tests and code reviews
before the code is commited. Messing with the internals wouldn't be the only bad
thing that could happen, you know... And I'd say that "x._private" where x is
not 'self', would be very easy to catch on those code reviews.

And, as someone else pointed out, even with 'strong' enforcement of data-hiding,
you can usually access the privates anyway - it's just a bit harder to do, and a
lot harder to discover.

You pointed out previously that Python wasn't up to the task of running
untrusted code on my own application, and I agreed. _That_ is what you seem
need, and enforced data hiding does very little towards it.