Does Python really follow its philosophy of "Readability counts"?

[Russ P.]

Properties by themselves are not the problem, quite on the contrary - as
you say, they actually help wrt/ encapsulation. What breaks
encapsulation is *automatic generation* for properties for *each and
any* implementation attribute. Might as well just makes them all public
attribute then.

Let me correct my statement about the automatic generation of
properties in Scala: it is only for public attributes, not all
attributes.

Getting back to the bigger point, I will gladly agree with you that
data hiding is not a magic bullet that will eliminate all bugs. The
idea that I or anyone else said that, however, is a red herring. Data
hiding is just one safeguard in a portfolio of safeguards that can
*help* to prevent certain kinds of bugs as well as deliberate acts of
sabotage or fraud. When you have a tough problem to solve, you need
all the help you can get.

You keep saying that if you hire competent, trustworthy developers,
you don't need data hiding. Well, maybe, but when you have a team of
dozens or hundreds of developers, your chances of avoiding any bad
ones is zero for all practical purposes.

And even if all your developers were excellent, data hiding would
still be a convenient mechanism to simplify their jobs so they can
focus on higher level problems -- and not have to rely on an ugly
naming convention.

Now, if developers become careless because they think data hiding will
save them, then that would be a problem. That much I will concede. But
I doubt that happens much.

 

[Tim Rowe]

I.e. the cast was wrong because of the failure of an unstated
assumption that a certain sensor reading was in a certain range.
Spark may still have allowed the cast only if the assumption was
stated explicitly in the specification.

Unless it's changed since I used it, technically, SPADE doesn't allow
or disallow anything. It produces a predicate (a proof obligation)
that you have to prove is always true (or is it always false? It's
been 18 years since I worked on that stuff, and SPADE and MALPAS
produced their proof obligations with opposite values). So it's still
up to you to show that it won't overflow, it just gives you the
predicate calculus expression that you need to do that.

Since the value appears to come from a sensor, the only way one could
prove that there would be no overflow would be to state it as a part
of the specification of what is read in. If that specification doesn't
match the specification of the actual sensor, that's nothing to do
with the programming language or, for that matter, the program itself.
It's a specification mismatch.

I was actually at the European Space Agency's Toulouse site the week
after the Ariane 5 incident. I've been at jollier funerals. I can't
help thinking that thinking that the team would have benefited from
reading David Parnas's work on the specification of the A-7E avionics.

 

[bearophileHUGS]

r:

Of course it would not run in C or Python but the point here is readability.<

With languages like Genie you can go close:
http://live.gnome.org/Genie

Or better Delight (based on the D language), plus some my "Python
compatibility" libs (plus Pyd, if you want) you can run that code:
http://delight.sourceforge.net/

Bye,
bearophile

 

[bearophileHUGS]

alex23:

Paul, have you looked into Cython at all?

I can also suggest ShedSkin and the D language.

I have often used D for data munging, producing quick & short
programs, when Python isn't fast enough for a certain purpose.

Bye,
bearophile

 

[bearophileHUGS]

Michele Simionato:

I don't like that. Scala was designed with the idea of putting
together the two worlds, by I think the result was to get the
complications of both worlds.

But some other people may like it, and it's a design experiment worth
doing. Every programming paradigm has advantages, so it's normal for
people to try to design language that have the best of different
worlds. Even if Scala is a failure (and I don't think it is), it's
good to keep trying to design a mixed language (Like Python, that
mixes procedural, some OOP and a bit of functional stiles. It has not
pattern matching stile (Mathematica, OcaML), logic-inferential style
(Prolog), constraint style (Oz, Mozart), data flow style, generic
programming (C++, D), etc).

Today lot of people understand that functional languages have
advantages, so they are trying to created hybrids (Scala, F#, D V.2,
etc), and they may succeed only trying. The purpose is of course to
create a language that isn't too much complex, but has those
advantages anyway. I presume Scala is quite less complex than C++
anyway.

Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary. -- William Clinger<

I think it's false, but it requires me lot of space to explain why. In
few words: Perl is more useful than Scheme if you have to solve a lot
of practical computational problems, despite Scheme looks much more
nicer.

Bye,
bearophile

 

[Bruno Desthuilliers]

Russ P. a écrit :

Let me correct my statement about the automatic generation of
properties in Scala: it is only for public attributes, not all
attributes.

I must be missing the point : if it's a public attribute, it doesn't
need a "property" ? I guess we use the same words for different things here.

Getting back to the bigger point, I will gladly agree with you that
data hiding is not a magic bullet that will eliminate all bugs. The
idea that I or anyone else said that, however, is a red herring. Data
hiding is just one safeguard in a portfolio of safeguards that can
*help* to prevent certain kinds of bugs

s/data-hiding/encapsulation/ and I'll wholefully agree.

as well as deliberate acts of
sabotage or fraud.

I definitively wouldn't bet my ass on language-level access restriction
to protect software from fraud or sabotage.

When you have a tough problem to solve, you need
all the help you can get.

As far as I'm concerned, *enforcing* access-restriction is of no help.

You keep saying that if you hire competent, trustworthy developers,
you don't need data hiding.

Yes. And I also think that trust (and even - to a certain extent -
competence) is better built on trust than on distrust. When treated as
an irresponsible morons just barely able to type code, most peoples tend
to become just that : code-monkeys.

Well, maybe, but when you have a team of
dozens or hundreds of developers, your chances of avoiding any bad
ones is zero for all practical purposes.

I don't know how many developpers work for google, but I bet there all
smart enough to not need enforced access restriction. Very few people
have a burning desire to shoot themselves in the foot, you know.

Take some not-that-trivial projects like Zope/Plone. There are quite a
few lines of code involved, and quite a lot of programmers worked on it.
Some of them being very average joe programmer FWIW. Guess what ? From
experience, it JustWork(tm). Granted, this is not a critical system -
but that's not the point here. The point is that _from experience_, most
programmers are wise enough to avoid doing stupid things.

And even if all your developers were excellent, data hiding would
still be a convenient mechanism to simplify their jobs so they can
focus on higher level problems

Sorry, but this makes no sense. How could the lack of
*language-enforced* access restriction makes anything more complicated ???

-- and not have to rely on an ugly
naming convention.

And here we are, finally : *you* don't like this convention (and we've
aready been thru that discussion IIRC).

Guess what ? As far as I'm concerned, I just *love* this convention.
Because I *never* have to ask myself if some attribute is interface or
implementation.

Now, if developers become careless because they think data hiding will
save them, then that would be a problem.

If you believe data-hiding will protect your code from fraud, sabotage
or any other malevolence, then you already have this problem IMHO.

 

[Rhamphoryncus]

And even if all your developers were excellent, data hiding would
still be a convenient mechanism to simplify their jobs so they can
focus on higher level problems -- and not have to rely on an ugly
naming convention.

That's just it — the cost of maintaining friend lists and similar
mechanisms exceeds the benefit enforced privates.

Doing it for performance or for a secure sandbox of course changes
this, but they're separate issues.

Java hides low-level details because it thinks the programmer is
incompetent. Python hides them because it thinks the programmer has
better things to do. An underscore is all a competent programmer
needs.

 

[Russ P.]

I must be missing the point : if it's a public attribute, it doesn't
need a "property" ? I guess we use the same words for different things here.

Yes, you are missing more than one point.

Scala automatically converts public data members into properties,
apparently to save the programmer the trouble of doing it manually. If
you are interested, I'm sure you can find publicly available
information on it.

I definitively wouldn't bet my ass on language-level access restriction
to protect software from fraud or sabotage.

You're missing the point here too. I'll try one more time to explain
it.

Access restriction enforced by the language is just one safeguard
among several that are usually available. I think the lock analogy is
very appropriate here. Think about door locks. Would you "bet your
ass" on them to keep you secure? Of course not. You might also have a
firearm or a dog. But even if you don't have those, you surely have a
police department, and surely they carry guns.

Because you cannot "bet your ass" on door locks for your security,
does that mean you just abandon them altogether? Maybe so, if you live
in a very remote place, but most people still seem to think they need
locks on their doors. Well, think of access restrictions enforced by
the language as analogous to door locks.

Let's take the analogy a bit further. Suppose you work in a secure
building with strict access control, and everyone in the building is
"trusted" (whatever that means). This situation is analogous to a
group of "trusted" developers working on a project together. Would you
feel comfortable having no locks on the doors?

Maybe you would, but I wouldn't. I wouldn't want to just let my co-
workers, trusted or not, have access to all my personal stuff. And
what if I need to store something of value from time to time? They may
be honest, but why even tempt them? What if one of them decides he
needs to borrow my favorite Python book while I'm gone, then forgets
to return it?

But what if you don't want your door locked? Well, you have the option
of not using your lock. You can get your way whether the doors have
locks or not. But if I want mine locked, and no locks are available, I
can't get what I want. When you say that a language should not even
have access control available, you are essentially saying that you
should get your way -- and to hell with anyone who disagrees with my
way.

Yes. And I also think that trust (and even - to a certain extent -
competence) is better built on trust than on distrust. When treated as
an irresponsible morons just barely able to type code, most peoples tend
to become just that : code-monkeys.

That may well apply in some environments, but it certainly does not
apply in all environments. It certainly does not apply in a defense
contracting environment, or a financial software development
environment, to give just two of many possible examples. Maybe you
don't personally care about such environments, but who are you to
decide what environments Python should or should not be suitable for?

Take some not-that-trivial projects like Zope/Plone. There are quite a
few lines of code involved, and quite a lot of programmers worked on it.
  Some of them being very average joe programmer FWIW. Guess what ? From
experience, it JustWork(tm). Granted, this is not a critical system -
but that's not the point here. The point is that _from experience_, most
programmers are wise enough to avoid doing stupid things.

And would data hiding have impeded those projects? Or could it perhaps
have helped? I don't know, but I don't think you can simply assume it
would have been a net minus. I know for fact that the strict access
restrictions in Ada were indispensable in the integration of the
flight software in the Boeing 777, for example. Without them, they'd
probably *still* be trying to get it right!

Sorry, but this makes no sense. How could the lack of
*language-enforced* access restriction  makes anything more complicated ???

Because it forces humans to check for access restrictions when the
language could do it.

And here we are, finally : *you* don't like this convention (and we've
aready been thru that discussion IIRC).

Guess what ? As far as I'm concerned, I just *love* this convention.
Because I *never* have to ask myself if some attribute is interface or
implementation.

If you like it that much, you would be perfectly free to continue
using that convention even if access control was enforced.

If you believe data-hiding will protect your code from fraud, sabotage
or any other malevolence, then you already have this problem IMHO.

I addressed this red herring above.

 

[Luis Zarrabeitia]

here.

Yes, you are missing more than one point.

Scala automatically converts public data members into properties,
apparently to save the programmer the trouble of doing it manually. If
you are interested, I'm sure you can find publicly available
information on it.

Russ, I think _you_ are missing the point.
If the attribute is already public, why does it need properties? Why would a
programmer go to the trouble of adding them manually, just to get one level of
indirection for an already public attribute?

You're missing the point here too. I'll try one more time to explain
it.

And I think you are conflating the idea of "private" as in "secret information
that should not be known by the _public_" and "private" as in "static safeguards
enforced by the compiler to prevent accidents" [and you are missing the third,
"compiler feature to prevent namespace pollution without having to use extremely
unlikely variable names", i.e, self.__x in python].

No wonder you can't get Bruno's point. For the second, static checks to prevent
accidents, you have pylint. For the first, not only you are using the wrong
tool, but you are barking at python for not having it. Assuming that pylint is
perfect (big assumption, but it is up to you to prove where it fails), what
would be the difference between only accepting/running "pylint-authorized code"
and the enforced hiding you desire?

This thread is starting to remind me of a professor of mine, who once claimed
that python didn't have private attributes because it "is opensource and anyone
can see the source code anyway", and the obvious confusion of his students ("why
should I make my entrypoint '_public_ static void main', if it is _my_
sourcecode and I don't want to share it?").

What you want is not enforced data hiding. You want something actually designed
to try to prevent abuses from hostile programmers - go use .Net or Java, who
attempt to do that (I don't know with what level of success, but they at least
provide you the 'locks' and 'police' that you need). Or better yet, write a
proposal about how to implement code trust in Python. I'll support you on that
one, and I think many others will. But if you keep presenting data hiding as a
solution to that problem... I doubt that you will be heard.

 

[Russ P.]

Russ, I think _you_ are missing the point.
If the attribute is already public, why does it need properties? Why would a
programmer go to the trouble of adding them manually, just to get one level of
indirection for an already public attribute?

You don't understand the purpose of properties -- and you tell me that
*I* am the one missing the point?

I give up. I really don't have time for this.

 

[James Mills]

You don't understand the purpose of properties -- and you tell me that
*I* am the one missing the point?

Russ, he is correct. Your opinions are not founded on solid knowledge.

Python programmers tend to not have a need for
properties. Quite honestly they are a waste of time.
They come from traditional OO approaches to software design
(and mostly from the Java world).

I give up. I really don't have time for this.

You've said that before

cheers
James

 

[Steven D'Aprano]

You don't understand the purpose of properties -- and you tell me that
*I* am the one missing the point?

Well, I *thought* I did, and (unlike Bruno) I'm not hostile to the idea
Russ is proposing. But I must admit it's not clear to me why Russ thinks
it is a good idea to automatically turn this:

class Parrot(object):
def __init__(self):
self.x = 1


into this:

class Parrot(object):
def __init__(self):
self._x = 1
def getx(self):
return self._x
def setx(self, value):
self._x = value
x = property(getx, setx)


Because frankly, that's how I read Russ' explanation for what Scala is
doing. Have I missed something?

 

[Russ P.]

Well, I *thought* I did, and (unlike Bruno) I'm not hostile to the idea
Russ is proposing. But I must admit it's not clear to me why Russ thinks
it is a good idea to automatically turn this:

class Parrot(object):
    def __init__(self):
        self.x = 1

into this:

class Parrot(object):
    def __init__(self):
        self._x = 1
    def getx(self):
        return self._x
    def setx(self, value):
        self._x = value
    x = property(getx, setx)

Because frankly, that's how I read Russ' explanation for what Scala is
doing. Have I missed something?

 

[Russ P.]

Well, I *thought* I did, and (unlike Bruno) I'm not hostile to the idea
Russ is proposing. But I must admit it's not clear to me why Russ thinks
it is a good idea to automatically turn this:

class Parrot(object):
    def __init__(self):
        self.x = 1

into this:

class Parrot(object):
    def __init__(self):
        self._x = 1
    def getx(self):
        return self._x
    def setx(self, value):
        self._x = value
    x = property(getx, setx)

Because frankly, that's how I read Russ' explanation for what Scala is
doing. Have I missed something?

Whoops, I accidentally hit send on the last post.

One of the main benefits of properties is that they allow you to more
safely put attributes in the public interface. If you later decide
that the attribute shouldn't have been in the public interface, you
can convert it to a property and make it do whatever you want it to
do. That won't always save you, but sometimes it can.

As a trivial example, suppose you have a circle class with public
attributes of radius and area. You later realize that it is unwise to
let the user change one without changing the other, leaving them
inconsistent. With properties, you can convert them each to a property
and write functions to force consistency.

The benefit of automatically converting public data into properties, I
assume, is to relieve the programmer of doing it manually. The
programmer will obviously need to add any functionality when needed,
but he does not need to create the function header.

I don't know the implications for efficiency in Scala. Perhaps there
is a slight cost for one level of indirection. But please recall that
public data is supposed to be rarely if ever used anyway. Unlike some
folks here, the Scala folks understand the value of encapsulation and
data hiding.

 

[Russ P.]

Python programmers tend to not have a need for
properties. Quite honestly they are a waste of time.
They come from traditional OO approaches to software design
(and mostly from the Java world).

With statements like that, it's no wonder you don't understand the
value of encapsulation.

With encapsulation (whether enforced or voluntary), no one has the
"need" for properties. You can always just provide functions to do the
same job. What properties do is to essentially make the function call
simpler for the client. The client just sets the radius of the circle,
and, lo and behold, the area gets changed automatically and
consistently.

If you didn't plan ahead and encapsulate the radius from the start,
properties allow you to save yourself and encapsulate it later without
breaking the client's code.

 

[Guest]

coz he gets paid per line of code....

it's a mindless way of getting paid more for the same functionality...

 

[Russ P.]

coz he gets paid per line of code....

it's a mindless way of getting paid more for the same functionality...

Why did you email your post to me? Did you really think I wanted to
see it in my inbox? I assure you I did not.

And who "gets paid per line of code"? The whole point is that Scala
saves the programmer the trouble of writing that boilerplate code that
Java programmers write themselves, genius.

 

[Luis Zarrabeitia]

One of the main benefits of properties is that they allow you to more
safely put attributes in the public interface. If you later decide
that the attribute shouldn't have been in the public interface, you
can convert it to a property and make it do whatever you want it to
do. That won't always save you, but sometimes it can.

That's true in Java, and python pre-'descriptor protocol'.

It boggles me when I see python code with properties that only set and get the
attribute, or even worse, getters and setters for that purpose. In my university
they teach the students to write properties for the attributes in C# ("never
make a public attribute, always write a public property that just gets and sets
it"). I never understood that practice either, given that the syntax for
attribute access and property access in C# is exactly the same. (Could it be
that even if the syntax is the same, the compiled code differs? Don't know
enough about .NET to answer that).

I think I'm getting offtopic now. I better leave

[snip the rest of the email, as I agree with it]

 

[Russ P.]

The benefit of automatically converting public data into properties, I
assume, is to relieve the programmer of doing it manually. The
programmer will obviously need to add any functionality when needed,
but he does not need to create the function header.

Let me just correct this. As I said before, I haven't yet even used
Scala. The automatic generation of properties for public data members
is done automatically by the compiler and is apparently transparent to
the programmer. The programmer can then override the default
properties if he needs to add functionality. As far as the programmer
is concerned, this is really no different than Python properties. The
only difference is the way the two languages implement properties, not
the way they are used.

 

[Luis Zarrabeitia]

Python programmers tend to not have a need for
properties. Quite honestly they are a waste of time.
They come from traditional OO approaches to software design
(and mostly from the Java world).

With statements like that, it's no wonder you don't understand the
value of encapsulation.
[snip]

If you didn't plan ahead and encapsulate the radius from the start,
properties allow you to save yourself and encapsulate it later without
breaking the client's code.

Python programmers don't _need_ to plan ahead and encapsulate the radius from
the start. That's the whole point. No clairvoyance needed. I kind of like that.