Does Python really follow its philosophy of "Readability counts"?

[Mark Wooding]

Paul Rubin a écrit :

I'd say that Python's FP characteristics are an important part of its
expressiveness.

Indeed - but they do not make Python a functional language[1]. Python is
based on objects, not on functions,

I'd have a good go at defining a functional language as one which treats
functions as first-class objects -- i.e., can be passed as arguments,
returned as results, stored in data structures, etc. In that regard,
Python is a proper paid-up functional programming language. It's not a
/pure/ language -- i.e., there are side effects -- but ML has those too
and it's widely considered functional; and it's not lazily evaluated --
but again neither is ML.

-- [mdw]

 

[Luis Zarrabeitia]

And now I have accidentally broken the spam() method, due to a name clash.

True, that's bad. I wish that were 'fixed'.

Besides, double-underscore names are a PITA to work with:

Isn't that example the point of having self.__private variables? To make sure
that the Parrot.__private attributes don't clash with the RedParrot.__private
attributes with the same name?

[parrot example follows just for context, but my reply ends here]

 

[James Mills]

What I've seen engineers do when they need extra safety is to put in place
independently developed and operated redundant systems, at least three, and
the system will do whatever two of the independent systems agree on. So I
guess if I really wanted to be safe with a system I'd have it made by at
least three different teams (yes, also the requirements should be written by
different teams) in different languages under different OS's and different
independent machines. And THEN I would have some measure of safety (and this
would certainly be more cost effective than the formal specification route).

This is quite honestly the best thing I've seen
written on this very topic of safety and mission
critical systems! Thank you!

My entire point(s) throughout this very long and tediously
boring thread have been that one language over another
doesn't necessarily bear much weight on the safety
requirements of a system.

One thing that those that have argued for Data Protection,
Static Typing, Strong Typing and Stricter Encapsulation (all of
which Python -IS NOT- and -WILL NOT- be); is that Safety
Requirements are normally intangible, unquantifable
measures. How do you rate the safety of your system ? By
the number of deaths and injuries it not cause ? By the fact
that your Shuttle doesn't blow up ? Your plane landing correctly ?

Ricardo's point is very well put and Safety Critical systems
that specify requirements, tangible and quantifiable requirements
are what makes a system safe and gives assurance - not the language
or the platform os the os or the environment. For example:

FR1: When the landing gears button is pressed, the landing gears will
be lowered and monitored and confirms by the Confirmation System and
Verified by the Verification System.

Have a pleasant day,

cheers
James

 

[Paul Rubin]

Ricardo's point is very well put and Safety Critical systems
that specify requirements, tangible and quantifiable requirements
are what makes a system safe and gives assurance - not the language
or the platform os the os or the environment.

But it is characteristics of the language, platform os, and
environment, that make it easier or more difficult to provide high
assurance that the specifications are actually met. See this article
for the meaning of "high assurance":

http://www.dwheeler.com/essays/high-assurance-floss.html

Basically, high assurance in the view of the folks who issue
specifications for it and fund it, requires a lot of static
verification.

 

[Paul Rubin]

Having once been a more type-A, I labored for a couple of years trying
to build a restricted language that provably terminated for work on an
object-oriented database research. I finally gave it up as a bad idea,
because, in practice, we don't care if a loop will terminate or not in
database work; a transaction that takes a year to commit is equivalent
to an infinite loop for all applications that I have interacted with
(and yes, I have worked allowing four day transactions to commit).

I think the purpose of the termination proofs is to ensure that
nonterminating functions don't introduce inconsistency into the
verification logic, rather than out of concern that some function
might actually loop. Consider the function

def f(x):
return 1 + f(x)

aside from the "minor" issue of infinite recursion, this is a nice,
well-typed function that doesn't have any side effects, doesn't raise
exceptions, etc. It is referentially transparent, so one can
substitute any call to it with the value from another call to the same
arg, i.e. we can substitute x=3 and deduce the equation

f(3) = 1 + f(3)

Subtracting f(3) from each side, we get 0 = 1, an erroneous "theorem"
from which the verification system can infer all kinds of other bogus
results. By requiring proof that any function purporting to return a
value really DOES return a value (rather than looping, raising an
exception, etc.), we get rid of this problem. The actual number of
computing steps before termination isn't an issue for this purpose, as
long as it is finite.

 

[Steven D'Aprano]

Well, your claim /was/ just wrong. But if you want to play dumb: the
interface is what's documented as being the interface.

But you miss my point.

We're told Python doesn't have private attributes. We're told that we're
allowed to "mess with the internals", we're *encouraged* to do so: Python
gives you the freedom to do so, and any suggestion that freedom might be
reduced even a tiny bit is fought passionately. When people ask how to
implement private attributes, they're often told not to bother even using
single-underscore names. When it is suggested that Python should become
stricter, with enforced data hiding, the objections come thick and fast:
people vehemently say that they like Python just the way it is, that they
want the ability to mess with the internals.

You even argued that you disliked data structures implemented in C and
preferred those written in Python because you have more ability to mess
with the private attributes. In context, I had just mentioned that lists'
internals were inaccessible from Python code. I neglected to give an
example at the time, but a good example is the current length of the
list. Consider the experience of Microsoft and Apple. No matter how often
they tell people not to mess with the internals, people do it anyway, and
always believe that their reason is a good reason.

And Python culture encourages that behaviour (albeit the consequences are
milder: no buffer overflows or core dumps).

Add to that the culture of Open Source that encourages reading the source
code. You don't need to buy a book called "Undocumented Tips and Tricks
for Python" to discover the internals. You just need to read the source
code.

And then you have at least two places in the standard library where
_attributes are *explicitly* public:

http://bugs.python.org/issue3152

Given this permissive culture, any responsible library writer must assume
that if he changes his so-called "private" attributes, he will break
other people's code. In principle it could break just as much code as if
he didn't even bother flagging them with a leading underscore, which is
probably why many people don't even bother with _names.

In other words, if you make it easy for people to mess with your
internals, if you have a culture that allows and even encourages them to
mess with your internals, then you don't have internals. Everything is de
facto public.

You can tell that your claim is simply wrong by pushing it the other
way. If everything you have free access to is interface then all
behaviour observable by messing with the things you have access to is
fair game: you can rely on cmp returning one of {-1, 0, 1} on integer
arguments, for example.

But no: the Library Reference says only that it returns a negative, zero
or positive integer, and /that/ defines the interface. Everything else
is idiosyncrasy of the implementation, allowed to change at whim.

And yet people still assume that cmp returns -1, 0 or 1. Even Guido
himself makes that mistake occasionally. Quoting from PEP 285:

"...you might be tempted to believe that cmp() also returned a
truth value, whereas in reality it can return three different values
(-1, 0, 1)."

http://www.python.org/dev/peps/pep-0285/

No, cmp() can return an infinite number of values. It just never does, at
least not yet, but it might. But when Guido himself says that cmp() can
return three values, can you blame people for acting as if cmp() can
return three values?

Here's a thought experiment for you. You've suggested that the values
returned by cmp() are allowed to change "at whim". Okay, let's do it:
make a patch that changes cmp() to return -17, 0 or 53, and promise to
support it for at least three years. Try to get it accepted on python-
dev. What do you expect they will say?

My money is on them saying "No, this will pointlessly break code for no
good reason. Rejected."

 

[Terry Reedy]

Here's a thought experiment for you. You've suggested that the values
returned by cmp() are allowed to change "at whim". Okay, let's do it:
make a patch that changes cmp() to return -17, 0 or 53, and promise to
support it for at least three years. Try to get it accepted on python-
dev. What do you expect they will say?

My money is on them saying "No, this will pointlessly break code for no
good reason. Rejected."

I have occasionally thought that things documented to possibly change
*should* be changed just to expose the *bug* of depending on them not
changing. Or maybe, the doc should be changed.

 

[Russ P.]

But you miss my point.

We're told Python doesn't have private attributes. We're told that we're
allowed to "mess with the internals", we're *encouraged* to do so: Python
gives you the freedom to do so, and any suggestion that freedom might be
reduced even a tiny bit is fought passionately. When people ask how to
implement private attributes, they're often told not to bother even using
single-underscore names. When it is suggested that Python should become
stricter, with enforced data hiding, the objections come thick and fast:
people vehemently say that they like Python just the way it is, that they
want the ability to mess with the internals.

You even argued that you disliked data structures implemented in C and
preferred those written in Python because you have more ability to mess
with the private attributes. In context, I had just mentioned that lists'
internals were inaccessible from Python code. I neglected to give an
example at the time, but a good example is the current length of the
list. Consider the experience of Microsoft and Apple. No matter how often
they tell people not to mess with the internals, people do it anyway, and
always believe that their reason is a good reason.

And Python culture encourages that behaviour (albeit the consequences are
milder: no buffer overflows or core dumps).

Add to that the culture of Open Source that encourages reading the source
code. You don't need to buy a book called "Undocumented Tips and Tricks
for Python" to discover the internals. You just need to read the source
code.

And then you have at least two places in the standard library where
_attributes are *explicitly* public:

http://bugs.python.org/issue3152

Given this permissive culture, any responsible library writer must assume
that if he changes his so-called "private" attributes, he will break
other people's code. In principle it could break just as much code as if
he didn't even bother flagging them with a leading underscore, which is
probably why many people don't even bother with _names.

In other words, if you make it easy for people to mess with your
internals, if you have a culture that allows and even encourages them to
mess with your internals, then you don't have internals. Everything is de
facto public.



And yet people still assume that cmp returns -1, 0 or 1. Even Guido
himself makes that mistake occasionally. Quoting from PEP 285:

"...you might be tempted to believe that cmp() also returned a
truth value, whereas in reality it can return three different values
(-1, 0, 1)."

http://www.python.org/dev/peps/pep-0285/

No, cmp() can return an infinite number of values. It just never does, at
least not yet, but it might. But when Guido himself says that cmp() can
return three values, can you blame people for acting as if cmp() can
return three values?

Here's a thought experiment for you. You've suggested that the values
returned by cmp() are allowed to change "at whim". Okay, let's do it:
make a patch that changes cmp() to return -17, 0 or 53, and promise to
support it for at least three years. Try to get it accepted on python-
dev. What do you expect they will say?

My money is on them saying "No, this will pointlessly break code for no
good reason. Rejected."

Needless to say, I agree with your post.

My understanding is that the vast majority of Python software is
provided as open source. Hence, I am a bit confused by all the talk
about the need for freedom and openness in Python. If data hiding were
enforced, and you need access to something marked as private, you can
just change it to public in the source code. What's the problem?

Note: that change would be much easier to make if a "private" (or
perhaps "priv") keyword were used instead of the leading-underscore
rule. The former would require a change in only one place (per
attribute), whereas the latter would require a change of every
occurrence of the attribute.

So "enforced" access restrictions are trivial to work around with open
source. But doesn't that mean that access restriction is pointless
because it cannot be enforced anyway? Of course not. In a team
project, the access restrictions will be enforced on the checked-in
code. You can play around with the internals all you want in your own
little world, but as when you are working with a team, you need to
adhere to the interfaces they define (if any).

 

[Rhamphoryncus]

I have occasionally thought that things documented to possibly change
*should* be changed just to expose the *bug* of depending on them not
changing.  Or maybe, the doc should be changed.

Aye. There are places where we're deliberately open, but this isn't
one of them. I think this is just a C-ism and the docs should be more
specific.

Or finish ripping it out in 3.x

 

[Russ P.]

Russ P. a écrit :
(snip)


Calling people names is certainly not the best way to defend your
opinions here. Adios, Mr. P.

You are absolutely right, Mr. D. I regret that post, and I have since
apologized to the person I was addressing. This stuff really isn't
important enough to get that worked up about.

 

[James Mills]

My understanding is that the vast majority of Python software is
provided as open source. Hence, I am a bit confused by all the talk
about the need for freedom and openness in Python. If data hiding were
enforced, and you need access to something marked as private, you can
just change it to public in the source code. What's the problem?

Note: that change would be much easier to make if a "private" (or
perhaps "priv") keyword were used instead of the leading-underscore
rule. The former would require a change in only one place (per
attribute), whereas the latter would require a change of every
occurrence of the attribute.

So "enforced" access restrictions are trivial to work around with open
source. But doesn't that mean that access restriction is pointless
because it cannot be enforced anyway? Of course not. In a team
project, the access restrictions will be enforced on the checked-in
code. You can play around with the internals all you want in your own
little world, but as when you are working with a team, you need to
adhere to the interfaces they define (if any).

I'm going to say this very politely.

Python IS NOT Java.

--JamesMills

 

[Bruno Desthuilliers]

Steven D'Aprano a écrit :

But you miss my point.

We're told Python doesn't have private attributes.
Yes.

We're told that we're
allowed to "mess with the internals",

Given that we're willing and able to cope with possible consequences.

we're *encouraged* to do so

Certainly not.

Python
gives you the freedom to do so, and any suggestion that freedom might be
reduced even a tiny bit is fought passionately.

Won't comment on this.

When people ask how to
implement private attributes, they're often told not to bother even using
single-underscore names.

"often" ? Not as far as I can tell. I think you're confusing this with
the advice to not use getters/setters for no good reason, given Python's
support for computed attributes - which is not exactly the same thing.

When it is suggested that Python should become
stricter, with enforced data hiding, the objections come thick and fast:
people vehemently say that they like Python just the way it is, that they
want the ability to mess with the internals.

Indeed. There's no shortage of B&D languages, and well, Python is OSS,
so if you want a B&D Python (now that's an oxymoron), please feel free
to implement it. But by all mean, leave my favorite language alone. Thanks.

You even argued that you disliked data structures implemented in C and
preferred those written in Python because you have more ability to mess
with the private attributes.

In context, I had just mentioned that lists'
internals were inaccessible from Python code. I neglected to give an
example at the time, but a good example is the current length of the
list. Consider the experience of Microsoft and Apple.

Yes, two great examples of freedom champions.

No matter how often
they tell people not to mess with the internals, people do it anyway, and
always believe that their reason is a good reason.

And who are *you* to pronounce any judgement about that ?

And Python culture encourages that behaviour (albeit the consequences are
milder: no buffer overflows or core dumps).

Add to that the culture of Open Source that encourages reading the source
code.

Indeed. A *very* good thing FWIW.

You don't need to buy a book called "Undocumented Tips and Tricks
for Python" to discover the internals. You just need to read the source
code.
Exactly.

And then you have at least two places in the standard library where
_attributes are *explicitly* public:

http://bugs.python.org/issue3152

Given this permissive culture, any responsible

For your personal definition of "responsible".

library writer must assume
that if he changes his so-called "private" attributes, he will break
other people's code.

You still don't get the point. If someone's code breaks because he
messed with my implementation code, then *he* is responsible. The
contract is very clear : "warranty void if unsealed".

In principle it could break just as much code as if
he didn't even bother flagging them with a leading underscore, which is
probably why many people don't even bother with _names.
>
In other words, if you make it easy for people to mess with your
internals, if you have a culture that allows and even encourages them to
mess with your internals, then you don't have internals. Everything is de
facto public.

Now that you've exposed your opinions, let's face reality (I mean,
*facts*): Python developpers very rarely mess with implementation,
usually do so for very good (and documented) reasons, and from what I've
seen usually tend to get in touch with the library author to explain
their case and find a better solution.

Funny enough, it looks that the more you treat programmers as
responsible, normally intelligent adult person, the more they tend to
behave as such. And the other way around, too.

Now, you comprehensively exposed your personnal distaste for Python's
and more generally OSS philosophy. So I can only - as I already did way
before in this thread - wonder *why* are you using Python ?

I mean, is it because your bosses forces you to do so ? If yes, then,
I'm truly sorry for you - I sometimes have to work with languages I
really dislike so I can feel your pain (but OTHO, I never complained on
these languages newsgroups about how wrong they were nor how they should
IMHO be).

Else, well, I just don't get the point. What you are fussing about are
fundamental design choices (and philosophic points) that are well known,
explained, advertized, etc. - and are really not likely to change
anytime soon.

 

[Bruno Desthuilliers]

Russ P. a écrit :

You are absolutely right, Mr. D. I regret that post, and I have since
apologized to the person I was addressing. This stuff really isn't
important enough to get that worked up about.

Ok, so welcome back then.

 

[Russ P.]

Steven D'Aprano a écrit :






Given that we're willing and able to cope with possible consequences.


Certainly not.


Won't comment on this.


"often" ? Not as far as I can tell. I think you're confusing this with
the advice to not use getters/setters for no good reason, given Python's
support for computed attributes - which is not exactly the same thing.


Indeed. There's no shortage of B&D languages, and well, Python is OSS,
so if you want a B&D Python (now that's an oxymoron), please feel free
to implement it. But by all mean, leave my favorite language alone. Thanks.


Yes, two great examples of freedom champions.


And who are *you* to pronounce any judgement about that ?



Indeed. A *very* good thing FWIW.


For your personal definition of "responsible".


You still don't get the point. If someone's code breaks because he
messed with my implementation code, then *he* is responsible. The
contract is very clear : "warranty void if unsealed".





Now that you've exposed your opinions, let's face reality (I mean,
*facts*): Python developpers very rarely mess with implementation,
usually do so for very good (and documented) reasons, and from what I've
seen usually tend to get in touch with the library author to explain
their case and find a better solution.

Funny enough, it looks that the more you treat programmers as
responsible, normally intelligent adult person, the more they tend to
behave as such. And the other way around, too.

Now, you comprehensively exposed your personnal distaste for Python's
and more generally OSS philosophy. So I can only - as I already did way
before in this thread - wonder *why* are you using Python ?

I mean, is it because your bosses forces you to do so ? If yes, then,
I'm truly sorry for you - I sometimes have to work with languages I
really dislike so I can feel your pain (but OTHO, I never complained on
these languages newsgroups about how wrong they were nor how they should
IMHO be).

Else, well, I just don't get the point. What you are fussing about are
fundamental design choices (and philosophic points) that are well known,
explained, advertized, etc. - and are really not likely to change
anytime soon.

I am curious about something. Have you ever needed to access a
"private" attribute (i.e., one named with a leading underscore) in
Python code that you did not have the source code for? For that
matter, have you ever even used a library written in Python without
having access to the source code?

As I said before, if you have the source code you can always change
private attributes to public in a pinch if the language enforces
encapsulation. But if you are working on a team project, you can't
change the code that another member of a team checks in. That is how
enforced data hiding helps teams of developers manage interfaces. The
bigger the team and the bigger the project, the more it helps.

Mr. D'Aprano gave an excellent example of a large banking program.
Without enforced encapsulation, anyone on the development team has
access to the entire program and could potentially sneak in fraudulent
code much more easily than if encapsulation were enforced by the
language.

I am certainly not saying that Python is useless without enforced data
hiding. It is obviously very useful for a wide range of applications
and domains already. I am only saying that it's usefulness could be
enhanced if enforced encapsulation can be added without somehow
comromising the language. I don't know enough about the inner workings
of the Python interpreter to know if that is the case or not (I am an
aeronautical engineer), but the objections I've seen so far on this
thread have not impressed me.

 

[Russ P.]

code. You can play around with the internals all you want in your own
little world, but as when you are working with a team, you need to
adhere to the interfaces they define (if any).

The word "as" should not be there:

.... but when you are working with a team, ...

Sorry, but that was bothering me. I sure wish these posts could be
edited after they are submitted.

 

[Antoon Pardon]

Because, as a library user, it should be my power to chose when and how I
_should_ mess with the implementation, not the compiler, and definitely not you.

Why should it be in your power? By messing with the implementation of a library
you risk the correctness of the code of all participant coders in that project.
I not that sure it should be your power to chose when and how to do that.

 

[Mark Wooding]

But you miss my point.
Evidently.

We're told Python doesn't have private attributes. We're told that
we're allowed to "mess with the internals", we're *encouraged* to do
so: Python gives you the freedom to do so, and any suggestion that
freedom might be reduced even a tiny bit is fought passionately.

Your deduction skills are faulty.

* Python gives us the freedom to do so, and we fight to protect that
freedom -- yes.

* But interpreting that as encouragement is wrong. It's permission,
not encouragement. If you don't want to, that's fine, and we won't
think less of you.

Many things are possible which aren't, as a general rule, good ideas.
Misinterpreting permission as encouragement will lead you to doing many
stupid things.

When people ask how to implement private attributes, they're often
told not to bother even using single-underscore names. When it is
suggested that Python should become stricter, with enforced data
hiding, the objections come thick and fast: people vehemently say that
they like Python just the way it is, that they want the ability to
mess with the internals.


You even argued that you disliked data structures implemented in C and
preferred those written in Python because you have more ability to
mess with the private attributes. In context, I had just mentioned
that lists' internals were inaccessible from Python code. I neglected
to give an example at the time, but a good example is the current
length of the list.

Umm... I'm pretty sure that that's available via the `len' function,
which is tied to list.__len__ (via the magic C-implemented-type mangler,
in C). Though it's read-only -- and this is a shame, 'cos it'd be nice
to be able to adjust the length of a list in ways which are more
convenient than

* deleting or assigning to a trailing slice, or
* augmenting or assigning to a trailing zero-width slice

(Perl has supported assigning to $#ARRAY for a long time. Maybe that's
a good argument against it.)

Consider the experience of Microsoft and Apple. No matter how often
they tell people not to mess with the internals, people do it anyway,
and always believe that their reason is a good reason.

And Microsoft and Apple can either bend over backwards to preserve
compatibility anyway (which effectively rewards the misbehaviour) or
change the internals. I'd prefer that they did the latter.

There are times when messing with internals is the only way to get
things done; but there's a price to be paid for doing that, and the
price is compatibility. The internals will change in later versions,
and your code will break, in subtle and complex ways. It's not always
an easy decision to make -- but I'm glad it's me that gets to decide,
and not some random who neither knows nor cares much about the problem
I'm trying to solve.

It's also important to bear in mind that programs' lifetimes vary. Some
programs are expected to live for years; some programs only for a week
or so; and some for just long enough to be typed and executed once
(e.g., at the interactive prompt). That Python is useful for all these
kinds of program lifetimes is testament to its designers' skill.
Programmers can, and should!, make different tradeoffs depending on the
expected lifetime of the program they're writing.

If I type some hacky thing at ipython, I know it's going to be executed
there and then, and if the implementation changes tomorrow, I just don't
care.

If I'm writing a thing to solve an immediate problem, I won't need it
much past next week, and I'll still probably get away with any awful
hacking -- but there's a chance I might reuse the program in a year or
so, so I ought to put a comment in warning the reader of a possible
bitrot site.

If I'm writing a thing that's meant to last for years, I need to plan
accordingly, and it's probably no appropriate to hack with internals
without a very good reason.

Making these kinds of decisions isn't easy. It requires experience,
subtle knowledge of how the systems one's using work, and occasionally a
little low cunning. And sometimes one screws up.

And Python culture encourages that behaviour (albeit the consequences
are milder: no buffer overflows or core dumps).

Add to that the culture of Open Source that encourages reading the source
code. You don't need to buy a book called "Undocumented Tips and Tricks
for Python" to discover the internals. You just need to read the source
code.

Indeed. Very useful.

Example: for my cryptographic library bindings, I needed to be able to
convert between Python's `long's and my library's `mp's. I have a
choice between doing it very slowly (using shift and masking operators
on the `long') or fast (by including Python/longintrepr.h and digging
about by hand). I chose to do it the fast way. I'm quite prepared to
rewrite my conversion code (64 lines of it) if the internals change;
that I haven't had to yet indicates that my judgement of the stability
of the internal representation was about right. The most important
point is that, /had/ I turned out to be wrong, I'd only have myself to
blame.

And then you have at least two places in the standard library where
_attributes are *explicitly* public:

And documented as being so. It's a convention, with explicitly
documented exceptions. That's a slight shame because it weakens the
convention, but it's not a disaster.

Given this permissive culture, any responsible library writer must
assume that if he changes his so-called "private" attributes, he will
break other people's code.

He will, but he can also assume that the maintainers of that code are
/willing/ to see it break.

This is tough on people who depend on internals by accident. Maybe
they'll learn to be more careful. It's not a pleasant way to learn, but
it's a lesson worth learning anyway.

In principle it could break just as much code as if he didn't even
bother flagging them with a leading underscore, which is probably why
many people don't even bother with _names.

This comes down to documentation. The Python standard library is
largely quite well documented, and is clear about what assumptions one
can make and what one can't. In the absence of such clear
documentation, we're left with conventions -- _things are likely to
change in future so avoid messing on them if you don't want stuff to
break.

In other words, if you make it easy for people to mess with your
internals, if you have a culture that allows and even encourages them
to mess with your internals, then you don't have internals. Everything
is de facto public.

And here you've made a semantic leap that I'm afraid I just can't
follow.

No, cmp() can return an infinite number of values. It just never does,
at least not yet, but it might. But when Guido himself says that cmp()
can return three values, can you blame people for acting as if cmp()
can return three values?

Possibly not! It's worth thinking about codifying the existing practice
and documenting the more constrained behaviour. Note that the C
interface -- the tp_compare slot (Python/C API Reference Manual 10.3) --
/is/ defined to return -1, 0, or +1; so presumably the performance
issues have already been considered.


I think all of this comes down to issues of trust and responsibility.
Python, though its `we're all consenting adults' approach, encourages a
culture where we trust one another to make decisions for ourselves, and
to take responsibility for the consequences of those decisions.

Language features such as attribute (or member) visibility or access
control, on the other hand, imply a culture without trust, and with an
built-in assumption of irresponsibility. That seems rather unpleasant
to me.

Suppose that you write a Python library module and release it. I find
that it's /almost/ the right thing for some program of mine, but it
doesn't quite work properly unless I hack about like so... perfect! I'm
a happy bunny; you've gained a user (maybe that's a good thing, maybe it
isn't!). Now, I've hacked about in your module's internal stuff: how
has this affected you? Answer: not at all; you probably didn't feel a
thing. You release a new version with improved internal structure and
my program breaks: how has this affected you? Answer: still not at all.
How did it affect me? Quite a bit, but then again, I knew what I was
getting into. I gambled and lost; oh, well, that happens sometimes.

I've not dealt with granularity much yet; but that's easy. Basically,
decisions should be made at the level at which the consequences of those
decisions are felt. This isn't directly practical, but there are
mechanisms to manage it: users generally delegate technical decisions to
the development team; maybe there's a hierarchy in the dev team. And
the team members need to be trusted not to make decisions at the wrong
level. If you can't manage that, then Python probably isn't a good
match for the team; replace one or the other.


Finally, I notice that you completely snipped the part of my reply which
dealt with your ConfigParser module. I'm going to assume that this
means that you accepted that part of my response.

-- [mdw]

 

[Steven D'Aprano]

Yes, two great examples of freedom champions.

What does that little dig at Microsoft and Apple have to do with what
we're discussing? I dare say Linux has private internals too. Let's see
if we can find some in 30 seconds of googling.

http://forum.soft32.com/linux2/Bug-406747-linux-kernel-headers-private-
symbols-u16-u32-asm-ftopict67443.html

http://groups.google.com/group/linux.kernel/browse_thread/thread/
a08fd6423204e918

Those fascists! Private data in the kernel!!! Why can't my userland app
mess with the kernel's private data??? It's so unfair!!!

And who are *you* to pronounce any judgement about that ?

Did I pronounce any judgement?


....

[Any] library writer must assume
that if he changes his so-called "private" attributes, he will break
other people's code.

You still don't get the point. If someone's code breaks because he
messed with my implementation code, then *he* is responsible. The
contract is very clear : "warranty void if unsealed".

You've built something full of user serviceable parts. You've insisted,
publicly and loudly, that the ability to modify those parts is absolutely
essential, you've rejected every effort to lock down those internals, and
then when somebody does exactly what you encourage, you suddenly turn on
them and say they're on their own.

That's pretty irresponsible behaviour.

As an aside, Apple and Microsoft have opposite approaches to dealing with
this problem. Apple hangs the developer out to dry, which is why there
are so few developers who make Mac software, while Microsoft goes to (or
at least did) heroic efforts to protect developers from their mistakes,
which is why the Windows internals is a dog's breakfast but there are
more Windows developers named "Steve" then there are Mac developers in
total. (BTW, I'm not one of them.)

http://www.joelonsoftware.com/articles/APIWar.html

Now that you've exposed your opinions, let's face reality (I mean,
*facts*): Python developpers very rarely mess with implementation,
usually do so for very good (and documented) reasons, and from what I've
seen usually tend to get in touch with the library author to explain
their case and find a better solution.

Facts, are they? Care to provide some reputable sources for these facts?

Funny enough, it looks that the more you treat programmers as
responsible, normally intelligent adult person, the more they tend to
behave as such. And the other way around, too.

You're the one who repeatedly declared that programmers who modified
internals were cretins who should be fired. I never said such a thing --
in fact, I defended them.

Now, you comprehensively exposed your personnal distaste for Python's
and more generally OSS philosophy.

Really? I did? Fancy that.

So I can only - as I already did way
before in this thread - wonder *why* are you using Python ?

Ah yes, the old "if you're not 100% for us, you must be 100% against us"
argument. Anyone who thinks that there are costs as well as benefits to
dynamic programming must be a freedom-hater, and probably kicks puppies
too.

I mean, is it because your bosses forces you to do so ?

Yeah, my mean old boss forces me to write Python scripts in my spare
time, he forces me to spend my personal time on comp.lang.python, he
forces me to write things like:

Oh yes, it is liberating to say "I don't care if my method crashes
(raises an exception), it's the caller's fault for messing with my class'
internals, and he can deal with it". I'm not being sarcastic by the way.
It really is liberating not to have to deal with unexpected input or
broken pre-conditions. Just let the caller deal with it!
[end quote]

What a mean old boss!

If yes, then,
I'm truly sorry for you - I sometimes have to work with languages I
really dislike so I can feel your pain (but OTHO, I never complained on
these languages newsgroups about how wrong they were nor how they should
IMHO be).

If people hadn't complained about missing features/misdesigns, would
Python have:

generator expressions
print as a function
booleans
nested scopes
None as a keyword
ABCs
closures

to name just a few?

I am curious though... you've made a few comments that suggest that your
in favour of more freedom, rather than less: you've made a disparaging
comment about Microsoft and Apple, you apparently think highly of Open
Source software, and more. Does this freedom extend to people who have
criticisms -- even mild criticisms -- of Python?

Else, well, I just don't get the point. What you are fussing about are
fundamental design choices (and philosophic points) that are well known,
explained, advertized, etc. - and are really not likely to change
anytime soon.

No, probably not. Any such changes would have to start off as an external
tool first. I'd guess that it couldn't even be considered as part of the
base language until Python 4, which is probably a decade away.

This is a good thing. There are, as you have so often pointed out, many
B&D languages out there, and they are a PITA to use. Nobody wants to make
Python unpleasant to use, but if you think data hiding is automatically
unpleasant, then you must *hate* calling len() on a list, because that
uses private data inaccessible to Python code. That's data hiding.

It will probably take a decade of experiments to find a way to get as
many of the benefits of data hiding as possible with as few of the costs
as we can get away with. It may even be that it is impossible to do so,
that you can't get the safety of data hiding without the pain. Who knows?
I think it's an experiment worth trying.

And for all those who insist that Python will never have data hiding
except by convention, a few years ago I was saying that Python would
never have static types, because of fundamental design choices to make
Python a dynamic typed language. And now Python has ABCs, and optional
type annotations, and Guido is encouraging people to experiment with
tools that do things with those type annotations, things like type
inference, and if they become popular and useful, they may end up in the
language itself.

As has been frequently pointed out, Python is not Java. Java is obsessed
with the past, while Python is forward-looking.

http://cafe.elharo.com/programming/java-is-dead-long-live-python/

Thank goodness for that.

 

[Bruno Desthuilliers]

Russ P. a écrit :
(snip)

I am curious about something. Have you ever needed to access a
"private" attribute (i.e., one named with a leading underscore) in
Python code that you did not have the source code for? For that
matter, have you ever even used a library written in Python without
having access to the source code?

No to both.

As I said before, if you have the source code you can always change
private attributes to public in a pinch if the language enforces
encapsulation.

And then have to maintain a fork. No, thanks.

But if you are working on a team project, you can't
change the code that another member of a team checks in.

Why on earth couldn't I change the code of another member of my team if
that code needs changes ? The code is the whole team's ownership.

Now and FWIW, in this case (our own code), I just don't need to "mess
with internals" - I just just change what needs to be changed.

That is how
enforced data hiding helps teams of developers manage interfaces.

I totally fails to find any evidence of this assertion in the above
"demonstration".

The
bigger the team and the bigger the project, the more it helps.

Your opinion.

Mr. D'Aprano gave an excellent example of a large banking program.
Without enforced encapsulation, anyone on the development team has
access to the entire program and could potentially sneak in fraudulent
code much more easily than if encapsulation were enforced by the
language.

My my my. If you don't trust your programmers, then indeed, don't use
Python. What can I say (and what do I care ?). But once again, relying
on the language's access restriction to manage *security* is, well, kind
of funny, you know ?

I am certainly not saying that Python is useless without enforced data
hiding. It is obviously very useful for a wide range of applications
and domains already. I am only saying that it's usefulness could be
enhanced if enforced encapsulation can be added

You might have a chance to sell this to a clueless pointy haired boss -
I mean, that enforced access restriction will make Python more suitable
for some big enterprizey project. As far as I'm concerned, I'm not
buying it.

without somehow
comromising the language.

Then just forget it.

I don't know enough about the inner workings
of the Python interpreter to know if that is the case or not (I am an
aeronautical engineer), but the objections I've seen so far on this
thread have not impressed me.

I've not seen much technical objections - but anyway: learn (I mean,
*really* learn) how Python's object model works, and you'll perhaps find
out why this would break the whole thing.

 

[Russ P.]

Suppose that you write a Python library module and release it.  I find
that it's /almost/ the right thing for some program of mine, but it
doesn't quite work properly unless I hack about like so... perfect!  I'm
a happy bunny; you've gained a user (maybe that's a good thing, maybe it
isn't!).  Now, I've hacked about in your module's internal stuff: how
has this affected you?  Answer: not at all; you probably didn't feel a
thing.  You release a new version with improved internal structure and
my program breaks: how has this affected you?  Answer: still not at all..
How did it affect me?  Quite a bit, but then again, I knew what I was
getting into.  I gambled and lost; oh, well, that happens sometimes.

Was this library module released in source form?

If so, then why would you care that it has enforced access
restrictions? You can just take them out, then do whatever you would
have done had they not been there to start with. I don't see how that
is any more work than figuring out what internals you need to access.
Either way you need to read and understand the code.

Wait ... it wasn't released in source form? Then how would you even
know what internals you need to access? And why would you use
something that goes against your philosophy of openness anyway?