Why does python not have a mechanism for data hiding?

[Bruno Desthuilliers]

Russ P. a écrit :

That is true. But with the "priv" keyword you'll discover quickly
enough that you are trying to access private data (as soon as you run
the program).

With the current convention, you don't even have to run the program - as
soon as you *type* the name, you know you're accessing implementation stuff.

And even if a "priv" keyword is added, you are still
free to use the leading underscore convention if you wish.

What does this "priv keyword" buy you then ? Seriously ?

The idea of being able to discern properties of an object by its name
alone is something that is not normally done in programming in
general.

Want to talk about the hungarian notation that is everywhere in MS
Windows code ?-) or about the so common C++ coding guideline that
insists on prefixing "data members" with a m_ or a w_ ?-)

More seriously: yes, you usually try to convey something about some
relevant properties of the object in the identifier. Like, you know,
using plurals for collections. Or i, j as loop indices.

Yes, of course you should choose identifiers to be
descriptive of what they represent in the real world, but you don't
use names like "intCount," "floatWeight," or "MyClassMyObject" would
you?
Nope.

Why not?

Because the naming convention for variables is all_lower_with_underscores.

Also, because 'count' is likely enough to be an integer and 'weight'
likely enough to be a float - and else another numeric. While 'counts'
and 'weights' are likely enough to be collections (likely lists) of
resp. integers and floats.

 

[Bruno Desthuilliers]

Russ P. a écrit :

My "very recent" discovery? Funny, I thought I knew that several years
ago.

Looks like I mistook your
"""
I also realize, by the way, that Python allows a client of a class to
define a new class member from completely outside the class
definition
"""

as "I just realized (...)"

Sorry for having read too fast.

Why should the client of a class not be able to declare a *private*
member of the class? You're kidding, right?

I'm dead serious. I often add implementation attributes to either a
class or an instance. These *are* implementation parts, not API.

Do you mind if I tell you
how to arrange the furniture in your bedroom?

I must be a bit dumb, but I don't see how human interaction problems
relate to enforced access restriction in an OO programming language.

For the record, I have made it abundantly clear that I don't think
Python should not have as rigorous an encapsulation regime as C++ or
Java. The worst that could happen with my proposition is that you
would need to use a "mangled" name to access private data or methods.

That's already the case - when you use __name_mangling. And if there's
no effective access restriction, then what the point of having this
'priv' keyword ?

But you will be using the name many times, you can reassign your own
name, of course, so the mangled name need not appear more than once
where it is needed.

Once again, I just don't see the point. Either you want effective access
restriction in Python, or you don't. And if you don't, what would this
'priv' keyword be useful to ?

 

[Bruno Desthuilliers]

Russ P. a écrit :

I'm not sure you understood what I meant. In current Python, if I need
access to data element __XX in class YourClass, I can use
ZZ._YourClass__XX, but if I don't want to clutter my code with that
mangled name, I can just write

XX = ZZ._YourClass__XX

and refer to it from that point on as XX.
>
Obviously if the meaning of
__XX changes within class ZZ, this will break, but that's why you are
supposed to avoid using private data in the first place.

AFAICT, What Dennis meant is that the binding of ZZ._YourClass__XX
changes between the moment you bind it to local XX and the moment you
use it, then you're out.

 

[Russ P.]

I'm dead serious. I often add implementation attributes to either a
class or an instance. These *are* implementation parts, not API.

If a client accesses a data member of a class, then by definition that
member is not really private, so letting the client create a new data
member and declare it as private seems a bit silly to me. Actually,
just letting the client create the new data member, private or not,
seems like a bit of a stretch to me, but I'll leave that be.

That's already the case - when you use __name_mangling. And if there's
no effective access restriction, then what the point of having this
'priv' keyword ?


Once again, I just don't see the point. Either you want effective access
restriction in Python, or you don't. And if you don't, what would this
'priv' keyword be useful to ?

In the end, I suppose it boils down to aesthetics and personal
preference.

The leading-underscore convention bothers me for two reasons: (1) like
the OP, I don't like the way it makes my code look, and (2) it is a
signal to a person reading the code, but it has no actual effect in
the interpreter.

I think the concept of private data and methods is important enough to
be implemented with more than just a tacky naming convention. That is
why I suggested the "priv" keyword. At the same time, I realize that
people will occasionally be frustrated if they are rigorously denied
access to all private data, which is why I suggested an "indirect"
method of access through mangled names.

You can argue that such indirect access defeats the whole idea of
private data, but at least it alerts the client to the fact that he
(or she or it) is accessing private data -- and it does so without
using Hungarian notation.

I would let the "priv" keyword also be used for data or functions at
file scope. It just seems logical to me. Again, some name mangling
convention could be concocted for those who think they really need
access.

Actually, the whole objection to denied access baffles me a bit. Does
anyone object to not having access from outside a function to local
variables within the function? I doubt it. The other thing is that the
vast majority of Python software, I would guess, is provided with
source code. How many Python applications or libraries are provided
without source code? If you have the source code, you can obviously
just delete the "priv" keyword anywhere or everywhere it appears. And
if you have a major client who insists on access to all the internals,
just delete all occurrences of "priv" before you ship the code (or
don't use it to start with).

 

[Russ P.]

Russ P. a écrit :








AFAICT, What Dennis meant is that the binding of ZZ._YourClass__XX
changes between the moment you bind it to local XX and the moment you
use it, then you're out.

Perhaps I should have stipulated that this should be done only in a
local scope and in an application that is not multi-threaded. Then I
don't see how you can have a problem.

 

[BJörn Lindqvist]

Now of course noone would defend such a limitation on the grounds
that one doesn't need the general case and that the general case
will only save you some vertical space.

But when it came to the ternary operator that was exactly the
argument used, to defend the lack of it.

As far as I remember, the primary motivation was developers experience
with the ternary operator in other languages, especially C, where it
was found to hurt readability. At least in my experience, it is much
much more common to see the ternary operator making code more
obfuscated than easing readability. Time will tell if Python's if-else
expression will be abused in the same way.

 

[John Nagle]

As far as I remember, the primary motivation was developers experience
with the ternary operator in other languages, especially C, where it
was found to hurt readability. At least in my experience, it is much
much more common to see the ternary operator making code more
obfuscated than easing readability. Time will tell if Python's if-else
expression will be abused in the same way.

Expression IF statements are most useful in situations where you can't
have multiple statements. In C, that was usually in macros, or occasionally
in a function call:

printf("Boolean value: %s\n", b ? "True" : "False");

was useful, for example. The syntax was probably a bad choice.
The classic expression IF (an ALGOL extension) was

x := IF b THEN 1 ELSE 0;

Surrounded with parentheses, that form could be used in any expression
context.

Given Python's approach to indentation, conditionals
within expressions (and iteration; one could have FOR / YIELD) don't
fit the syntax. That's why lambda expressions in Python are so limited.
Python already has local functions; you can write:

def foo(n) :
def bar(m) :
print "In BAR",n, m
bar(n+1)

which also allows you to do anything you can do with a lambda expression.
You can have control structure, and it reads like ordinary Python.
So there's no real need for an expression IF operator.

John Nagle

 

[Mark Wooding]

This is bogus about 95% of the time though. For the cases where it is
really desired, I think it's best to require the target class to be
enable it specifically somehow, maybe by inheriting from a special
superclass. That could let the compiler statically resolve member
lookups the rest of the time.

No it wouldn't. Think about multiple inheritance.

-- [mdw]

 

[Mark Wooding]

How about #define class struct

Won't work. Consider `template<class T> ...'.

-- [mdw]

 

[Mark Wooding]

The idea of being able to discern properties of an object by its name
alone is something that is not normally done in programming in
general.

Really? You obviously haven't noticed Prolog, Smalltalk, Haskell, ML,
or Erlang then. And that's just the ones I can think of off the top of
my head.

* Prolog and Erlang distinguish atoms from variables by the case of
the first letter; also `_' is magical and is equivalent to a new
variable name every time you use it.

* Smalltalk distinguishes between global and local variables according
to the case of the first letter.

* Haskell distinguishes between normal functions and constructors
(both data constructors and type constructors) by the case of the
first letter, and has Prolog's `_' convention.

* ML allows a single-quote in variable names, but reserves names
beginning with a single-quote for type variables. It also has
Prolog's `_' convention.

As far as I can see, discerning properties of a thing from its name
seems relatively common.

-- [mdw]

 

[Mark Wooding]

So, you are stating that no API programmer using Python *ever* has a
valid or genuine reason for wanting (even if he can't have it) genuine
'hiding' of internal state or members from consumers of his (or
her...) API?

I don't want to speak for whoever you were responding to, but from my
point of view...

Yes.

I understand the difference between the documented interface of a system
and the details of its implementation. But sometimes it can be useful
to take advantage of implementation details, particularly if the
published interface is inadequate in some way. Whether or not I choose
to make use of implementation details is a trade-off between immediate
convenience and maintainability, but that's something I can make a
rational decision about.

By enforcing your `data hiding', you're effectively telling me that I'm
too stupid to make rational decisions of this sort. And that's actually
extremely insulting.

-- [mdw]

 

[Roy Smith]

* Prolog and Erlang distinguish atoms from variables by the case of
the first letter; also `_' is magical and is equivalent to a new
variable name every time you use it.

Can you explain that in more detail? A literal reading of what you wrote
would mean that if you did (assuming this is even legal syntax):

_ = 1;
y = _;

the _'s are different variables, which is absurd enough to make me believe
I just misunderstood you.

 

[Roy Smith]

By enforcing your `data hiding', you're effectively telling me that I'm
too stupid to make rational decisions of this sort. And that's actually
extremely insulting.

I think that's taking it a bit far. Python doesn't let you manipulate
pointers directly. For example, I can't do:

s = "foo"
sp = address(s)
sp[2] = 'x'
print s

and have it print "fox". Is this because I'm too stupid to make rational
decision of this sort? No, it's because the Python programming model
exposes some things and hides others which are deemed inappropriate or too
low level. One of the things it hides is direct access to raw memory. I
don't see that as fundamentally different from a C++ string class which
declares its internal buffer to be private. If the goose's pot is black,
then the gander's kettle is an equal shade of dark grey.

 

[Luis Zarrabeitia]

I don't know about Erlang (though I'd think it's behaviour sould be similar to
prolog), but at least in Prolog, yes, _ and _ are different variables. The whole
idea of "_" is that it is a placeholder that can bind to anything, but will be
immediately discarded. It's just syntactic sugar so you don't have to create new
names for things you don't care about, which could be a problem in prolog (once
bound, cannot be bound again on the same branch) or erlang (once bound, bound
forever).

I just tried on erlang:

f(0,_) -> 1+_;
f(X,_) -> X*f(X-1,_).

produces an error:

../t.erl:4: variable '_' is unbound
../t.erl:5: variable '_' is unbound

(referring to the right side uses of the _ symbol)

while the definition:

f(0,Y)->1+Y;
f(X,Y)->X*f(X-1,Y).

produces a [very weird, just for testing purposes, don't use it at home] version
of 'factorial'.

So, you understood well.

As I haven't been following this thread, I won't go offtopic talking about
functional languages. But yes, on functional (and declarative?) languages, it
makes a lot of sense to have a 'symbol' that represents a new variable any time
you use it.

Cheers,

 

[Russ P.]

By enforcing your `data hiding', you're effectively telling me that I'm
too stupid to make rational decisions of this sort. And that's actually
extremely insulting.

1) I suggest you not take it personally.

2) Local data within functions is hidden. Should you have access to
that too? Are you insulted that you don't?

3) I have suggested that "indirect" or "back door" access could be
provided to private data and methods via some sort of name mangling
rule akin to the current rule for leading double underscores. This
would provide access in a pinch, but it would make sure the client is
aware that he or she or it is accessing private data (and it would do
so without leading underscores).

4) My understanding is that most Python software is released or
shipped as source code (or at least with the source code included).
That means that the client would need only to remove my suggested
"priv" keyword to gain access. Have you ever actually had to use
Python software for which you had no access to the source code?

 

[Russ P.]

Really? You obviously haven't noticed Prolog, Smalltalk, Haskell, ML,
or Erlang then. And that's just the ones I can think of off the top of
my head.

* Prolog and Erlang distinguish atoms from variables by the case of
the first letter; also `_' is magical and is equivalent to a new
variable name every time you use it.

* Smalltalk distinguishes between global and local variables according
to the case of the first letter.

* Haskell distinguishes between normal functions and constructors
(both data constructors and type constructors) by the case of the
first letter, and has Prolog's `_' convention.

* ML allows a single-quote in variable names, but reserves names
beginning with a single-quote for type variables. It also has
Prolog's `_' convention.

As far as I can see, discerning properties of a thing from its name
seems relatively common.

Well, "common" in Prolog, Smalltalk, Haskell, ML, and Erlang is hardly
common in general. I'll bet that Java and C/C++ are used more in North
Dakota than all those languages combined are used in the entire world.
That's not to say they aren't interesting academic languages, of
course.

As for using the case of the first letter to distinguish between local
and global variables, is that just a naming convention or is it
actually enforced by the compiler (or interpreter)? If it's actually
enforced, that seems rather restrictive to me. What if I want to name
a local variable after a proper name or an uppercase acronym? I guess
I'm just out of luck.

 

[John Salerno]

data hiding. I know member vars are private when you prefix them with
2 underscores, but I hate prefixing my vars, I'd rather add a keyword
before it.

Others touched on this, but I thought I'd throw it in here as well since
I was just reading about this. Apparently the double underscore
convention has nothing to do with data hiding. It's simply a mechanism
for avoiding name clashes, and data hiding only *seems* to be a result
of it.

Not that that helps your situation any.

 

[Bruno Desthuilliers]

Mark Wooding a écrit :

Really? You obviously haven't noticed Prolog, Smalltalk, Haskell, ML,
or Erlang then. And that's just the ones I can think of off the top of
my head.

* Prolog and Erlang distinguish atoms from variables by the case of
the first letter; also `_' is magical and is equivalent to a new
variable name every time you use it.

* Smalltalk distinguishes between global and local variables according
to the case of the first letter.

* Haskell distinguishes between normal functions and constructors
(both data constructors and type constructors) by the case of the
first letter, and has Prolog's `_' convention.

* ML allows a single-quote in variable names, but reserves names
beginning with a single-quote for type variables. It also has
Prolog's `_' convention.

You could add Ruby's naming rules here. Or the very very common
convention of using ALL_UPPER for symbolic constants (or
pseudo-constants as in Python). And *quite a lot* of other either
cross-language or language-specific naming conventions.

 

[Bruno Desthuilliers]

Russ P. a écrit :

Well, "common" in Prolog, Smalltalk, Haskell, ML, and Erlang is hardly
common in general. I'll bet that Java and C/C++ are used more in North
Dakota than all those languages combined are used in the entire world.

And you'll very probably loose.

That's not to say they aren't interesting academic languages, of
course.

Erlang an "academic" language ? Man, you're either a troll or totally
clueless.

 

[Bruno Desthuilliers]

Russ P. a écrit :

If a client accesses a data member of a class,

Please stop thinking in C++. This is Python, and all you have are
attributes. Whether they're callable of not doesn't change much here.

then by definition that
member is not really private,

Who cares about it being "private" ? The important point is that it's
*implementation*, not *interface*.

so letting the client create a new data
member and declare it as private seems a bit silly to me.

There are two ways to decorate a class or instance object (nb: Python's
classes are objects too): the "static" way where you wrap the object in
a decorator class or instance and use delegation, and the "dynamic" way
where you just modify the original class or object at runtime. The fact
that an attribute is added (or replaced) outside the class statement
doesn't mean it has to be part of the interface. You sometime have
pretty legitimate reason to modify the implementation at runtime.

Actually,
just letting the client create the new data member, private or not,
seems like a bit of a stretch to me, but I'll leave that be.

You're way too much in the Java/C++ way of thinking.

In the end, I suppose it boils down to aesthetics and personal
preference.

The leading-underscore convention bothers me for two reasons: (1) like
the OP, I don't like the way it makes my code look, and (2) it is a
signal to a person reading the code, but it has no actual effect in
the interpreter.

Indeed. The target of the signal is definitively the person reading the
code.

I think the concept of private data and methods is important enough to
be implemented with more than just a tacky naming convention.

The concept of "private" attributes is not important. What's important
is the concept of implementation attributes.

That is
why I suggested the "priv" keyword. At the same time, I realize that
people will occasionally be frustrated if they are rigorously denied
access to all private data, which is why I suggested an "indirect"
method of access through mangled names.

We already have all this. Either you want language-enforced access
restriction - then you might be happier with another language - or you
just want to have a clear way to know whether an attribute is part of
the API or not - in which case a naming convention is not only enough,
but even better.

You can argue that such indirect access defeats the whole idea of
private data, but at least it alerts the client to the fact that he
(or she or it) is accessing private data

So does the naming convention, with much less work.

-- and it does so without
using Hungarian notation.

I wouldn't label this "hungarian notation" - or at least, not the way
"hungarian notation" is now commonly understood.

I would let the "priv" keyword also be used for data or functions at
file scope. It just seems logical to me. Again, some name mangling
convention could be concocted for those who think they really need
access.

Actually, the whole objection to denied access baffles me a bit.

Free your mind from the very peculiar, restricted and IMHO braindead
conception of "OO" they taught you with Java and C++. Python is older
than Java, it's by now a very very commonly used language on all major
platforms, and experience prooves that you just don't need anything more
than a simple naming convention.

Does
anyone object to not having access from outside a function to local
variables within the function? I doubt it. The other thing is that the
vast majority of Python software, I would guess, is provided with
source code. How many Python applications or libraries are provided
without source code? If you have the source code, you can obviously
just delete the "priv" keyword anywhere or everywhere it appears.

Yes, fine. And then have to maintain a fork of the source code, and
distribute it with the application. Honking great idea. doh :-(

And
if you have a major client who insists on access to all the internals,
just delete all occurrences of "priv" before you ship the code (or
don't use it to start with).

Or don't even bother about this useless access restriction stuff. Which
will save you quite a lot of valuable time.