Why does python not have a mechanism for data hiding?

A

Antoon Pardon

For the programmer who writes or uses the private API it isn't really
"private", he must document it or know how it works.

How does that make it not private. Private has never meant "accessible
to noone". And sure he must document it and know how it works. But that
documentation can remain private, limited to the developers of the
product. It doesn't have to be publicly documented.
 
M

Marc 'BlackJack' Rintsch

How does that make it not private. Private has never meant "accessible
to noone". And sure he must document it and know how it works. But that
documentation can remain private, limited to the developers of the
product. It doesn't have to be publicly documented.

If the audience is the programmer(s) who implement the "private" API it
is not private but public. Even the "public" API is somewhat "private" to
a user of a program that uses that API. The public is not virtually
anyone here. Depends at which level you look in the system.

Ciao,
Marc 'BlackJack' Rintsch
 
A

Antoon Pardon

If the audience is the programmer(s) who implement the "private" API it
is not private but public. Even the "public" API is somewhat "private" to
a user of a program that uses that API. The public is not virtually
anyone here. Depends at which level you look in the system.

I think there is a general consensus about on what level to look when we
are talking about private and public attributes. You can of course
start talking at a whole different level and as such use these words
with a meaning different than normally understood. But that will just
make it harder for you to get your ideas accross.
 
B

Bruno Desthuilliers

Antoon Pardon a écrit :
@NickC : InMyArms(tm) !
But the leading underscore doesn't tell you whether it is your own
private date, which you can use a you see fit, or those of someone
else, which you have to be very carefull with.

That's why we have __name_mangling too. Consider '_' as 'protected' and
'__' as private.
 
B

Bruno Desthuilliers

Russ P. a écrit :
(snip)
(answering to Carl Bank) I thought you were saying that encapsulation or so-called "data
hiding" is worthless.

As far as I'm concerned, I view encapsulation as very desirable, and
data-hidding as totally worthless when applied to Python's object model.
Here's what I think Python should have. I think it should have a
keyword, something like "priv," to identify data or functions as
"private." As I said earlier, "private" for class data or functions
("methods") could be implemented like "protected" in C++. That means
that derived classes would have access to it, but clients of the class
would not. If the client really needs or wants access, he could be
given a sort of "back door" access similar to the current Python rule
regarding double leading underscores. Thus, the client would have
access, but he would know very well that he is using something that
the original designer did not intend for him to use.

It's just a suggestion. I'm not a language expert, and I realize that
I could be missing something important.

Given your very recent discovery of what 'dynamic' *really* means in
Python (like, for exemple, dynamically adding / replacing attributes -
including methods - on a per-class or per-instance basis), possibly, yes.
I also realize, by the way, that Python allows a client of a class to
define a new class member from completely outside the class
definition. Obviously, that cannot be declared private.

Why so ?
But if the
same identifier is already declared private within the class, than the
new definition should not be allowed (because it would defeat the
whole idea of "private" class members).

Why so ?

Metaprogramming (including monkeypatching) is part of the pythoneer's
toolbox, and while it's not something to use without pretty good
reasons, it has many times proven to be a real life saver. In languages
not allowing it, the solutions to the class of problems easily solved by
monkeypatching happens to be at best a kludge, at worst plain
unsolvable, at least without too much effort to be even worth it. Your
above proposition would arbitrarily make possible and useful things
either uselessly complicated or near impossible.
 
F

Fuzzyman

Python has no data hiding because C++ has (void *).

Python underscores does some name mangling, but does not attempt any
data hiding.

Python and C has about the same approach to data hiding. It is well
tried, and works equally well in both languages:

# this is mine, keep your filthy paws off!!!

Irresponsible programmers should not be allowed near a computer
anyway. If you use data hiding to protect your code from yourself,
what you really need is some time off to reconsider your career.

So, you are stating that no API programmer using Python *ever* has a
valid or genuine reason for wanting (even if he can't have it) genuine
'hiding' of internal state or members from consumers of his (or
her...) API?

Michael Foord
http://www.ironpythoninaction.com/
 
S

sturlamolden

So, you are stating that no API programmer using Python *ever* has a
valid or genuine reason for wanting (even if he can't have it) genuine
'hiding' of internal state or members from consumers of his (or
her...) API?

Michael Foordhttp://www.ironpythoninaction.com/

If you are an API programmer, the __all__ attribute of a package or
module provides all the internal data hiding you need.
 
R

Russ P.

Antoon Pardon a écrit :


@NickC : InMyArms(tm) !


That's why we have __name_mangling too. Consider '_' as 'protected' and
'__' as private.

Only in some vague, fuzzy sense.

My understanding is that the single underscore in a class definition
is a convention only and has no actual effect whatsoever. In C++ (and
Java?), on the other hand, the "protected" keyword *really* prevents
the client from accessing the data or method, but it allows access to
derived classes. The "private" keyword goes further and prevents
access even by derived classes. The double leading underscore in
Python does no such thing.

By the way, people often claim that "friend" classes in C++ violate
encapsulation. That is a common misunderstanding. They do not violate
encapsulation because a class must declare its own "friends." In other
words, the determination of who gets acces to the private data in a
class is determined within the class itself. Declaring another class a
friend gives it access to your data but does not give you access to
its data. (At least that's my recollection, though I haven't used C++
for several years.)
 
G

George Sakkis

The "private" keyword goes further and prevents
access even by derived classes. The double leading underscore in
Python does no such thing.

Who develops these derived classes ? A competitor ? A malicious
hacker ? A spammer ? Who are you trying to hide your precious classes
from that the double leading underscore is not good enough
protection ? Even with a 'private' keyword, what stops them from doing
s/private/public/g ? Seriously, the "underscores are ugly" argument
has some merit but language enforced data hiding is overrated, if not
downright silly.
 
R

Russ P.

Russ P. a écrit :
Given your very recent discovery of what 'dynamic' *really* means in
Python (like, for exemple, dynamically adding / replacing attributes -
including methods - on a per-class or per-instance basis), possibly, yes.

My "very recent" discovery? Funny, I thought I knew that several years
ago. You must know more about me than I know about myself.

Why should the client of a class not be able to declare a *private*
member of the class? You're kidding, right? Do you mind if I tell you
how to arrange the furniture in your bedroom?
Why so ?

Metaprogramming (including monkeypatching) is part of the pythoneer's
toolbox, and while it's not something to use without pretty good
reasons, it has many times proven to be a real life saver. In languages
not allowing it, the solutions to the class of problems easily solved by
monkeypatching happens to be at best a kludge, at worst plain
unsolvable, at least without too much effort to be even worth it. Your
above proposition would arbitrarily make possible and useful things
either uselessly complicated or near impossible.

For the record, I have made it abundantly clear that I don't think
Python should not have as rigorous an encapsulation regime as C++ or
Java. The worst that could happen with my proposition is that you
would need to use a "mangled" name to access private data or methods.
But you will be using the name many times, you can reassign your own
name, of course, so the mangled name need not appear more than once
where it is needed.
 
R

Russ P.

Who develops these derived classes ? A competitor ? A malicious
hacker ? A spammer ? Who are you trying to hide your precious classes
from that the double leading underscore is not good enough
protection ? Even with a 'private' keyword, what stops them from doing
s/private/public/g ? Seriously, the "underscores are ugly" argument
has some merit but language enforced data hiding is overrated, if not
downright silly.

I did not claim that Python should have the same encapsulation rules
as C++. I was merely comparing the two in reply to a post that claimed
a similarity.
 
R

Russ P.

For the record, I have made it abundantly clear that I don't think
Python should not have as rigorous an encapsulation regime as C++ or
Java. The worst that could happen with my proposition is that you
would need to use a "mangled" name to access private data or methods.
But you will be using the name many times, you can reassign your own
name, of course, so the mangled name need not appear more than once
where it is needed.

Let me try that again:

For the record, I have made it abundantly clear that I don't think
Python should have as rigorous an encapsulation regime as C++ or
Java. The worst that could happen with my proposition is that you
would need to use a "mangled" name to access private data or methods.
But if you will be using the name many times, you can reassign your
own
name, of course, so the mangled name need not appear more than once
where it is needed.
 
B

bruno.desthuilliers

Only in some vague, fuzzy sense.

My understanding is that the single underscore in a class definition
is a convention only and has no actual effect whatsoever.

It has the expected effect: warn adult programmers that this is
implementation, not interface, so mess with it and you're on your
own.
In C++ (and
Java?), on the other hand, the "protected" keyword *really* prevents
the client from accessing the data or method, but it allows access to
derived classes.

And ?
The "private" keyword goes further and prevents
access even by derived classes.

And ?
The double leading underscore in
Python does no such thing.

No. It only make sure a child class won't *accidentally* mess things
up. And that's enough as far as I'm concerned.
 
R

Roy Smith

"Russ P. said:
In C++ (and
Java?), on the other hand, the "protected" keyword *really* prevents
the client from accessing the data or method, but it allows access to
derived classes. The "private" keyword goes further and prevents
access even by derived classes.

In C++, it does no such thing. Consider this class declaration in
MySecretClass.h:

class MySecretClass {
private:
int foo;
};

All somebody has to do to get at the private data is:

#define private public
# include <MySecretClass.h>
#undef private

If playing preprocessor games isn't your thing, there's a whole multitude
of other tricks you can play with pointers and typecasts that will get you
access to foo in other ways.

But, you protest, you're not supposed to do that! Well, of course not.
But you're not supposed to ignore the leading underscore convention in
Python either. That's the nice thing about freedom of religion; you get to
pick which particular sins you want to get freaked out about.

I'm pretty weak on Java, but my understanding is that it's in better shape
here, since it has neither textual inclusion of header files, nor pointers.
You might have to resort to JNI :)
 
R

Russ P.

All somebody has to do to get at the private data is:

#define private public
# include <MySecretClass.h>
#undef private

Well, that shows the weakness of the C/C++ header files. The "include"
directive merely does a simple text substitution, which is pretty lame
as far as I am concerned. As you say, Java has moved beyond that, and
Ada has always been more sophisticated than that.

By the way, my recollection is that in C++ access defaults to private
if nothing is declared explicity. So normally the "private"
declaration is unnecessary. If it is left out, your little trick won't
work. But your point is still valid.
 
D

David

Ben Finney said:
By definition, "private" functions are not part of the publicly
documented behaviour of the unit. Any behaviour exhibited by some
private component is seen externally as a behaviour of some public
component.

You know the difference between theory and reality? In theory, there is
none... Sometimes it's useful to test internal components. Imagine this
class:

class ArmegeddonMachine:
def pushTheButton(self):
"Destroy a random city"
city = self._pickCity()
self._destroy(city)

def _pickCity():
cities = ['New York', 'Moscow', 'Tokyo', 'Beijing', 'Mumbai']
thePoorSchmucks = random.choice(cities)
return 'New York'

def _destroy(self, city):
missle = ICBM()
missle.aim(city)
missle.launch()

The only externally visible interface is pushTheButton(), yet you don't
really want to call that during testing. What you do want to do is test
that a random city really does get picked.

You can do one of two things at this point. You can say, "But, that's not
part of the externally visible interface" and refuse to test it, or you can
figure out a way to test it. Up to you.

Sorry for the long post in advance. I'm busy studying unit testing$B!"(Band
here is how I would go about testing the above code.

1) Make the code more testable through public (not private)
interfaces. ie, split out the functionality into more public, but
still properly encapsulated forms (modules/classes, etc). Don't need
to make everything public, just enough that it's testable without
digging into privates attributes, but still well-encapsulated.

2) Link the split-up logic together (eg, in class constructors). Use
public attributes for this. Or use private attributes, but make them
updatable through public methods.

3) In the unit tests update the object being tested, so it uses mock
objects instead of the ones it uses by default (aka dependency
injection).

4) Run the methods to be tested, and check that the mock objects were
updated correctly.

Here is an example (very rough, untested & incomplete):

# Updated armageddonmachine module:

class CityPicker:
def pick(self):
cities = ['New York', 'Moscow', 'Tokyo', 'Beijing', 'Mumbai']
thePoorSchmucks = random.choice(cities)
return 'New York' # Your bug is still here

class Destroyer:
def destroy(self, city):
missle = ICBM()
missle.aim(city)
missle.launch()

class ArmegeddonMachine:
def __init__(self):
self.city_picker = CityPicker()
self.destroyer = Destroyer()

def pushTheButton(self):
"Destroy a random city"
city = self.city_picker.pick()
self.destroyer.destroy(city)

# unit test module for armageddonmachine
# Only has tests for CityPicker and ArmageddonMachine

import armageddonmachine

# Unit test code for CityPicker


# -- Mock objects --


class MockRandom:
def __init__(self):
self.choose = None
def choice(self, list_):
assert self.choose is not None
return list_[self.choose]


class TestCityPicker:
def setup()
# Setup code run before each unit test in this class
self.city_picker = CityPicker()
self._bkp_random = armaggedonmachine.random

def teardown()
# Teardown code run after each unit test in this class
armaggedonmachine.random = self._bkp_random

def test_should_pick_random_cities_correctly(self):
armaggedonmachine.random = MockRandom()

armaggedonmachine.choose = 0
assert city_picker.pick() == 'New York'

armaggedonmachine.choose = 1
assert city_picker.pick() == 'Moscow' # This will catch your bug

armaggedonmachine.choose = 2
assert city_picker.pick() == 'Tokyo'

armaggedonmachine.choose = 3
assert city_picker.pick() == 'Beijing'

armaggedonmachine.choose = 4
assert city_picker.pick() == 'Mumbai'

# Unit test code for ArmageddonMachine

# -- Mock Classes --


class MockCityPicker:
def __init__(self):
self.city_to_pick = "Test City"

def pick(self):
return self.city_to_pick


class MockDestroyer:
def __init__(self):
self.destroyed_cities = set()

def destroy(self, city):
self.destroyed_cities.append(city)


# -- Unit Tests

class TestArmageddonMachine:

def setup(self):
# Setup code that runs before each unit test in this class
self.machine = ArmageddonMachine()
self._bkp_picker = self.machine.city_picker
self._bkp_destroyer = self.machine.destroyer
self.machine.city_picker = MockCityPicker()
self.machine.destroyer = MockCityDestroyer()

def test_should_destroy_a_city(self):
self.machine.pushTheButton()
assert self.machine.destroyer.destroyed_cities == "Test City"
 
H

Hrvoje Niksic

Russ P. said:
By the way, my recollection is that in C++ access defaults to private
if nothing is declared explicity. So normally the "private"
declaration is unnecessary. If it is left out, your little trick won't
work.

How about #define class struct
 
R

Russ P.

Which will break the first time the "innards" rebind a value to the
mangled name, as the "simplified" external name will still be bound to
the previous value.

I'm not sure you understood what I meant. In current Python, if I need
access to data element __XX in class YourClass, I can use
ZZ._YourClass__XX, but if I don't want to clutter my code with that
mangled name, I can just write

XX = ZZ._YourClass__XX

and refer to it from that point on as XX. Obviously if the meaning of
__XX changes within class ZZ, this will break, but that's why you are
supposed to avoid using private data in the first place.
 
R

Russ P.

How about #define class struct

I never thought of that one. I wonder what the C++ gurus would say
about that.

Let me guess. They'd probably say that the access restrictions are for
your own good, and bypassing them is bound to do you more harm than
good in the long run. And they'd probably be right. Just because you
can break into a box labeled "DANGER HIGH VOLTAGE," that doesn't make
it a good idea.

This just goes to show that the whole idea of using header files as
simple text insertions is flaky to start with, and adding the
preprocessor just compounds the flakiness. Needless to say, I'm not a
big fan of C and C++.
 
C

cokofreedom

Someone asked about Java;

class FieldTest {
public String publicString = "Foobar";
private String privateString = "Hello, World!";
}

import java.lang.reflect.Field;

public class Test4 {
public static void main(String args[]) {
final Field fields[] =
FieldTest.class.getDeclaredFields();
for (int i = 0; i < fields.length; ++i) {
System.out.println("Field: " + fields);
}
}
}

OUTPUT >>>>
Field: public java.lang.String FieldTest.publicString
Field: private java.lang.String FieldTest.privateString

And to edit it;

import java.lang.reflect.Field;

public class Test7 {
public static void main(String args[])
throws Exception {
final Field fields[] =
FieldTest.class.getDeclaredFields();
for (int i = 0; i < fields.length; ++i) {
if ("privateString".equals(fields.getName())) {
FieldTest fieldTest = new FieldTest();
Field f = fields;
f.setAccessible(true);
System.out.println(f.get(fieldTest));
f.set(fieldTest, "Modified Field");
System.out.println(f.get(fieldTest));
break;
}
}
}
}

OUTPUT >>>>
Hello, World!
Modified Field

Enjoy.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,774
Messages
2,569,596
Members
45,128
Latest member
ElwoodPhil
Top