Calling private base methods

[Jorgen Bodde]

Hi All,

Now that I am really diving into Python, I encounter a lot of things
that us newbies find difficult to get right. I thought I understood
how super() worked, but with 'private' members it does not seem to
work. For example;
.... def __baseMethod(self):
.... print 'Test'

Deriving from A, and doing;
.... def someMethod(self):
.... super(A, self).__baseMethod()
.... print 'test3'

Will not work;
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "<interactive input>", line 3, in someMethod
AttributeError: 'super' object has no attribute '_D__baseMethod'

Is it possible to call a private base method? I come from a C++
background, and I liked this construction as my base class has helper
methods so that I do not have to duplicate code.

When I do;
.... def someMethod(self):
.... print 'Hello'
........ def otherMethod(self):
.... super(F, self).someMethod()
.... print 'There'
....

Hello
There

This seems to work.

Thanks in advance,
- Jorgen

 

[Enrico]

Hi All,

Now that I am really diving into Python, I encounter a lot of things
that us newbies find difficult to get right. I thought I understood
how super() worked, but with 'private' members it does not seem to
work. For example;

... def __baseMethod(self):
... print 'Test'

Deriving from A, and doing;

... def someMethod(self):
... super(A, self).__baseMethod()
... print 'test3'

Will not work;

if you type
you'll get a method named
_A__baseMethod

From the documentation:
Private name mangling: When an identifier that textually occurs in a class
definition begins with two or more underscore characters and does not end in
two or more underscores, it is considered a private name of that class.
Private names are transformed to a longer form before code is generated for
them. The transformation inserts the class name in front of the name, with
leading underscores removed, and a single underscore inserted in front of
the class name. For example, the identifier __spam occurring in a class
named Ham will be transformed to _Ham__spam. This transformation is
independent of the syntactical context in which the identifier is used. If
the transformed name is extremely long (longer than 255 characters),
implementation defined truncation may happen. If the class name consists
only of underscores, no transformation is done.

Enrico

 

[7stud]

Is it possible to call a private base method? I come from a C++
background, and I liked this construction as my base class has helper
methods so that I do not have to duplicate code.

I'd like to see some C++ code that does that!

 

[Duncan Booth]

I'd like to see some C++ code that does that!

Easy:

#define private public
#include <someheader>
#undef private

then call the private methods as much as you want.

 

[7stud]

Easy:

#define private public
#include <someheader>
#undef private

then call the private methods as much as you want.

lol. I don't see any private methods being created there.

 

[Duncan Booth]

lol. I don't see any private methods being created there.

You should have looked in someheader:

class Whatever {
private:
void ohnoyoudont(int);
}

then back in the C file:


....
Whatever foo = new Whatever();
int ohyesido = 42;
foo.ohnoyoudont(ohyesido);
....


Really, it does work (probably). There are other ways to get at private
members in C++ but this is the easiest.

 

[Dan Bishop]

You should have looked in someheader:

class Whatever {
private:
void ohnoyoudont(int);

}

then back in the C file:

...
Whatever foo = new Whatever();
int ohyesido = 42;
foo.ohnoyoudont(ohyesido);
...

Really, it does work (probably). There are other ways to get at private
members in C++ but this is the easiest.

I have a job as a C++ programmer and once tried this trick in order to
get at a private member function I needed. Didn't work: Apparently, VC
++ includes the access level in its name mangling, so you get linker
errors.

After that, I had a much greater appreciation for Python's lack of
"private".

 

[7stud]

You should have looked in someheader:

class Whatever {
private:
void ohnoyoudont(int);

}

then back in the C file:

...
Whatever foo = new Whatever();
int ohyesido = 42;
foo.ohnoyoudont(ohyesido);
...

Really, it does work (probably). There are other ways to get at private
members in C++ but this is the easiest.

I can also access private methods of a class if my sister backspaces
over "private" and types "public" instead.

In your example, no private methods were ever created, and therefore
you are not accessing private methods.

 

[Duncan Booth]

I can also access private methods of a class if my sister backspaces
over "private" and types "public" instead.

In your example, no private methods were ever created, and therefore
you are not accessing private methods.

Are you being deliberately obtuse? The methods are private throughout the
program *except* for the one source file where you do this trick.

It won't work if the compiler mangles the protection into the name, but I
have used this successfully with a C++ library where I absolutely couldn't
do what I needed without getting at a private method. The fact that I had
to resort to this trick is a big indication of course that genuinely
private members (as opposed to a 'keep off' naming convention) are a bad
idea in general.

If the name does get mangled then you may of course have to do other much
messier tricks to get at the address of the function and then call it.

 

[Duncan Booth]

I have a job as a C++ programmer and once tried this trick in order to
get at a private member function I needed. Didn't work: Apparently, VC
++ includes the access level in its name mangling, so you get linker
errors.

I don't have a copy of VC to hand to test, but I might have tried something
along these lines:

somheader contains:

class Whatever {
private:
void ohnoyoudont(int);
}


My C file:

#define ohnoyoudont blah(){};public: \
inline void sneak(int x){this.ohnoyoudont(x);};private:void ohnoyoudont
#include <someheader>
#undef ohnoyoudont

....
Whatever foo = new Whatever();
int ohyesido = 42;
foo.sneak(ohyesido);

I don't think injecting another couple of non-virtual methods into the
class ought to break anything.

 

[Isaac Rodriguez]

The fact that I had
to resort to this trick is a big indication of course that genuinely
private members (as opposed to a 'keep off' naming convention) are a bad
idea in general.

The fact that you had to resort to this trick is a big indication that
the library you were using is bad designed; it has nothing to do with
private members being a bad idea. You were using a library which
interface was in-complete (provided that you "genuinely" really needed
to access the private member to do what you wanted to do).

Private members is a great thing. They are the foundations of
encapsulation and object oriented design. The fact that Python allows
you to access "private" methods because the only "restriction" is the
name mangling does not mean you should access them and use them
directly.

I don't see the way private members are handled in Python a strenght
or a weakness of the language as compared to other languages. However,
I do see libraries that do not provide all the needed interface
methods as poor designed, or programmers that try to work around the
public interface of a class when it is not needed as poor programmers.

Thanks,

- Isaac.

 

[Duncan Booth]

The fact that you had to resort to this trick is a big indication that
the library you were using is bad designed; it has nothing to do with
private members being a bad idea. You were using a library which
interface was in-complete (provided that you "genuinely" really needed
to access the private member to do what you wanted to do).

I agree with that to a certain extent, but I've never found any situation
where I gained any benefit because someone had made part of the
implementation private, only ever found annoyance because someone had made
something private which shouldn't have been.

The problem is that when people design interfaces they don't (and
cannot) know all the situations in which the code is going to be used in
the future. Clearly separating the published interface from the
implementation details is a good thing, but physically preventing access to
those details is IMHO a bad thing.

 

[Paul Rubin]

The problem is that when people design interfaces they don't (and
cannot) know all the situations in which the code is going to be used in
the future. Clearly separating the published interface from the
implementation details is a good thing, but physically preventing access to
those details is IMHO a bad thing.

The idea is to make the implementation details independent from the
calling program. For example, I'm using a library now that does
something highly CPU intensive. To speed up the application, I may
put the library on a completely separate computer, so that the
published API exposed to the caller becomes a wrapper for a network
client, and all those library implementation details are on the other
machine. That's the ultimate in physical access prevention, there's
good reason to do it, and it completely breaks if the calling program
is using anything except the published API.

 

[Diez B. Roggisch]

The idea is to make the implementation details independent from the
calling program. For example, I'm using a library now that does
something highly CPU intensive. To speed up the application, I may
put the library on a completely separate computer, so that the
published API exposed to the caller becomes a wrapper for a network
client, and all those library implementation details are on the other
machine. That's the ultimate in physical access prevention, there's
good reason to do it, and it completely breaks if the calling program
is using anything except the published API.

So what? This will happen under under circumstances as well. Think of an
interface returning None in an earlier version, then [] to indicate an
empty result-set.

the point is that privacy is good to indicate that "you shouldn't mess
with this unless you know what you do, which includes not to use new
versions of the library."

I totally agree with Duncan that I've been in plenty situations where
exploiting inner workings of some 3rd-party-code made my programming
easier, only of course when nothing else helped.

After all, that's what duck-typing is about. There is no official
interface declaration, just an implicit protocol. And "private" methods
or members are part of that protocol as well.

Diez

 

[Alex Martelli]

The fact that you had to resort to this trick is a big indication that
the library you were using is bad designed; it has nothing to do with
private members being a bad idea. You were using a library which

This is just like the airlines invariably concluding "human error" in
just about every instance of an airplane crash or similar problem... Don
Norman has some choice reflection about that issue from the point of
view of a user-experience specialist, or, one could try Charles Perrow's
classic "Normal Accidents" for more of a systems-level view from a
sociologist's viewpoint.

C++'s and Java's approaches are vitiated by an unspoken assumption that
the library's designer is some kind of demigod, while the writer of code
that uses the library is presumably still struggling with the challenge
of opposable thumbs. In real life, the skills of the two people in
question are likely to be much closer, and since designing libraries for
use in all kinds of applications is a really hard task, it's likelier
than the library designer will make an error in designing his or her
library, rather than the application programmer in using that library.

Purely-advisory encapsulation approaches, like Python's, even the odds.
Actually, I'd argue that even double-leading-underscores are overkill
more often than not (and single-leading-underscores the compromise that
is generally prefereable).


Alex

 

[Isaac Rodriguez]

After all, that's what duck-typing is about. There is no official
interface declaration, just an implicit protocol. And "private" methods
or members are part of that protocol as well.

I don't think so. Duck-typing is about implementing the expected
public interface, and has nothing to do with accessing private members
of a class, nor overriding those members.

- Isaac.

 

[Isaac Rodriguez]

C++'s and Java's approaches are vitiated by an unspoken assumption that
the library's designer is some kind of demigod, while the writer of code
that uses the library is presumably still struggling with the challenge
of opposable thumbs.

That might be your point of view. To me, the library designer is the
one that has done the homework and should know better how to simplify
things for others not a God. No one uses a high level library if
implementing the low-level your self is easier. Libraries provide
functionality that allow the application programmer to concentrate in
what he is being paid for, making the application. An application
programmer will have to define what the correct interface for the
application is (for example, what UI to provide). There will be users
that will say, I wish this application had a way of doing this, but
unless they were technically savy and wanted to spend the necessary
time to understand how the application works, they would not write the
feature themselves; they will request that feature to the programmer.

In real life, the skills of the two people in
question are likely to be much closer, and since designing libraries for
use in all kinds of applications is a really hard task, it's likelier
than the library designer will make an error in designing his or her
library, rather than the application programmer in using that library.

Those are called defects or "bugs". When I find a bug in a library or
an application, I submit a bug report. I might have to work around it
for a little bit, but I don't just work around it and call it goods.
Library designers are normal programmers that work under a set of
requirements like any other programmer (application programmers
included), but when I find a bug in an application, I report it and
try to work around it until it gets fixed; I don't hack the
application or re-write my own just because a found a bug.

Purely-advisory encapsulation approaches, like Python's, even the odds.
Actually, I'd argue that even double-leading-underscores are overkill
more often than not (and single-leading-underscores the compromise that
is generally prefereable).

You see it as an intelligence challenge, where I see it as making
things easier for everybody.

Thanks,

- Isaac.

 

[Duncan Booth]

Those are called defects or "bugs". When I find a bug in a library or
an application, I submit a bug report. I might have to work around it
for a little bit, but I don't just work around it and call it goods.
Library designers are normal programmers that work under a set of
requirements like any other programmer (application programmers
included), but when I find a bug in an application, I report it and
try to work around it until it gets fixed; I don't hack the
application or re-write my own just because a found a bug.

You appear to have led a very sheltered life if the only libraries you ever
use are ones where you can always get a change to the library api in a
timely manner.

In the particular case where I hacked around a private variable, the
library in question was burned into ROM on thousands of consumer devices.
Getting the library to change simply wasn't an option.

 

[Isaac Rodriguez]

You appear to have led a very sheltered life if the only libraries you ever
use are ones where you can always get a change to the library api in a
timely manner.

The thing here is that we are not talking about my life. I may not
have expressed my self correctly, but you are not understanding the
point I am trying to make. You can say that you don't like C++ or Java
because they put too much restriction when members are declared
private. That you prefer Python's approach because in a time of need
it will make your life easier. All that is fine with me. It is just a
matter of taste.

But the truth is that C++ and Java made a decision to do that for a
reason, and the times when you have to work around those language
features come once in a blue moon; they are the exception, not the
rule, and you don't implement features in a language, or for that
matter in an application, to simplify the exceptions; you try to
implement the most common scenarios. Which features you add to your
programs? The features your customers' ask for because they need them
and they use them all the time or the ones that you like to implement
even if they are not ever used?

Thanks,

- Isaac

 

[Marc 'BlackJack' Rintsch]

But the truth is that C++ and Java made a decision to do that for a
reason, and the times when you have to work around those language
features come once in a blue moon; they are the exception, not the
rule, and you don't implement features in a language, or for that
matter in an application, to simplify the exceptions; you try to
implement the most common scenarios.

So the most common scenario is that programmers try to poke around all the
time in the internals of classes even if the need to do so is
very rare? Otherwise it would not be necessary to have and use a
mechanism to declare everything private. ;-)

Ciao,
Marc 'BlackJack' Rintsch