I want to see all the variables

[johnf]

Hi,
When I use dir() I don't see the __ underscore items. Is there anything
that will show all the private vars and functions?

johnf

 

[Larry Bates]

Hi,
When I use dir() I don't see the __ underscore items. Is there anything
that will show all the private vars and functions?

johnf

The idea of the underscore items is that they aren't to be used by
you. If you wish to access private variables and functions you will
almost certainly have to look at the source code to make sure of
what they are and how they can be utilized.

-Larry

 

[wittempj]

What do you mean? Can you specify which special functions you don't
see?
I get:
py> class X:
pass
py> dir(X)
['__doc__', '__module__']
py> class X:
def __getitem__(self, x):
pass


py> dir(X)
['__doc__', '__getitem__', '__module__']

 

[Steven D'Aprano]

What do you mean? Can you specify which special functions you don't
see?
I get:
py> class X:
pass
py> dir(X)
['__doc__', '__module__']

How about these?
()

Now that's interesting... if __name__ and __bases__ don't live in the
class __dict__, where do they live? What other methods and attributes are
invisibly in X?

 

[Steven D'Aprano]

The idea of the underscore items is that they aren't to be used by
you.

Double leading+trailing underscore attributes are NOT private, they are
*special* but public (e.g. __dict__, __class__, __str__, etc.). If you
don't believe me, have a look at dir(int) and count the underscored
attributes listed. Then try to find __dict__, __name__, __bases__,
__base__ or __mro__ within the list. Why are they suppressed?

But even if underscored attributes were private, the Python philosophy is
that private attributes are private by convention only -- even
name-mangled __private methods can be reached if you know how.

If you wish to access private variables and functions you will
almost certainly have to look at the source code to make sure of
what they are and how they can be utilized.

Not so.
.... def _private(self):
.... """Private method, returns a magic string."""
.... return "Don't touch!!!"
...."Private method, returns a magic string."

 

[wittempj]

What do you mean? Can you specify which special functions you don't
see?
I get:
py> class X:
pass
py> dir(X)
['__doc__', '__module__']How about these?

X.__dict__{'__module__': '__main__', '__doc__': None}>>> X.__name__ 'X'
X.__bases__()

Now that's interesting... if __name__ and __bases__ don't live in the
class __dict__, where do they live? What other methods and attributes are
invisibly in X?

Well, then we have to lookup what dir() does --
http://docs.python.org/lib/built-in-funcs.html -- it tries to bring up
the interesting attributes of an object, apparently __name__ and
__bases__ are considered to be not so interesting....

 

[johnf]

Double leading+trailing underscore attributes are NOT private, they are
*special* but public (e.g. __dict__, __class__, __str__, etc.). If you
don't believe me, have a look at dir(int) and count the underscored
attributes listed. Then try to find __dict__, __name__, __bases__,
__base__ or __mro__ within the list. Why are they suppressed?

But even if underscored attributes were private, the Python philosophy is
that private attributes are private by convention only -- even
name-mangled __private methods can be reached if you know how.



Not so.

... def _private(self):
... """Private method, returns a magic string."""
... return "Don't touch!!!"
...
"Private method, returns a magic string."

Ok then how do debug when I have something like "__source" and I need to
know what is available for the object?

John

 

[Steven D'Aprano]

Ok then how do debug when I have something like "__source" and I need to
know what is available for the object?

Outside of a class, objects with two leading underscores are just ordinary
objects with no special behaviour:
2

Objects with a single leading underscore like _source are slightly
special: if you do "from module import *" any names with a single leading
underscore are not imported.

Class attributes with two leading underscores like __source are considered
private to the class, so in general you shouldn't need to know anything
about them. To enforce that, Python mangles the name so it is harder to
reach.

But if/when you do need to get to them, they are easy to get to:
.... __attribute = 2
....Traceback (most recent call last):
2

Notice that Python doesn't generally try to hide "private" attributes:
['_Spam__attribute', '__doc__', '__module__']

There are three other underscore conventions in use:

(1) Objects with a single leading underscore like _attribute are private
by convention, but Python doesn't enforce it. Starting an object with a
single underscore is like writing "# Private! Don't touch!" after it.

(2) By convention, if you want to create a name that is the same as a
built-in object without shadowing (hiding) the built-in, put a single
trailing underscore after it like "file_". That's just a style convention
though, you are free to call it FiLE ,or anything else, if you prefer.

(3) Last but not least, class attributes with two leading and trailing
underscores are considered special but public, like __init__ and __repr__.
It is probably a bad idea to invent your own.

 

[Gabriel Genellina]

()

Now that's interesting... if __name__ and __bases__ don't live in the
class __dict__, where do they live? What other methods and attributes are
invisibly in X?

They live in the C structure implementing the type; members tp_bases
and tp_name respectively. Most predefined type/class attributes have
an slot for storing its value, for efficiency. See include/object.h


--
Gabriel Genellina
Softlab SRL






__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

 

[johnf]

Steven D'Aprano wrote:

There are three other underscore conventions in use:

(1) Objects with a single leading underscore like _attribute are private
by convention, but Python doesn't enforce it. Starting an object with a
single underscore is like writing "# Private! Don't touch!" after it.

(2) By convention, if you want to create a name that is the same as a
built-in object without shadowing (hiding) the built-in, put a single
trailing underscore after it like "file_". That's just a style convention
though, you are free to call it FiLE ,or anything else, if you prefer.

(3) Last but not least, class attributes with two leading and trailing
underscores are considered special but public, like __init__ and __repr__.
It is probably a bad idea to invent your own.

Very detailed. But I was attempting to debug some code which subclassed
other code. I got a traceback that something like "no
mySubClass.__source.query() did not exist". The superclass had something
like "myClass.__source.query(sql)" which worked
but "mySubClass.__source.query(sql)" did not work. So I tried to debug
using "dir(myClass.__source)" and got an error. And I also got error when
I "dir(mySubClass.__source". So how could I have debugged the problem if
dir() will not display information on the __source? I hope that explains
my issue.

Thanks
Johnf

 

[Rene Fleschenberg]

Very detailed. But I was attempting to debug some code which subclassed
other code. I got a traceback that something like "no
mySubClass.__source.query() did not exist".

By (strong) convention, "__" means "not to be accessed from outside the
class, not even from subclasses". The author of the original class does
not want you to access that attribute.

The superclass had something
like "myClass.__source.query(sql)" which worked
but "mySubClass.__source.query(sql)" did not work. So I tried to debug
using "dir(myClass.__source)" and got an error.

"dir(myClass._myClass__source)" should work, as should
"dir(mySubClass._myClass__source)" and "mySubClass._myClass__source".

But remember that it is usually really bad style to do this and that
there is probably a reason why the author of myClass does not want you
to do that. If myClass was written by yourself, simply do not use "__".
Only use "__" when you really know why you are doing it. Otherwise, just
use "_".
See also: http://docs.python.org/ref/atom-identifiers.html

And I also got error when
I "dir(mySubClass.__source". So how could I have debugged the problem if
dir() will not display information on the __source? I hope that explains
my issue.

A simple dir() on myClass could have given you a hint on what happens
with __source.

--
René
OpenPGP key id: 0x63B1F5DB
JID: (e-mail address removed)


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iEYEAREKAAYFAkWXIvkACgkQUVK8U2Ox9dtL1QCeKjjLlB+x06asdLxG2t6fTam9
DtEAn08y2FwmDgg4cMRRdvwJ0ZrFavLt
=ZZBt
-----END PGP SIGNATURE-----

 

[Steven D'Aprano]

Very detailed. But I was attempting to debug some code which subclassed
other code. I got a traceback that something like "no
mySubClass.__source.query() did not exist". The superclass had something
like "myClass.__source.query(sql)" which worked
but "mySubClass.__source.query(sql)" did not work.

That's right, exactly as expected.

As I said before, Python mangles the name of double-leading underscore
class attributes. So when the developer of myClass wrote

myClass.__source.query(sql)

at compile time Python changes that to

myClass._myClass__source.query(sql)


When you write mySubClass.__source.query() it doesn't work because there
is no attribute "__source". You need to change that to "_myClass__source"
-- even when it appears in mySubClass.

So I tried to debug
using "dir(myClass.__source)" and got an error. And I also got error when
I "dir(mySubClass.__source". So how could I have debugged the problem if
dir() will not display information on the __source? I hope that explains
my issue.

This isn't actually a problem with dir -- although dir does sometimes hide
methods, this isn't one of those cases. This is a case of you not knowing
that Python mangles the name of __attributes. What you should have done is
call dir(myClass), in which case you almost certainly would have seen
_myClass__source listed.

 

[Steven D'Aprano]

By (strong) convention, "__" means "not to be accessed from outside the
class, not even from subclasses". The author of the original class does
not want you to access that attribute.

What does the author of the original class know about *my* needs and
requirements? It may turn out that accessing his "private" attributes is
exactly what I need to solve my problem.

I'm with the Python philosophy on this one: post all the "Don't Touch
This" warning signs you like, but if somebody really wants to access my
private attributes, to do something I never even imagined, they should be
allowed to. If they break something, well, that's their fault, not mine.
I'm 100% in favour of language features which protect people from
*accidentally* shooting themselves in the foot, but if somebody wants to,
who am I to say they mustn't?

 

[Tom Plunket]

What does the author of the original class know about *my* needs and
requirements?

The only thing that the original class author might know is that mucking
with data marked private may well cause problems, and hiding it
therefore prevents those problems.

It may turn out that accessing his "private" attributes is exactly what
I need to solve my problem.

It may also turn out that you /think/ that's what you need to do, but
actually need to do something different.

E.g. it may well be possible to access the bytes on the disk that holds
your database directly, and that may be "easier" than going through the
db API, but doing so may also be highly prone to error.

I'm with the Python philosophy on this one: post all the "Don't Touch
This" warning signs you like, but if somebody really wants to access my
private attributes, to do something I never even imagined, they should be
allowed to.

You are allowed to. Have you been following the thread? Your answer's
been mentioned several times. At the same time, you should ponder very
carefully the reasons why the original author deemed it important to
make those attributes private in the first place.


-tom!

--

 

[Steven D'Aprano]

The only thing that the original class author might know is that mucking
with data marked private may well cause problems, and hiding it
therefore prevents those problems.

Sure. That's the risk I take when I muck about with somebody else's
private attributes.

It may also turn out that you /think/ that's what you need to do, but
actually need to do something different.

And how is that different from *any* programming task? I might think I
need a binary tree, when all I really need is a simple list. An error is
an error, whether it is with a public API or a private one.

E.g. it may well be possible to access the bytes on the disk that holds
your database directly, and that may be "easier" than going through the
db API, but doing so may also be highly prone to error.

No, I doubt it is "easier" -- but if I'm programming a database recovery
tool that fixes corrupted databases because the db vendor's supplied
tool is broken or non-existent, that might be exactly what I need to do.

You are allowed to. Have you been following the thread? Your answer's
been mentioned several times.

Yes, mostly by me. Do try to keep up.

At the same time, you should ponder very
carefully the reasons why the original author deemed it important to
make those attributes private in the first place.

In my experience, it is mostly because they come from
bondage-and-domination languages where it is expected that everything is
private except for a small, carefully chosen public API, rather than from
languages like Python that encourages openness and a philosophy of "we're
all adults here".

I wonder how many double-underscore "private" attributes are used in the
Python standard library? That should give you an idea of "best practice"
use of private attributes in Python.

 

[Paul Rubin]

In my experience, it is mostly because they come from
bondage-and-domination languages where it is expected that
everything is private except for a small, carefully chosen public
API, rather than from languages like Python that encourages openness
and a philosophy of "we're all adults here".

Funny thing, some of those B&D languages were designed by people who
had plenty of experience with Lisp and were tired of being bitten by
its Python-like looseness.

I wonder how many double-underscore "private" attributes are used in the
Python standard library? That should give you an idea of "best practice"
use of private attributes in Python.

If socket.py is an example of best practice use of private attributes,
I shudder to imagine what kind of cruft must lurk in legacy applications.

 

[Steven D'Aprano]

Funny thing, some of those B&D languages were designed by people who
had plenty of experience with Lisp and were tired of being bitten by
its Python-like looseness.

I never said they were wrong to do so. But those languages aren't
Python. Python is designed for those who are tied of the constant
restraint of B&D languages.

If socket.py is an example of best practice use of private attributes, I
shudder to imagine what kind of cruft must lurk in legacy applications.

socket.py doesn't have any double-underscore private attributes (at least
not in Python 2.4). So what's your point?