Is 'everything' a refrence or isn't it?

[Steven D'Aprano]

You might. Then again, you might also understand the concepts well
enough to realize that there isn't any difference between CBR and CBV
when you're passing immutable objects.

Consider this:

def do_nothing(x):
pass

huge_tuple = (None,) * 10000**4
do_nothing(huge_tuple)

If Python made a copy of huge_tuple before passing it to the function, you
would notice.

 

[Donn Cave]

We know that this will happen because it has happened time and time again
in the past. Are we incapable of learning from experience? Are we
intelligent sentient beings or do we just parrot what was said in the
past with no concern for the consequences of what we say?

The latter. I know this from the number of times I have read
that parameter passing has something to do with whether the
parameter is a mutable or immutable object.

Donn Cave, (e-mail address removed)

 

[Steven D'Aprano]

While telling them that "You can't do call by reference because Python
is call by object" may be correct,

Good to see you finally concede it.

it leaves out the critical information.

As does "call by reference" or "call by value". No three word phrase can
possibly explain Python's behaviour, or any other language for that
matter. You have to know what "call by whatever" *implies* for it to be
meaningful.

So they're liable to run into the same problem in a
different context - like the one the OP had.

That doesn't follow.

The problem isn't that talk about references is *wrong*, but that it is
misleading. Where "reference" is being used just as a generic term, it is
barely acceptable, but any time it is used in a context where it could be
misconstrued is dangerous precisely because people WILL misunderstand it,
because they *think* they know what references (assignment, call by
reference, whatever) means in this context.

The fact that call by object is unfamiliar is precisely its advantage.
Instead of people jumping to conclusions about what it means, like they do
with "reference", they will stop and think and if need be look for further
information about Python's object model. At the very least, they won't say
"Everything in Python is a reference, so this should work, but it
doesn't".

 

[Mike Meyer]

Thinking about Python's behaviour ("it always passes references to
objects") will invoke misleading frames in many programmers' minds. The
word "reference" is misleading and should be avoided, because what the
average non-Python programmer understands by the word is different from
what the experienced Pythonista understands by it.

Yes, but it's not misleading because "reference" doesn't mean what
they think it means. "reference" means the same thing in Python as it
does in C and similar languages. It's misleading because "variables"
and "assignment" in Python don't do the things they do in C. Some of
the places that this shows up is when you're dealing with call by
reference, or with references in a list.

We know that this will happen because it has happened time and time again
in the past. Are we incapable of learning from experience? Are we
intelligent sentient beings or do we just parrot what was said in the
past with no concern for the consequences of what we say?

No, we learned to say "binding" and "names". We also learned that
while giving newbies false information like "Python doesn't have
references" or "Python isn't call-by-reference" will help them get
past the immediate problem, it sets them up to be confused in some
other area where variables and assignments don't do what they
think. So we've learned to deal with the disease instead of the
symptoms, and tell them how variables in Python behave differently
from variables in C, or Pascal, or ALGOL, or FORTRAN, or whatever.

<mike

 

[Mike Meyer]

Good to see you finally concede it.

I'm not conceeding anything, because I never said otherwise. In
particular, I still claim that Pythons behavior when passing objects
as parameters is identical to call-by-reference for all the objects
that you can generate in Python.

That doesn't follow.

You're right. That observation is based on watching people repeatedly
beat their head against the difference between variable assignment in
other languages and name binding in object-based languages. Until you
get that difference into their heads, they're going to have
problems. What you use to describe the parameter passing mechanism is
immaterial to the real problem.

The problem isn't that talk about references is *wrong*, but that it is
misleading. Where "reference" is being used just as a generic term, it is
barely acceptable, but any time it is used in a context where it could be
misconstrued is dangerous precisely because people WILL misunderstand it,
because they *think* they know what references (assignment, call by
reference, whatever) means in this context.

This is where we disagree. I think their understanding of references
is dead on. What's broken is their understanding of what variables are
and what assignments mean. Once you fix that, the rest falls into
place.

The fact that call by object is unfamiliar is precisely its advantage.
Instead of people jumping to conclusions about what it means, like they do
with "reference", they will stop and think and if need be look for further
information about Python's object model. At the very least, they won't say
"Everything in Python is a reference, so this should work, but it
doesn't".

What you're advocating is intentionally misleading people to deal with
a symptom. I'd rather give them enough truth to deal with the disease.

<mike

 

[Terry Hancock]

Consider this:

def do_nothing(x):
pass

huge_tuple = (None,) * 10000**4
do_nothing(huge_tuple)

If Python made a copy of huge_tuple before passing it to
the function, you would notice.

Which succinctly demonstrates precisely why a newbie
*should* be told that Python passes references instead of
values. Without this basic knowledge, the newcomer can't
possibly be expected to make intelligent choices of
algorithm.

But it should probably also be made clear that
"reference" means a label assigned to an object, and not a
variable containing a memory location (which is what a C
"pointer" is).

The reference is usually a simple name, but it can also be
an container expression following list, dictionary,
or class instance (spam, spam[0], spam['eggs'], or
spam.ham, for example). Not tuple or string because they
are immutable, and so don't have assignable references.

Cheers,
Terry

 

[Ernst Noch]

Yes, but it's not misleading because "reference" doesn't mean what
they think it means. "reference" means the same thing in Python as it
does in C and similar languages. It's misleading because "variables"
and "assignment" in Python don't do the things they do in C. Some of
the places that this shows up is when you're dealing with call by
reference, or with references in a list.

Maybe next time showing something like the following trivial snippet
might help demonstrate that the core of the matter doesn't is not the
way python treats parameters?
if what:
param['foo'] = 'changed'
else:
temp = param['foo'] # temp is _not_ a reference!
temp = 'changed'

 

[Steven D'Aprano]

Which succinctly demonstrates precisely why a newbie
*should* be told that Python passes references instead of
values.

"I don't like Chinese food, and pizza gives me gas. Let's have a burger,"
says I.

And Terry helpfully answers: "Pizza gives you gas? Well there you go then,
that's precisely why we have to have Chinese food."

I have never suggested that we should tell newbies that Python is call
by value, and I deny that the only two possible choices for usefully
describing Python's behaviour is by CBV and CBR. The fact that Terry could
read this thread and still imagine that there is such a dichotomy is
worrying.

But it should probably also be made clear that
"reference" means a label assigned to an object, and not a
variable containing a memory location (which is what a C
"pointer" is).

The reference is usually a simple name, but it can also be
an container expression following list, dictionary,
or class instance (spam, spam[0], spam['eggs'], or
spam.ham, for example). Not tuple or string because they
are immutable, and so don't have assignable references.

Are you seriously suggesting that you can't include tuples in a list, or
pass them to functions? Because that's what it sounds like you're saying:
"Python passes references. The reference is usually a simple name, but it
can also be a container... Not tuple or string because they are immutable."

If you don't mean to say that Python can't pass strings to functions, what
do you mean to say?

I'll tell you what I say: Python passes objects to functions or
assignments.

Does this mean that the object is copied? No, I didn't say it copies
objects. I left the nature of the passing mechanism unspoken, which is how
it should be because it is an implementation detail.

Does this mean that the object can be modified in place? Certainly not,
since that depends on the object, not on the nature of Python's high-level
behaviour. It emphasises the object oriented nature of Python, and by its
very language warns the reader not to assume that Python behaves
identically to other languages. It invites further questions, rather than
lulling the reader into jumping to false conclusions.

The emphasis is on the *object*, not the passing mechanism. In C,
everything is mutable, and whether you can change an item depends on
whether you are working with a reference to that item or a copy of the
item. That's a language issue, not a data issue -- in some languages,
you can even choose whether to pass a reference to a function or a copy
of the value.

In Python, whether or not you can change an object depends on the object,
not the language itself: it is a data issue.

 

[bonono]

I'll tell you what I say: Python passes objects to functions or
assignments.

Which in C sense, is a reference(or pointer) to some opaque "table"
maintain by the system, identified by id().

 

[Steven D'Aprano]

Which in C sense, is a reference(or pointer) to some opaque "table"
maintain by the system, identified by id().

And that C sense is precisely why people try using C idioms in Python and
then are surprised that they don't behave as they would in C.

So how about we all agree to stop implying that Python behaves like C, and
try to teach people that Python behaves like Python?

 

[bonono]

And that C sense is precisely why people try using C idioms in Python and
then are surprised that they don't behave as they would in C.

So how about we all agree to stop implying that Python behaves like C, and
try to teach people that Python behaves like Python?

No one implies Python behaves like C but just trying to explain as
close as possible to people speaking that language.

"passed by object" can just as easilly be interpreted as "the function
received a seperate copy", with the caller's copy untouched.

 

[bonono]

But in programming, things do work that way. If my class Book contains a
reference to Smith's classic work, I can modify it. (Unless the language
deliberately restricts my ability to modify certain objects, as Python
does with immutable objects.)

That's what programmers expect when you talk about references, especially
if they come from a C (or Pascal) background. In Python, sometimes that's
true, and sometimes it is not, and the only way to tell is by looking at
the object itself, not by thinking about Python's high-level behaviour.

I believe one can restrict modification to pointer/reference parameter
passing to C function, so C programmer shouldn't have problem knowing
that reference doesn't not necessary mean you can modify it.

 

[Mike Meyer]

Maybe next time showing something like the following trivial snippet
might help demonstrate that the core of the matter doesn't is not the
way python treats parameters?

Did you insert an extra "doesn't" in that? If so, then I agree about
what isn't the core of the matter.

On the other hand, the snippet doesn't help without an
explanation. Especially with misleading comments.

if what:
param['foo'] = 'changed'
else:
temp = param['foo'] # temp is _not_ a reference!

Except temp *is* a reference. What it's not is a reference to
param['foo']. Instead, it's a reference to the same object that
param['foo'] is a reference to.

temp = 'changed'

<mike

 

[Ernst Noch]

Did you insert an extra "doesn't" in that? If so, then I agree about
what isn't the core of the matter.

Uhm, yes. Seems the whole message was written too hastily.

On the other hand, the snippet doesn't help without an
explanation. Especially with misleading comments.

Absolutely, it was mainly meant as an example to demonstrate in practice
that the way python passes parameters has nothing to do with what happens.

def func(param, what):

if what:
param['foo'] = 'changed'
else:
temp = param['foo'] # temp is _not_ a reference!

Except temp *is* a reference. What it's not is a reference to
param['foo']. Instead, it's a reference to the same object that
param['foo'] is a reference to.

Oh dear, yes. I inserted that comment as an afterthought.

 

[Donn Cave]

I'll tell you what I say: Python passes objects to functions or
assignments.

Does this mean that the object is copied? No, I didn't say it copies
objects. I left the nature of the passing mechanism unspoken, which is how
it should be because it is an implementation detail.

Thus conveniently deferring the necessary explanation of how
it actually works, which will not be too hard but resists
simplistic "use this word" solutions.

The emphasis is on the *object*, not the passing mechanism. In C,
everything is mutable, and whether you can change an item depends on
whether you are working with a reference to that item or a copy of the
item. That's a language issue, not a data issue -- in some languages,
you can even choose whether to pass a reference to a function or a copy
of the value.

In Python, whether or not you can change an object depends on the object,
not the language itself: it is a data issue.

Oh, baloney. Same in C -

$ cc -c /tmp/t.c
/tmp/t.c: In function 'churk':
/tmp/t.c:4: error: assignment of read-only location

As long as you pass by pointer, it is much the same -

void
churk(const char *data)
{
data[0] = 'C'; /* error - modify const object */
data = "C"; /* no problem here - rebind pointer parameter */
}

Maybe the cure for hardened C programmers who aren't
getting it is to emphasize the pointer angle - and note
that there isn't any way to write "*i = 4". "Everything
is a pointer", let's say -- or maybe, "Everything is a
reference" would be smarter.

C programmers are used to seeing their data called
objects, too. (Not to mention the unlucky many of
us who learned OOP via C++.)

Donn Cave, (e-mail address removed)

 

[Terry Hancock]

"I don't like Chinese food, and pizza gives me gas. Let's
have a burger," says I.

And Terry helpfully answers: "Pizza gives you gas? Well
there you go then, that's precisely why we have to have
Chinese food."

An annoyingly banal, off-topic, and incorrect analogy.

I have never suggested that we should tell newbies that
Python is call by value, and I deny that the only two
possible choices for usefully describing Python's
behaviour is by CBV and CBR. The fact that Terry could
read this thread and still imagine that there is such a
dichotomy is worrying.

I didn't say you did, and it's very worrying that you could
read my post and not understand that. You want to get
tetchy? I can *be* tetchy, if that's what you want. ;-)

As far as I am concerned "call by object" which is what you
*did* propose implies "call by value" to anyone who is just
learning the language, and that is the audience we're
talking about.

I mean, *I* know what you mean when you say "Python is 'call
by object'" just as *you* know what I mean when I say
"Python is 'call by reference'", but where does that leave
Mr. Newbie?

Sorry, but the burgers give him gas, too.

But it should probably also be made clear that
"reference" means a label assigned to an object, and not
a variable containing a memory location (which is what a
C "pointer" is).

The reference is usually a simple name, but it can also
be an container expression following list, dictionary,
or class instance (spam, spam[0], spam['eggs'], or
spam.ham, for example). Not tuple or string because
they are immutable, and so don't have assignable
references.

Are you seriously suggesting that you can't include tuples
in a list, or pass them to functions? Because that's what
it sounds like you're saying: "Python passes references.
The reference is usually a simple name, but it can also be
a container... Not tuple or string because they are
immutable."

If you don't mean to say that Python can't pass strings to
functions, what do you mean to say?

No, don't be intentionally thick.

I mean that you cannot dereference a string or tuple in
order to assign elements of them -- which is another way of
saying they are immutable.

I'll tell you what I say: Python passes objects to
functions or assignments.

Does this mean that the object is copied? No, I didn't say
it copies objects. I left the nature of the passing
mechanism unspoken, which is how it should be because it
is an implementation detail.

And that is where we part ways, my friend. Because
"implementation details" do matter when they make a
difference between execution in 10 seconds and executions in
10 days. Even a newbie is going to be hesitant to do things
that look like they are going to do the latter. He still
needs to have a reasonably correct mental model of what his
program is going to do.

Does this mean that the object can be modified in place?
Certainly not, since that depends on the object, not on
the nature of Python's high-level behaviour. It emphasises
the object oriented nature of Python, and by its very
language warns the reader not to assume that Python
behaves identically to other languages. It invites further
questions, rather than lulling the reader into jumping to
false conclusions.

Chigaimasu! It just leads to *different* false conclusions.

The emphasis is on the *object*, not the passing
mechanism.[...]
In Python, whether or not you can change an object depends
on the object, not the language itself: it is a data
issue.

True, but the passing mechanism still makes a difference in
understanding how Python programs operate.

Of course, this conversation is starting to get extremely
silly.

Terry

 

[Mike Meyer]

As far as I am concerned "call by object" which is what you
*did* propose implies "call by value" to anyone who is just
learning the language, and that is the audience we're
talking about.

From what I can tell, Liskov proposed *three* different names for
passing references to objects: call-by-sharing, call-by-object, and
call-by-object-reference.

Call by object is the worst choice among the three, because "object"
has such a vague meaning, so you never know what implications someone
will come away with. Of the other two, I like call by object
reference, because it's clear you're passing a reference to an object,
and it doesn't really matter what an object is in this case.

<mike

 

[Bryan Olson]

Mike Meyer wrote:
[...]

Correct. What's stored in a list is a reference.

Nonsense. What is stored in the list is an object.

According to the Python Language Reference:

Some objects contain references to other objects; these are
called containers. Examples of containers are tuples, lists
and dictionaries.
[http://docs.python.org/ref/objects.html]

Is it so hard to understand that the word "reference" has a general,
imprecise meaning in common English (which is clearly how the quote
above is using the word) while still having in the context of assignment
and argument passing a more precise meaning which is dangerously
misleading?

What's pretty easy to understand at this point, is that Mike
was right and you were wrong. "Reference" has a precise meaning
here, it's what the Python language uses, and contrary to your
reporting, it's what the rest of the discipline uses.

Words are important -- not only for what they mean, but for what the
connotations they carry. For people who come to Python from C-like
languages, the word "reference" means something that is just not true in
the context of Python's behaviour.

Wrong. C does not have references, and the Python use is consistent
with the rest of computer science. You seem to have read in things
that it does not mean. Fix *your* thinking.


That's why people come to Python with a

frame that tells that what call by reference implies ("I can do this...")
and then they discover that they often *can't* do that.

I'm sorry if you got confused, but please don't project it on
the rest of the discipline. C does not have even references.

[...]

Thinking about Python's behaviour ("it always passes references to
objects")

Just fix your thinking and don't attribute these problems to
others.

[...]

If we were writing academic papers, we could define "call by reference"
and "objects contain references" any way we liked,

That would be a terrible thing to do. Just learn to use the
meaning accepted in the discipline, and used in the Python doc.

 

[Steven D'Aprano]

Call by object is the worst choice among the three, because "object"
has such a vague meaning, so you never know what implications someone
will come away with.

So very unlike "call by reference", right?

 

[Steven D'Aprano]

Wrong. C does not have references, and the Python use is consistent
with the rest of computer science. You seem to have read in things
that it does not mean. Fix *your* thinking.

Bryan, I'll admit that I'm no C/C++ programmer, and I frequently assume
that if X is true for C++ it is also true for C, and vice versa. I
appreciate being corrected when I make a mistake.

However, in this case, I think you are confused by the term references. Of
course pure C does not have the C++ data structure known as "references",
but that is hardly the only meaning of the word. "Reference" is a generic
term for something which refers to something else, not just in plain
English but also in computer science. Pointers are references. Indexes
into a table are references. URLs are references. File names and file
handles are references, as are keys from databases, and even raw memory
addresses.

http://en.wikipedia.org/wiki/Reference_(computer_science)

I'll admit that my position on this question has changed. I no longer say
that "everything in Python is a reference" is *false*, not in the weak,
generic sense. How can it be? In the generic sense, Python names are
references. (Under the hood, names may even be implemented as C++
references for all I know.) I will still argue that the term reference is
harmful, not because it is technically incorrect, but because it activates
incorrect mental models of Python's high-level behaviour in too many
people.

I'm sorry if you got confused, but please don't project it on
the rest of the discipline.

Perhaps you should check out the beginning of the thread before making
any additional comments. It wasn't me who was confused and asked "Hey
what's going on? I was told everything in Python is a reference, but when
I do this it doesn't work." I'm not projecting anything on anyone.