Finding the instance reference of an object

[greg]

Python's behaviour is not the same as what
Pascal, or C, calls call-by-value.

Python's assignment is not the same as what Pascal or C calls
assignment, either. Yet we don't hear anyone claim that the
term "assignment" shouldn't be used in Python.

The difference between call-by-value in Python and Pascal is
exactly the same difference as there is between assignment
in Python and Pascal. Why should we throw out one term but
not the other?

 

[Steven D'Aprano]

In an attempt to keep this post from hitting the ridiculous length of one
of my posts last night, I'm going to skip over a lot of things Joe writes
that aren't critical. Just because I've skipped over a comment doesn't
mean I agree with it, merely that I don't think it gains much to argue
the point.

What would it take to convince you that Java and C have exactly the same
semantics, and differ only in syntax? Would equivalent code snippets
that do the same thing do the job?

Of course not. You would also have to demonstrate that there is nothing
you can do in C that can't be done directly in Java, and vice versa.

I say directly, because once you allow indirect techniques, you can do
all sorts of things. Here's an indirect proof that Python is "call-by-
reference":

def swap(x, y):
"""Swap the values referred to by x and y."""
x[0], y[0] = y[0], x[0]

x = [2]
y = [3]
swap(x, y)
assert x == [3] and y == [2]

See, Python is call-by-reference!!! Not.

Here's a short C snippet:

#include <stdio.h>
#include <stdlib.h>

struct record
{
int x;
int y;
int z;
};

void mutate(struct record p)
{
p.x = 0;
printf("Inside: %d %d %d\n", p.x, p.y, p.z);
}

struct record A;

int main(void)
{
A.x = 1; A.y = 1; A.z = 1;
printf("Outside: %d %d %d\n", A.x, A.y, A.z);
mutate(A);
printf("Outside: %d %d %d\n", A.x, A.y, A.z);
return 0;
}


It prints:

Outside: 1 1 1
Inside: 0 1 1
Outside: 1 1 1


Note that mutations to the struct inside the function aren't visible
outside the function. This is typical call-by-value behaviour: the
argument is copied. How do you get this behaviour in Java without relying
on "tricks" and indirect techniques as in the Python code above?

(Aside: I've learned one thing in this discussion. Despite the number of
sources I've read that claim that if you pass an array to a C function
the entire array will be copied, this does not appear to be true. Perhaps
C is more like Java than I thought. Or perhaps my C coding skills are
even more pathetic than I thought.)


[...]

I've stated repeatedly that this is fine shorthand. You only get into
trouble when, instead of assigning 1, you are assigning (a reference
to) some mutable object. Then you have to think about whether you are
copying the data or just copying a reference to it.

Please explain the nature of this "trouble" that you describe,
specifically in the context of Python.

If you understand the Python model (call-by-sharing, assignment is
binding to names and not an operation on objects) then there is no such
trouble *unless* you insist on interpreting things otherwise. You have a
choice: give up the commonsense meaning of the word "value" merely to
allow you to claim Python is "call-by-value" (where value is a pointer to
the value you care about). Or you can keep the commonsense definition of
"value" (that which a symbol represents) and stop claiming that Python is
call-by-value.


[...]

I'm not changing the definition of "value," I'm merely being precise
about it.

And yet you keep needing to say "where the value is a reference". The
fact that you need to do this proves that what you are doing is
surprising to people.

If I want somebody to put a book on top of a box sitting on a table, I
don't feel the need to say "Put the book on top of the box, where the top
is the part of the box pointing towards the sky". That's understood, from
the ordinary meaning of the word. But you say "Put the book on top of the
box, where the top is the part of the box in contact with the table.",
and then argue black and blue that for primitive boxes, the top points to
the sky but for all the other boxes, the top points to the ground. You
only need to give that explanation because you're changing the definition.

When I execute x=1 and then say the value of x is 1, I don't need to
explain what I mean by "value" because I'm using the general meaning of
the word: that which a symbol represents. In Python code, the symbol x
represents the object 1. It doesn't matter whether 1 is a primitive or an
object, or whether 1 is mutable or immutable.

But you insist on an extra layer of indirection, because you want to talk
about an implementation detail and give "value" a non-standard meaning:
"value", to you, is a reference to the thing which the symbol represents.
But only for certain things. For other things, the value is the thing
itself.

This isn't precision, it's obfuscation, partly because it complicates the
meaning of "value", but more importantly because you're not talking at
the relevant level any more. You're not discussing the Python object
model, or the behaviour of Python code, you're discussing one specific
implementation of that code. At the level of Python code, x represents
the object 1. At the implementation level, sure, the C code that
implements function calling and similar operates by passing pointers, or
references if you prefer. I've got no problem with that terminology if we
are talking about the implementation. That's precisely what the C code
does.

But that's not what the Python code does.

Here's an analogy: consider this function:

def sort(A): A.sort()

Suppose we looked it up in one reference manual and read this:

sort(A): takes one argument, a reference to a list, and assigns that
reference to A. Dereferences the variable A and sorts the references in
the list by the dereferenced value of the items in the list.


Now look it up in another reference manual:

sort(A): takes one argument, a list, and assigns that list to A. Sorts
the list A by the items in the list.

(Of course "Sorts list A" would be even more concise, but less explicit.)

You're trying to tell me that at the level of Python code, the first
description is appropriate, and in fact better than the second. I would
argue that the first description is only appropriate when dealing with
the specific implementation, not at the level of Python code.

You want to be loose about it -- and keep trying to support
that practice by citing examples where such looseness works fine --
but when dealing with mutable types, it is NOT fine, and leads people
into trouble.

Of course it is. Python's calling behaviour and assignment behaviour is
identical regardless of whether the objects are mutable or immutable.

If the "value" of x is a person named Sam of species Hobbit, then

y = x
y.species = 'Elf'

would not change Sam into an elf. But in Python, it does.

This explanation simply obfuscates things more than it clarifies. What is
a "person"? Is it a Python class or a semantic "kind"? How is it defined?
What determines the "species"? Is it merely a label? If so, then changing
that label absolutely changes the species, and I don't see why you say
that changing the label "would not change Sam into an elf". Of course you
have: y is the same object as x, namely Sam, so when you mutate y, you
mutate x.

At this
point, you will launch into your explanation of how, not only is
Python's calling convention different from other languages, but its
assignment operator is quite different too, so the above doesn't do
what you would expect it to do when dealing with object values.

You've said that "in Python, it does." Are you claiming that in Java it
doesn't? Then how come you've repeatedly said that Python's assignment
and calling conventions are exactly the same as Java?

I think your argument here is desperately incoherent.


[...]

Where, in the LISP community?

Is that supposed to be an insult?

Why can't I find this venerated name in any of my CS references?

Maybe you have shoddy CS references that are overly-influenced by Java.
Maybe your references are written by people with a lousy sense of
history. Maybe they care more about pigeon-holing all languages into the
minimum number of "call-by-foo" than they care about keeping the
distinction between a language behaviour and it's implementation
behaviour. I don't know, there could be all sorts of reasons.

No, they've realized that no new term is needed. "int foo;" declares
an integer. "Person foo;" declares a person reference.

But that's not what the code says. The code says "declare a Person foo".
You have to *interpret* the code differently depending on what Person is:
if it is a primitive type, then you use the straightforward what-you-see-
is-what-you-get interpretation:

"foo x;" declares a foo called x, and the value of x is a foo.

But if foo is not a primitive type, then you need a *second*
interpretation:

"foo x;" declares a (reference to a foo) called x, and the value of x is
a reference to a foo.

The exact same statement can mean two different things, one of which uses
the natural dictionary meaning of "value" and one of which requires an
artificial redefinition of the word. You yourself have admitted that to
describe the value of x in terms of references is obtuse and obfuscatory.
And yet here you are now, defending it as "mere streamlining".


[...]

But here you go again: you're forced to claim that Python's parameter
passing is different, AND its assignment is different, with the net
result that the behavior is *exactly the same* as Java, .NET, and so
on.

"Forced"??? Why would I be forced? This is what we've been saying all
along: if you think Python uses the same calling conventions as C or
Pascal, you are wrong.

As for Java etc, I've never suggested that Python's semantics are
different from them. I've said that they are wrong and foolish to use
Pascal/C terminology to describe behaviour which is different from Pascal
and C behaviour, since that insistence requires redefining simple words
like "value" to mean TWO things, one of which is radically different from
the normal meaning of the word.

 

[Steven D'Aprano]

In Python, AFAICT, there is only one type, the object reference. So,
the type of every variable is 'reference', and each one contains a
reference.

This is wrong. If we take "variable" to mean "name", then Python names do
not have types. But *objects* have types, and there are many of them:

a = 23; type(a)

a = "foo"; type(a)

a = []; type(a)

<type 'list'>

But a name that isn't bound to an object doesn't have a type:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined

The type information is associated with the object, not with the name.

It is possible that, in the C implementation, there is a C-type
'reference' and all(?) C variables relating to the implementation of
namespaces have that type. Possibly. But even if true, that's the wrong
level of description.

 

[Steven D'Aprano]

Python's assignment is not the same as what Pascal or C calls
assignment, either. Yet we don't hear anyone claim that the term
"assignment" shouldn't be used in Python.

Er, you're not from around here are you?

I admit that sometimes I slip into bad habits, but generally the accepted
terminology is that x = 1 binds the object 1 to the name x.

The difference between call-by-value in Python and Pascal is exactly the
same difference as there is between assignment in Python and Pascal. Why
should we throw out one term but not the other?

Exactly.

 

[greg]

You have said the value that is copied is a pointer to the object.

This assumes that "call by value" means "call by copying
the value".

That assumption is WRONG.

It doesn't mean that. It means "call by ASSIGNING the
value."

So, you can think of the value of an expression as being
an object, as opposed to a reference to an object, if
you want. But then you need to consider what it means
to assign that value to a name.

Obviously it doesn't mean copying the value -- it must
mean making the name refer to the value somehow.

The same thing happens when the value is passed to a
function.

 

[greg]

'Pass by value' is not relevant to Python as variables do not contain
anything.

Where abouts in the phrase "pass by value" does the word
"contain" appear?

You don't need a notion of containment in order for
"pass by value" to have meaning. All you need is some
notion of a "value" (it doesn't matter what) and
some way to "pass" that value.

'Pass by reference' is not relevant to Python as the language
doesn't have the concept of object reference (in the sense of e.g. C++
reference).

What it doesn't have is the concept of a *variable*
reference, which is what the "reference" in "pass by
reference" means.

Here lies, IMHO, the reason why you think you need Python to 'pass by
value'. As you believe that variables must contain something, you think
that assignment is about copying the content of a variable. Assignment
in Python is simply giving a new name to an object.

Yes, and so is passing by value!

 

[greg]

Here's where we depart, I guess. I think there's no such thing (see
<http://en.wikipedia.org/wiki/Evaluation_strategy > for example, and the
dead-tree references I have on hand agree).

Something has just occurred to me. If you take the
view that the value of an expression is an object,
then the terms "value" and "object" are synonymous.

So if you then insist that Python uses "call by object",
you're actually saying it uses call by value!

 

[Steven D'Aprano]

This assumes that "call by value" means "call by copying the value".

That assumption is WRONG.

Not according to my Comp Sci lecturers back in the day, and not according
to my Pascal books.

E.g. "Programming In Pascal", 2nd Edition (1985), by Peter Grogono of
Concordia University refers to "value" parameters and "variable"
parameters to refer to the standard Pascal call-by-value convention and
the call-by-reference convention you get when you declare a parameter
with the var keyword. He writes:

"When an object is passed to a procedure by value, a local copy of it is
made. If the object is a large array, the copying operation will make the
program slower and will increase its memory requirements."

This was written as an undergraduate textbook. Remember that in 1985, OOP
was far more exotic than now, and object in the above means any Pascal
type (integer, array, set etc.) and not object as we understand it today.

Grogono doesn't explicitly use the terms "call-by-value" and "call-by-
reference", but my lecture notes from the time make it clear that the
Melbourne University Comp Sci department understood those terms to imply
Pascal calling semantics.

It doesn't mean that. It means "call by ASSIGNING the value."

Which, in languages like Pascal, means COPYING the value. And that leads
to confusion when people who understand C-B-V to mean what Pascal means
by the term hear that Python is C-B-V.

So, you can think of the value of an expression as being an object, as
opposed to a reference to an object, if you want.

That's the natural interpretation based on the ordinary meaning of the
word "value". In a language like Python that doesn't have references,
it's the only sensible interpretation. Otherwise you're forced to argue
that following a statement x=1, the value of x is something that has no
existence in Python code.

But then you need to
consider what it means to assign that value to a name.

But you need to do that anyway.

Here's a simple C program:

struct Rec
{
int x;
};

struct Rec a;
struct Rec b;

int main(void)
{
a.x = 1;
b = a;
printf("Before: %d %d\n", a.x, b.x);
a.x += 1;
printf("After: %d %d\n", a.x, b.x);
return 0;
}


It prints:

Before: 1 1
After: 2 1



Here's a simple Python equivalent:

class Rec:
pass

a = Rec()
a.x = 1
b = a
print "Before: %d %d" % (a.x, b.x)
a.x += 1
print "After: %d %d" % (a.x, b.x)



It prints:


Before: 1 1
After: 2 2



Anyone want to argue that assignment in Python is the same as in C?

Obviously it doesn't mean copying the value -- it must mean making the
name refer to the value somehow.

There's no "obviously" about it. To anyone who has learned that "call-by-
value" means that a copy is made, "obviously" it does mean copying the
value. If you have learned a different meaning, then you will believe
differently.

The same thing happens when the value is passed to a function.

Sure. Calling conventions in Python are the same as name/value binding,
except that you have a slight extra complication due to having two
namespaces instead of one.

 

[Steven D'Aprano]

Something has just occurred to me. If you take the view that the value
of an expression is an object, then the terms "value" and "object" are
synonymous.

So far so good.

So if you then insist that Python uses "call by object", you're actually
saying it uses call by value!

Fail!

The terms "guy" and "man" are synonymous, but a wise guy and a wise man
are not.

 

[Arnaud Delobelle]

Where abouts in the phrase "pass by value" does the word
"contain" appear?

You don't quote enough context for it to appear.

You don't need a notion of containment in order for
"pass by value" to have meaning. All you need is some
notion of a "value" (it doesn't matter what) and
some way to "pass" that value.


What it doesn't have is the concept of a *variable*
reference, which is what the "reference" in "pass by
reference" means.

What's a variable reference?

Yes, and so is passing by value!

What you're saying is that in the code below, when foo(q) is called
then 'p' in foo is another name for q in main. Right?

struct point {
int x;
int y;
}

int foo(point p) {
p.x = 42;
}

int main() {
point q = {0, 0};
foo(q);
/* So now you're saying that q.x == 0 ? */
}

 

[Joe Strout]

Therefore objects don't need names to exist. Having a name is
sufficient but not necessary to exist. Being in a container is
neither necessary -nor- sufficient.

What do you mean? Being in a container isn't necessary, but it
certainly is sufficient.

Having ANY references to it is both necessary and sufficient to
exist. And isn't that the easiest way to say it?

Best,
- Joe

 

[Aaron Brady]

What do you mean?  Being in a container isn't necessary, but it  
certainly is sufficient.

Having ANY references to it is both necessary and sufficient to  
exist.  And isn't that the easiest way to say it?

No, you forgot about cyclically-linked garbage, which Python
implementations are allowed to, but not required to, collect.

 

[Aaron Brady]

<Quote in favor of Steven snip>

There's no "obviously" about it. To anyone who has learned that "call-by-
value" means that a copy is made, "obviously" it does mean copying the
value. If you have learned a different meaning, then you will believe
differently.

I don't think it's obvious to everyone what a copy constructor is and
when it's called. That's ok, it's something you can learn. I think
Joe's idea is that you can think of every variable in Python as a
pointer, and that clears up some confusions about its variable model.
What happens when a pointer is copied? What is an example of copying
a pointer in spoken language?

a= [ 1, 2, 3 ]

Are the following true?

The value of 'a' is an object.
The value of that object is [ 1, 2, 3 ].
The value of 'a' is [ 1, 2, 3 ].

If so, 'value' is ambiguous and therefore not very useful as a term.

 

[Terry Reedy]

Joe Strout wrote:

Something has just occurred to me. If you take the
view that the value of an expression is an object,
then the terms "value" and "object" are synonymous.

Nope. The result of an expression is an object with an id, class, and
'value', where 'value' can include attributes (other than class) or
non-attribute content or both. The distinction does not matter much for
immutables but it definitely does for mutable objects.

So if you then insist that Python uses "call by object",
you're actually saying it uses call by value!

Both Joe and you seem to be engaging in the following bit of sophistry:

"In order for code A to call code B, some information must be
communicated from A to B." Something we all know ...

"That information is a value of some sort." True...

"Therefore all calling is calling by value."

Well, yes, if you insist. *But*, a distinction that does not
distinguish is useless. One might as well say "Calling is calling."

In order to successfully program in Python, one must understand how
functions are called. That 'how' is distinct from the 'how' of some
other languages. Therefore, some of us claim, that 'how' should have a
distinct name.

Terry Jan Reedy

 

[Terry Reedy]

In an attempt to keep this post from hitting the ridiculous length of one

(Aside: I've learned one thing in this discussion. Despite the number of
sources I've read that claim that if you pass an array to a C function
the entire array will be copied, this does not appear to be true....)

Since C does not have an array type, it is impossible to pass an array.
int *a, b[10] declares *both* a and b as int pointers. As I remember,
the only difference is that b is initialized to the address of an
allocated block of 10. In expressions, b is an int pointer, just like
a, and a and b are defined the same, as *(a+i) and *(b+i), where
the addition is pointer arithmetic. Function definitions can only define
parameters as pointers (and lots else), not arrays. So passing either a
or b passes the address it represents. (Function names also represent
addresses in expressions, whereas, I believe, in C99 at least, struct
names represent the struc, not its address.)

tjr

 

[Aaron Brady]

"In order for code A to call code B, some information must be
communicated from A to B."  Something we all know ...

"That information is a value of some sort." True...

"Therefore all calling is calling by value."

Well, yes, if you insist.  *But*, a distinction that does not
distinguish is useless.  One might as well say "Calling is calling."

Actually I like that. We should call all call methods call by
calling, because it is a useful and consistent generalization.

 

[Joe Strout]

Both Joe and you seem to be engaging in the following bit of
sophistry:

"In order for code A to call code B, some information must be
communicated from A to B." Something we all know ...

"That information is a value of some sort." True...

"Therefore all calling is calling by value."

Hey now, hang on. That doesn't characterize my argument at all. My
argument is much simpler:

Python's call semantics are exactly the same as other modern OOP
languages, where it is commonly called call-by-value (and it is, in
fact, call-by-value, as you can see with the simple test of assigning
to the formal parameter). Therefore, we would reduce confusion if
we'd get on board and call it that in Python, too.

Of course, I've softened my position somewhat, since being shown that
"call by sharing" is simply a term for call-by-value in the case where
the values are object references. That clearly does apply to Python
(as well as other OOP languages, where object references are
involved), and as long as we can explain it as just a restricted form
of call-by-value, I'm OK with that.

In order to successfully program in Python, one must understand how
functions are called. That 'how' is distinct from the 'how' of some
other languages.

Which languages? Certainly none of those at <http://www.strout.net/info/coding/valref/

>. (Except that some of them have a by-reference option, which

Python and Java do not.)

Best,
- Joe

 

[Aaron Brady]

Of course, I've softened my position somewhat, since being shown that  
"call by sharing" is simply a term for call-by-value in the case where  
the values are object references.  That clearly does apply to Python  
(as well as other OOP languages, where object references are  
involved), and as long as we can explain it as just a restricted form  
of call-by-value, I'm OK with that.

Humor me briefly. In 'x= [1, 2, 3]', do you think that the value of x
is a memory address?

 

[greg]

Not according to my Comp Sci lecturers back in the day, and not according
to my Pascal books.

Pascal books will tell you what call-by-value means
in Pascal. You can't just blindly take that description
and apply it to other languages, just as you can't
take what your Pascal book says about assigment and
apply it blindly to Python without getting into
trouble.

Describing call-by-value as "copying the value" works
in Pascal, because "assignment" and "copy" are synonyms
in that language. But they're not synonyms in all
languages, so a language-independent definition of
call-by-value needs to describe it in a more general
way.

Refusing to accept that a more general definition
exists just because it's not mentioned in your Pascal
book is a little absurd.

 

[greg]

What's a variable reference?

It's a reference to a variable. It's what gets passed behind
the scenes when you use a VAR parameter in Pascal, or a
ByRef parameter in VB.

What you're saying is that in the code below, when foo(q) is called
then 'p' in foo is another name for q in main. Right?

struct point {
int x;
int y;
}

int foo(point p) {
p.x = 42;
}

int main() {
point q = {0, 0};
foo(q);
/* So now you're saying that q.x == 0 ? */
}

No. Passing q by value means that the value of the expression 'q',
whatever that is in the language concerned, gets assigned to the
local variable 'p', whatever *that* means in the language concerned.

Because of the way C assignment works, the result is that p ends
up with a copy of the whole struct.

Because of the way Python assignment works, the result is that
p and q end up referring to the same object.

The difference is *entirely* due to the difference in the semantics
of assignment between the two languages. Once you've taken that
into account, there is no need to look for difference in the
parameter passing scheme.