Finding the instance reference of an object

J

Joe Strout

No, a "by ref" parameter would mean that this:

def foo(ByRef x):
x = x + [42]

a = [1,2,3]
foo(a)

...would result in a = [1,2,3,42].
In [8]: def foo(x):
...: x += [42]
...:

In [9]: a = [1, 2, 3]

In [10]: foo(a)

In [11]: a
Out[11]: [1, 2, 3, 42]

This demonstrates that lists are mutable, and that += is a mutating
operator (and NOT an assignment).
In [12]: def ffoo(x):
....: x.append(43)
....:

In [13]: ffoo(a)

In [14]: a
Out[14]: [1, 2, 3, 42, 43]

Ditto (but for the .append method).
In [15]: def fffoo(a):
....: a = a + [42]
....:

In [16]: fffoo(a)

In [17]: a
Out[17]: [1, 2, 3, 42, 43]

And here, you're doing an assignment -- this is the only test of the
three that tests whether the parameter is passed by reference or by
value. The result: it's by value.
So, is it call by reference or not?
Not.

Does that depend on the implementation of specific operators?
No.

You problem seems to be that you ar still stuck with the notion of a
name as a reference to a specific area of memory. Which it's not,
excepting only if you want to consider the area of memory that holds a
reference to some value.

Which is exactly what it is, so that's what you should consider.

And my real point is that this is exactly the same as in every other
modern language. Nothing unusual here at all (except that some of us
here seem to want to make up new terminology for standard behavior,
perhaps in order to make Python seem more exotic).
In the case of lists,

a = a + [something]

rebinds a

In standard terminology, it assigns a new value to a.
while

a += [something]

doesn't.
Correct.

So where does that leave you?

In exactly the same happy boat as Java, RB, .NET, etc. (even C++ if
you consider an object pointer to be the equivalent of an object
reference).
Or, use of one of the compound operators like +=. That's the only
real
"gotcha" in Python to somebody coming from another language; you
might
naively expect that x += y is the same as x = x+y, but in Python
this is
not generally the case; instead += is a mutation operator, like the
examples you show above.
Be careful though:

In [18]: a = 42

In [19]: id(a)
Out[19]: 14237932

In [20]: a += 1

In [21]: id(a)
Out[21]: 14237920

Good point -- obviously += can't mutate an immutable type. In those
cases it IS equivalent to an assignment. I can certainly see why
these operators can trip people up at first.
If you mean it's a reference assigment, of course it is. That's what
he
was trying to say (I believe). But in C, for example,

int i;
i = 42;

actually stores the value 42 in the location associated with the
name c.

Yes, but let's get away from numbers, since those are a bit of a
special case, and not where the argument is revolving. (Since
Python's number-wrappers are immutable, they are behaviorally
equivalent to raw values, and so it really doesn't matter whether you
know that they're actually objects or not).

The discussion at hand is how best to explain and understand mutable
types. I don't remember C too well, but in C++ that'd be something
like:

Foo *x;
x = FooFactory();

This stores the address of the object FooFactory builds into x. It's
equivalent to what Python does with

x = FooFactory()

....except, of course, that Python's syntax is cleaner, and you don't
have all the rope that C/C++ gives you with which to shoot yourself in
the foot.
Hey, I remember Pascal... that was the language used on the Apple
IIGS,
back when I was in junior high. I also remember spending $800 for a
40MB hard drive for it. Ah, those were the days!
40 Mb! You were lucky! Etc., etc., [drools into beard.]
:)

Python's assignment semantics (as opposed to its "object handling, a
term for which I have no referent) are not the same as those of, say
C.

They are, though. The only difference you've pointed out is that
*numbers* are different in Python vs. C, and that's an internal
implementation detail I was blissfully unaware of until this
discussion. (I'm grateful to know it, but it really doesn't matter in
day-to-day coding.)
I believe they are pretty much the same ass those of Icon, another
non-standard interpreted language.

They're also the same as RB, Java, .NET... I'm hoping somebody who
knows some other modern OOP language (e.g. Ruby) will weigh in,
because I bet it's the same as those too.

Comparing it to C isn't really fair, because C isn't even an OOP
language. C++ would at least be in the same ballpark.
There are close similarities between Python's names and C reference
variables, but the semantics are not exactly parallel.

I agree; reference variables are an odd beast, most commonly used (and
most similar to) a ByRef parameter in modern languages. And it seems
that Python simply doesn't have that.
People here don't describe Python as different just because they
*want*
it to be different. Python acknowledges intellectual debts to many
languages, none of which is exactly like it.

Sure, nothing's exactly like it in all ways. But its object handling
[*] is, in fact, exactly like every other modern OOP language.

Best,
- Joe

[*] Object handling: the ways in which objects are created, stored,
referenced, assigned, and passed into and out of method calls.
 
J

Joe Strout

No, it isn't. In many other languages (C, Pascal, etc.), a
"variable" is commonly thought of as a fixed location in memory
into which one can put values. Those values may be references
to objects.

Right, though not in languages like C and Pascal that don't HAVE the
notion of objects. We really ought to stop bringing up those
dinosaurs and instead compare Python to any modern OOP language.
In Python, that's not how it works. There is no
"location in memory" that corresponds to a variable with a
particular name the way there is in C or Pascal or Fortran or
many other languages.

No? Is there any way to prove that, without delving into the Python
source itself?

If not, then I think you're talking about an internal implementation
detail.
All that exists in Python is a name->object mapping.

And what does that name->object mapping consist of? At some level,
there has to be a memory location that stores the reference to the
object, right?
That's because it is fundamentally different from what happens
in languages like C.

What happens in a modern OOP language is just as fundamentally
different (which is to say, not really very much) from what happens in
C or FORTRAN or COBOL, too.

But if there's any demonstrable difference between Python and any
other modern OOP language, I'd love to hear about it.
a = a + [something]

rebinds a

In standard terminology, it assigns a new value to a.

The problem is that listeners from a C or FORTRAN background
will infer that there is a fixed region of memory named "a" and
"assigning a value to a" means writing that new value into the
region of memory named "a".

And they'd be correct. The value being written, in this case, as a
reference to some data that lives somewhere on the heap. Point that
out, and now their understanding is correct.

But if you instead say something like "all parameters are passed by
reference in Python," then the user gets the wrong idea. That
statement has a specific meaning which is, quite demonstrably, not true.

If you make up new terms like "rebinds," well, I guess at least that
avoids giving the listener the wrong idea. Instead it gives them no
idea at all, which may be better, but not as good as giving them the
right idea. It still leaves them to investigate what actually
happens, and ultimately they will find that the parameters are always
passed by value in Python. May as well save them the trouble and
point that out up front.

And how many recovering FORTRAN or C programmers do we get around
here, anyway? Java is what they've been teaching in school for the
last several years, and it was C++ for a good decade before that. The
semantics of Python (insofar as objects, assignments, and parameters
are concerned) are exactly the same as Java, and Java is just a
cleaned-up and streamlined C++. Even old-timers (like me) who learned
FORTRAN and COBOL way back in the day have long since moved on.

Best,
- Joe
 
A

Aaron \Castironpi\ Brady

On Oct 17, 2008, at 2:36 PM, Steve Holden wrote: snip
And here, you're doing an assignment -- this is the only test of the  
three that tests whether the parameter is passed by reference or by  
value.  The result: it's by value.


Not.

But it's not by value, q.e.d.
snip
...except, of course, that Python's syntax is cleaner...

Excepting that thou then proceedst to 3.

snip
 
G

George Sakkis

^^^^^^^^^
No, it isn't. In many other languages (C, Pascal, etc.), a

^^^^^^^^^^^^^^^^^
Methinks the only real disagreement in this thread is what's a
"modern" language; Joe has made clear that he's not comparing Python
to C, Pascal or Fortran.

George
 
M

MRAB

On Oct 17, 2008, at 2:36 PM, Steve Holden wrote:
[big snip]
I believe they are pretty much the same ass those of Icon, another
non-standard interpreted language.

They're also the same as RB, Java, .NET... I'm hoping somebody who  
knows some other modern OOP language (e.g. Ruby) will weigh in,  
because I bet it's the same as those too.
[snip]
The difference is that Python uses dictionaries. When you write:

x = 42

what actually happens is something like:

globals()["x"] = 42

or:

locals()["x"] = 42

The name of the "variable" is a key in a dictionary with the value (a
reference to an object) that was "assigned" being the associated
value. When you "del" a "variable", eg del x, you're removing the
entry from the dictionary.

In Java the variables are like members in a struct (or fields in a
record in Pascal parlance) with the value that was assigned being its
content. (In Java there's also the detail that primitive types like
int are stored directly and not as references for reasons of speed.)
 
S

Steven D'Aprano

Right, though not in languages like C and Pascal that don't HAVE the
notion of objects. We really ought to stop bringing up those dinosaurs
and instead compare Python to any modern OOP language.

You really should stop assuming that languages still in use are
"dinosaurs". There are many, many people whose understanding of such
terms as "call by value" or "variable" are based on C or Pascal or even
Fortran.

Besides, while the Pascal language itself doesn't have objects, plenty of
Pascal implementations do.

No? Is there any way to prove that, without delving into the Python
source itself?
135536420

Notice that the address of x changes. Actually, that's a lie. The name
'x' doesn't have an address, except in the sense that the string 'x' must
occur somewhere in memory. But that's an unimportant implementation
detail. In reality, what we see is:

the object 1 is bound to the name 'x';
we ask for the id() of the object bound to 'x' (1), which is 135536432;
the object 2 is bound to the name 'x';
we ask for the id() of the object bound to 'x' (2), which is 135536420.

That's what is happening at the Python level of code. Anything else is
just implementation details.


If not, then I think you're talking about an internal implementation
detail.

Namespaces aren't an internal implementation detail, they are fundamental
to how Python works.

And what does that name->object mapping consist of? At some level,
there has to be a memory location that stores the reference to the
object, right?

Who cares? That's the *wrong* level. At *some* level, everything is just
flipping electrons from one state to another, but I think even you would
object if I claimed that:

"All computer languages are call-by-flipping-electrons, with no
exceptions."


Somebody could implement a Python interpreter using literal post-it notes
for names and literal boxes for objects. The fact that our CPUs are
optimized for flipping bits instead of (say) water flowing through tubes,
or clockwork, or punching holes in paper tape, is irrelevant. When
discussing what Python code does, the right level is to discuss the
Python virtual machine, not the implementation of that VM.

If you want to discuss the deeper levels, it's perfectly acceptable to
say that the CPython implementation uses memory locations holding
pointers. But that's not *Python*, that's hidden detail that the Python
programmer can't reach.


What happens in a modern OOP language is just as fundamentally different
(which is to say, not really very much) from what happens in C or
FORTRAN or COBOL, too.

Which is why modern OOP languages like Python shouldn't use the same
terminology invented to describe what Fortran and Pascal do.

If you make up new terms like "rebinds,"

This is not something that is just "made up", it is an actual description
of what the Python VM does.

well, I guess at least that
avoids giving the listener the wrong idea.

As opposed to call-by-value, call-by-reference, and "call-by-value-where-
the-value-is-the-reference", all of which do give the listener the wrong
idea.

Instead it gives them no
idea at all, which may be better, but not as good as giving them the
right idea.

But it is the right idea. They just don't know what it means, because
they've been listening to people like you who insist on using Pascal
terminology with a definition unrecognizable to Pascal programmers. To an
ex-Pascal programmer like myself, when you talk about "call by value
where the value is a reference", it sounds to me as if you are insisting
that cars are ACTUALLY horse and buggies, where the horse is the engine,
why are we inventing new terms like 'automobile', that just confuses
people.


It still leaves them to investigate what actually happens,
and ultimately they will find that the parameters are always passed by
value in Python.

But they aren't. When you call a function, the arguments are not copied
before being passed to the function.


May as well save them the trouble and point that out
up front.

Well, sure, if you want to be misleading and confusing.


And how many recovering FORTRAN or C programmers do we get around here,
anyway?

Enough.

It's not just people who have programmed Pascal, but those whose
understanding of CBV and CBR have been shaped by ancient half-remembered
Pascal lessons.

Java is what they've been teaching in school for the last
several years, and it was C++ for a good decade before that.

Have they really? Which school is that?

My company has hired a part-time young gun programmer. He's still at Uni,
doing Comp Sci, and has *far* wider IT experience than the average
programmer his age. His skills? PHP, Perl, C, oh and he's just starting
to learn C++ this semester.

There are plenty of programmers who have never been to university or
collage, and even some who have never learnt programming in school at
all. Believe it or not, it's not compulsory.
 
S

Steven D'Aprano

People here don't describe Python as different just because they *want*
it to be different. Python acknowledges intellectual debts to many
languages, none of which is exactly like it.

I understand that Python's object and calling semantics are exactly the
same as Emerald (and likely other languages as well), and that both
Emerald and Python are explicitly based on those of CLU, as described by
by Barbara Liskov in 1979:

"In particular it is not call by value because mutations
of arguments performed by the called routine will be
visible to the caller. And it is not call by reference
because access is not given to the variables of the
caller, but merely to certain objects."

http://www.lcs.mit.edu/publications/pubs/pdf/MIT-LCS-TR-225.pdf

quoted by Fredrik Lundh here:
http://mail.python.org/pipermail/python-list/2003-May/204379.html


"Call by object/sharing" isn't some new-fangled affectation invented by
comp.lang.python dweebs to make Python seem edgy and different. It's a
term that has been in use in highly respected Comp Sci circles for over
thirty years. In case anybody doesn't recognise the name:

http://en.wikipedia.org/wiki/Barbara_Liskov
 
S

Steven D'Aprano

The reference to the object IS the value of an object reference
variable.

That's a bizarre and unnatural way of looking at it. To steal a line from
the effbot, that's like valuing your child's Social Security number over
the child herself:

http://mail.python.org/pipermail/python-list/2003-May/204560.html


If we execute a line of Python code:

x = "parrot"

and then ask "What's the value of x?", I think that even you would think
I was being deliberately obtuse, difficult and obfuscatory if I answered
"location 0xb7cdeb2c".

So good, parameters are passed ByVal in Python as they appear
to be, and as is the default in every other modern language.

Nonsense. Python doesn't copy a parameter before passing it to the
function. You get the same parameter inside the function as outside:
.... print id(x)
....
a = ['some', 'thing']
print id(a); foo(a)
3083441036
3083441036


I'm going to anticipate your response here: you're going to deny that
call by value implies that the list ['some', 'thing'] will be copied
before being passed to the function. According to *some* definitions of
CBV, you might even be right. But according to *other* definitions,
including the one that I learned in comp sci at university, that copying
of data is an essential part of CBV.

These other definitions aren't necessarily a formal definition from some
Computer Scientist. They're just as likely to be informal understandings
of what CBV and CBR mean: "if it's call by value, don't pass big data
structures because they will be copied and your code will be slow".

Assignment copies the RHS value to the LHS variable. In the case of an
object reference, the value copied is, er, an object reference.

No, assignment binds an object to a name. That's what Python does.

Of course, at the implementation level, name binding might be implemented
by copying object references. Or it might not. That's an implementation
detail that isn't relevant at the Python level.

Or at least, it shouldn't be relevant until the abstraction leaks.
http://www.joelonsoftware.com/articles/LeakyAbstractions.html



[snip]
For object references (including the mutable ones that may treat people
up), Python's behavior is no different from any other language.

That's an exceedingly broad claim. No different from Java? Well, perhaps.
No different from Lisp? Doubtful. No different from Forth? Yeah, riiight.

Speaking of Java, there's one major difference between Java and Python
with respect to names. In a statically typed language like Java, you
define names before you use them, and from that point the name is bound
to both a type and an object. But the binding to the object is optional,
and such unbound names are said to be null.

In a dynamically typed language like Python, names are bound only to
objects. You can't have an unbound name: if a name exists, it must be
bound to an object, and if it doesn't exist, you get a NameError
exception when you try to access it. And objects are typed, not names.

http://www.ferg.org/projects/python_java_side-by-side.html



Thanks for that explanation. Clearly that's not what's going on in
Python.

Ah no, that's my bad. I have a strange and disturbing mental stutter that
substitutes "call by name" when I mean to say "call by object" at the
most embarrassing times. Sorry.
 
S

Steve Holden

Steven D'Aprano wrote:
[...]
when you talk about "call by value
where the value is a reference", it sounds to me as if you are insisting
that cars are ACTUALLY horse and buggies, where the horse is the engine,
why are we inventing new terms like 'automobile', that just confuses
people.
+1 QOTW
 
S

Steve Holden

Steven D'Aprano wrote:
[...]
when you talk about "call by value
where the value is a reference", it sounds to me as if you are insisting
that cars are ACTUALLY horse and buggies, where the horse is the engine,
why are we inventing new terms like 'automobile', that just confuses
people.
+1 QOTW
 
F

Fuzzyman

Right, though not in languages like C and Pascal that don't HAVE the  
notion of objects.  We really ought to stop bringing up those  
dinosaurs and instead compare Python to any modern OOP language.


No?  Is there any way to prove that, without delving into the Python  
source itself?


..NET makes a clear distinction between 'value types' and reference
types. These are analagous to the call by value and call by reference
that have been discussed in this thread.

My understanding of the difference is that value types (all numeric
types, structs, enumerations etc) actually live on the stack, whereas
reference types (strings, classes etc) live in the heap and have a
pointer on the stack.

It is complicated by the fact that .NET perpetuates the myth that the
primitives (the value types) inherit from System.Object (a reference
type). The runtime does auto-boxing for you where this matters.

This means that when you pass a value type into a method the value
*is* copied. Structs can be arbitrarily big, so this can be a
performance problem. It is often a performance win for working with
the numeric types as you remove a level of indirection.

It isn't without problems though - and some of these can be seen in
IronPython.

System.Drawing.Point is a struct, but it is effectively a mutable
value type. However, when you access it you are often accessing a copy
- and if you attempt to mutate it then your changes will be lost.

The following code illustrates the problem:
0

Because r.Location returns a value type, the update to it is 'lost'.

By this definition Python objects (all of them) are clearly reference
types and not value types.

In .NET you can specify that a parameter to a method takes a reference
('out' in C# or ByRef in VB.NET). If you pass in a value type as a
reference parameter then the .NET runtime will do boxing / unboxing
for you.

This means that we can write code like the following (roughly C#
pseudo code):

int x = 3;
SomeMethod(out x);

x can now be mutated by 'SomeMethod' and can have a different value.

In a 'way' immutable Python types behave a bit like .NET value types,
and mutable types like reference types. As you can see, this analogy
breaks down.

Michael Foord
 
D

Douglas Alan

Steven D'Aprano said:
I understand that Python's object and calling semantics are exactly the
same as Emerald (and likely other languages as well), and that both
Emerald and Python are explicitly based on those of CLU, as described by
by Barbara Liskov in 1979:

"In particular it is not call by value because mutations of
arguments performed by the called routine will be visible to the
caller. And it is not call by reference because access is not
given to the variables of the caller, but merely to certain
objects."

It is quite true that Python's calling semantics are the same as
CLU's, and that CLU called these semantics "call by sharing". It's
not quite true that CLU invented these calling semantics, however.
They were the calling semantics of Lisp long before CLU existed. I'm
not sure that Lisp had a name for it, however. Lisp people tended to
refer to all variable assignment as "binding".

I agree with those who object to calling the Python/CLU/Lisp calling
semantics as either "call by value" or "call by reference", which is
why I've been a bit dismayed over the years that the term "call by
sharing" hasn't caught on. When this terminology is used, the meaning is
quite unambiguous.

It should be noted that many other mainstream languages now use call
by sharing. It's not at all the least peculiar to Python. Such
languages include Java, JavaScript, Ruby, C#, and ActionScript.

|>oug
 
G

gooberts

Right, though not in languages like C and Pascal that don't HAVE the  
notion of objects.  We really ought to stop bringing up those  
dinosaurs and instead compare Python to any modern OOP language.


No?  Is there any way to prove that, without delving into the Python  
source itself?

If not, then I think you're talking about an internal implementation  
detail.


I think this "uncontrived" example addresses the C/Python difference
fairly directly (both were tested):
----------------------------------
C:

struct {int a;} s1, s2;

int main()
{
s1.a = 1;
s2 = s1;

printf("s1.a %d s2.a %d\n", s1.a, s2.a);
s1.a = 99;
printf("s1.a %d s2.a %d\n", s1.a, s2.a);
}

----------------------------------
Python:

class mystruct:
pass

s1 = mystruct()
s1.a = 1
s2 = s1

print "s1.a %2d s2.a %2d" % (s1.a,s2.a)
s1.a = 99
print "s1.a %2d s2.a %2d" % (s1.a,s2.a)

---------------
C OUTPUT:
s1.a 1 s2.a 1
s1.a 99 s2.a 1

Python OUTPUT:
s1.a 1 s2.a 1
s1.a 99 s2.a 99

Note that in C (or C++) the value of s2.a remains unchanged, because
the VALUE of s1 (the contents of the memory where s1 resides) was
COPIED to the memory location of s2, and subsequently, only the VALUE
of s2.a was changed. In Python, s2.a is "changed" (but not really)
because it turns out that s2 is just another name for the object that
s1 pointed to.

So there is no programatically accessible "location in memory" that
corresponds to s1 or s2 in Python. There is only a location in memory
that corresponds to the object that s1 is currently pointing to. In C,
by contrast, there are definite locations in memory that correspond to
both variables s1 and s2, and those locations remain always separate,
distinct and unchanged throughout the execution of the program.

This is not an "implementation detail", it is a fundamental part of
each language. As C was an "improvement" on assembler, the variable
names have always just been aliases for memory locations (or
registers). You can see this in the output of any C/C++ compiler.

In Python, variables are just names/aliases for *references* to
objects, not names for the objects/values themselves. The variable
names themselves do not correspond directly to the objects' memory
locations. While yes, technically, it is true that those reference
values must be stored somewhere in memory, *that* is the
implementation detail. But is is not the *locations* of these
references (i.e., the locations of the Python *variables*) that are
copied around, it is the references themselves (the locations of the
Python *objects*) that are copied.
And what does that name->object mapping consist of?  At some level,  
there has to be a memory location that stores the reference to the  
object, right?

I think this is answered above, but just to drive it home, in Python
the memory locations of the variables themselves (an implementation
detail), which hold the references to the objects, are inaccessible .
In C/C++, by contrast, variable names correspond directly to memory
locations and objects, and you can easily find the addresses of
variables.

In C/C++, if you choose, you may have a variable that is itself a
reference/pointer to some other memory/object. In C, we would say that
the VALUE of that variable is the memory address of another object.
But you can, if you need to, get the address of the pointer variable,
which points to the *address* of the other object.

In Python, a variable is ONLY EVER a reference to an object. You
cannot get the address of a Python variable, only of a Python object.

Hope this clears things up.

dale
 
D

Dale Roberts

Right, though not in languages like C and Pascal that don't HAVE the
notion of objects. We really ought to stop bringing up those
dinosaurs and instead compare Python to any modern OOP language.


No? Is there any way to prove that, without delving into the Python
source itself?

If not, then I think you're talking about an internal implementation
detail.


I think this "uncontrived" example addresses the (C/C++)/Python
difference fairly directly.
----------------------------------
C:

struct {int a;} s1, s2;

int main()
{
s1.a = 1;
s2 = s1;

printf("s1.a %d s2.a %d\n", s1.a, s2.a);
s1.a = 99;
printf("s1.a %d s2.a %d\n", s1.a, s2.a);
}

----------------------------------
Python:

class mystruct:
pass

s1 = mystruct()
s1.a = 1
s2 = s1

print "s1.a %2d s2.a %2d" % (s1.a,s2.a)
s1.a = 99
print "s1.a %2d s2.a %2d" % (s1.a,s2.a)

---------------
C OUTPUT:
s1.a 1 s2.a 1
s1.a 99 s2.a 1

Python OUTPUT:
s1.a 1 s2.a 1
s1.a 99 s2.a 99

Note that in C (or C++) the value of s2.a remains unchanged, because
the VALUE of s1 (the contents of the memory where s1 resides) was
COPIED to the memory location of s2, and subsequently, only the VALUE
of s2.a was changed. In Python, s2.a is "changed" (but not really)
because it turns out that s2 is just another name for the object that
s1 pointed to.

So there is no programatically accessible "location in memory" that
corresponds to s1 or s2 in Python. There is only a location in memory
that corresponds to the object that s1 is currently pointing to. In C,
by contrast, there are definite locations in memory that correspond to
both variables s1 and s2, and those locations remain always separate,
distinct and unchanged throughout the execution of the program.

This is not an "implementation detail", it is a fundamental part of
each language. As C was an "improvement" on assembler, the variable
names have always just been aliases for memory locations (or
registers). You can see this in the output of any C/C++ compiler.

In Python, variables are just names/aliases for *references* to
objects, not names for the objects/values themselves. The variable
names themselves do not correspond directly to the objects' memory
locations. While yes, technically, it is true that those reference
values must be stored somewhere in memory, *that* is the
implementation detail. But is is not the *locations* of these
references (i.e., the locations of the Python *variables*) that are
copied around, it is the references themselves (the locations of the
Python *objects*) that are copied.
And what does that name->object mapping consist of? At some level,
there has to be a memory location that stores the reference to the
object, right?

I think this is answered above, but just to drive it home, in Python
the memory locations of the variables themselves (an implementation
detail), which hold the references to the objects, are inaccessible .
In C/C++, by contrast, variable names correspond directly to memory
locations and objects, and you can easily find the addresses of
variables, and the addresses do not change, although the values can.

In C/C++, if you choose, you may have a variable that is itself a
reference/pointer to some other memory/object/array. In C, we would
say that the VALUE of that variable is the memory address of another
object. But you can, if you need to, get the address of the pointer
variable, which points to the *address* of the other object.

In Python, a variable is ONLY EVER a reference to an object. You
cannot get the address of a Python variable, only of a Python object.

Hope this clears things up.

dale
 
J

Joe Strout

I think this "uncontrived" example addresses the C/Python difference
fairly directly (both were tested):

That's correct, but of course, C is a decades-old language barely a
step above assembler. For a fair comparison, pick any modern OOP
language, including the C derivatives (C++, Objective-C, Java), and
compare a Python object to an object in that language, rather than to
a struct.
In Python, variables are just names/aliases for *references* to
objects, not names for the objects/values themselves. The variable
names themselves do not correspond directly to the objects' memory
locations.

Exactly! In C++, this would be like a pointer variable. In Java, RB,
or .NET, it's like an object reference.
While yes, technically, it is true that those reference
values must be stored somewhere in memory, *that* is the
implementation detail.

Agreed. (And this was my point in response to someone arguing that no
such location exists.)
But is is not the *locations* of these
references (i.e., the locations of the Python *variables*) that are
copied around, it is the references themselves (the locations of the
Python *objects*) that are copied.

Right. The variables contain object references; these object
references are copied (not referenced!) when you pass them into a
function. That's call by value. In the case of an assignment, the
reference is copied.
I think this is answered above, but just to drive it home, in Python
the memory locations of the variables themselves (an implementation
detail), which hold the references to the objects, are inaccessible .

Right. So too in Java, RB, .NET, etc. Pointers are nasty. But lack
of pointers does not change the calling semantics.
In C/C++, by contrast, variable names correspond directly to memory
locations and objects, and you can easily find the addresses of
variables.

Hmm, no, the C++ equivalent of an object reference would be:

SomeClass* foo;

In fact, many C++ programmers prefer to typedef a pointer to each
class, since it's the pointer (which is rough equivalent of a
reference in newer languages) that you almost always want, rather than
the class itself. So it would actually be:

SomeClassPtr foo;

Now, when you do:

foo2 = foo;

you are copying the object reference from foo to foo2, just as in
Python. When you pass it into a function:

void NiftyMethod(SomeClassPtr arg) {...}
...
NiftyMethod(foo);

You are copying the object reference right onto the call stack. This
is pass by value, plain and simple. And because it is pass by value,
you know that nothing NiftyMethod does can possibly change the value
of foo -- that is, can make it point at something else. (It may well
change the data that the object it points to contains, if SomeClass
objects are mutable, but it can't change foo itself.) This is, again,
just like Python and every other OOP language.

Call by reference would be this:

void NiftyMethod2(SomeClassPtr &arg) {...}
...
NiftyMethod2(foo);

Now, arg here is passed by reference (just like a ByRef parameter in
RB or .NET). That means that NiftyMethod2 could very well change the
value that is passed in. It could actually make foo point to
something else.

No Python method can do that, because Python arguments are ALWAYS
passed by value. There is no call by reference in Python. Period,
end of story, nothing to see here.

Cheers,
- Joe
 
T

Terry Reedy

In particular, memory is a linear sequence of writable cells. In Turing
machines, they have no address, and access is by one-at-a-time movement.
Subsequently, cells were given count addresses 0, 1, 2, ... .

The burden of proof is on those who claim a positive. Grant and I claim
that there is no pink elephant in the room, and that we have looked
around pretty thoroughly. You claim that there is. Show us what we missed.

Anyway, suppose you read and execute "L123 = [1,2,3]". You can, I
presume, perform any list method on that object. Where is it? Some
memory theories say 'distributed, nowhere in particular'. Others say
'concentrated, some particular neurons', but certainly nothing like the
'where' of linear computer memory. Consider another object s = 'boo'.
Which has the lower address?

Python, the abstract information algorithm language, has no 'source',
only a specification of syntax and semantics. The CPython source is the
source for a particular computer machine implementation. Any such
implemetation has to map abstractly locationless objects to particular
blocks of computer memory without adding extraneous, Python-accessible
semantics, such as relative position.

I think *you* are. Giving Python objects an integer address is an
(optional) implementation detail. Giving them a *fixed* integer address
is an optional CPython detail that makes certain aspects of the code
easier but which prevents re-locating garbage collection (which I
believe Jython does).

Anyone is free to view Python as only a linear-memory computer
programming language (and, in some respects, as that, inferior to C).
However, I have also viewed it as an algorithm language and, as a
beginner (about 10 years ago), quickly dubbed it 'executable pseudocode'
(and in that respect, much superior to C).

Terry Jan Reedy
 
S

Steven D'Aprano

That's correct, but of course, C is a decades-old language barely a step
above assembler.

So what? It's not like C is no longer in common use.

And even if C had disappeared off the landscape, there are millions of
programmers, including beginning programmers, whose understanding of
terms CBR and CBV are defined by the behaviour of languages like C and
Pascal -- even if they haven't learned either language themselves.

For a fair comparison, pick any modern OOP language,
including the C derivatives (C++, Objective-C, Java), and compare a
Python object to an object in that language, rather than to a struct.

But we already know that many such languages use the exact same calling
convention as Python, so all you would be showing is that languages with
the same calling convention as Python have the same calling convention as
Python.

By common usage and technical definition, C is call by value. Argument
passing in Python does not behave like C. So why insist that Python is
also call by value?

[snip]
Agreed. (And this was my point in response to someone arguing that no
such location exists.)

Obviously any computer which is based on the Von Newman architecture of
CPU plus memory is going to store the reference *somewhere*. That's a
detail unimportant at the Python level. But since clockwork or Conway's
"Life" cellular automata are both Turing complete, it would be possible
to build a Python implementation where the reference values weren't
localised to a particular place in memory, but distributed over the
system.

Right. The variables contain object references; these object references
are copied (not referenced!) when you pass them into a function. That's
call by value. In the case of an assignment, the reference is copied.

The value of a Python name is the Python object assigned to it, not an
arbitrary memory location that points to the object. Even you would
consider it obfuscatory if I executed this code:

x = "Norwegian Blue"

and then insisted that the value of x was "3086179808L, but if I run that
line of code again it could get another value, and naturally if you run
it on your computer you're almost certain to get a different value".

By your definition of "value=reference", the above is perfectly correct,
and utterly, completely pointless, useless and unhelpful. It's rather
like listing the ingredients of a cake as "Atoms". Technically true, but
missing the point.



[snip]
You are copying the object reference right onto the call stack. This is
pass by value, plain and simple.

Who *cares* about copying the pointers? That's the WRONG LEVEL.
Ultimately EVERY programming language that runs on a computer with memory
is "Copy By Bit Flipping", but it would be useless and unhelpful to tell
people that every programming language uses the exact same calling
conventions: "bits are flipped, and that's the end of story".


[snip]
No Python method can do that, because Python arguments are ALWAYS passed
by value. There is no call by reference in Python.

Except that for millions of programmers who have learnt their terminology
from Pascal and C, that implies that the values -- the data itself, not
pointers to the data -- are copied. It implies that this can never fail:

x = [1]
y = function(x)
assert x == [1]

but of course it can fail, if function modifies x -- something which CBV
implies can't happen.


Period, end of story, nothing to see here.

I think that this entire argument hinges on the poverty of your mental
toolbox. You have a hammer (call by value) and a screwdriver (call by
reference) and refuse to accept that there could possibly be any other
calling model. Hence you force any actual calling model into the
framework of CBV or CBR, no matter how much violence you have to do to
simple terms like "value" to make it fit.
 
C

Chuckk Hubbard

I'm sorry to say I'm pretty confused by the example, but if you want
something like

bob = module.object()
frank = module.object()
and then to know that bob is bob from a list of instances, you could
instead do something like:

for person in listofnames:
temp = module.object(person)
list.append(temp)

where the __init__ function assigns the argument to object.nametag or
something. This way you can retrieve the name from an index into the
list, or retrieve the index by searching for the name...
-Chuckk

Why do you need to find that? You know that its name is 'bob'.

I'm creating mulitple instances, putting them in a list, iterating
through the list to send them to some functions where process them
with some instance specific parameters. Something along the lines of:

bob = someobject()
harry = someobject()
fred = someobject()

parameterdict = {'bob':(0,1,2),'harry':(3,4,5),'fred':(6,7,8)}
people_list = (bob, harry, fred)

for person in people_list:
add_parameters(person)

def add_parameters(person)
mytuple = parameterdict[??????instance.name????]
person.x = mytuple[0]
person.y = mytuple[1]
person.z = mytuple[2]

... alternatively there is probably a very much easier way of doing
it.
 
G

greg

Steven said:
By common usage and technical definition, C is call by value. Argument
passing in Python does not behave like C. So why insist that Python is
also call by value?

Whether it behaves like C is not the test.

Let's look at the definitions of the terms:

(1) Call by value: The actual parameter is an expression. It is
evaluated and the result is assigned to the formal parameter.
Subsequent assignments to the formal parameter do not affect
the actual parameter.

(2) Call by reference: The actual parameter is an lvalue. The
formal parameter becomes an alias for the actual parameter,
so that assigning to the formal parameter has the same
effect as assigning to the actual parameter.

Seems to me that (1) describes exactly how parameter passing
works in Python. So why insist that it's *not* call by value?
 
D

Dale Roberts

[I am actually enjoying this discussion, even though it does not address
the OP's question. It is helping to solidify *my* understanding.]

Joe said:
That's correct, but of course, C is a decades-old language barely a step
above assembler. For a fair comparison, pick any modern OOP language,
including the C derivatives (C++, Objective-C, Java), and compare a
Python object to an object in that language, rather than to a struct.

Okay, sorry, should have had C++ from the start. See my spiffed-up
example here, where I magically convert to C++ by replacing "struct"
with "class" and adding the word "public:". Still behaves the same, though.

I added the functions ByVal() and ByRef() to show that when you pass by
value, the contents of the object cannot be changed. This is not
possible in Python unless a copy of the object is made (which is what
C++ does automatically for you in pass-by-value).

In Python, the contents of the object are changed, which is most similar
to the pass by reference (or by pointer) construct in C++. But, yes, as
you say, technically:

passing the object by reference
== passing the address of the object by value.

But I think that is, to most programmers, a surprising use of the term
"pass by value".

And note in my example, nowhere do I explicitly take the address of s1.
The compiler does this for me in the ByRef() case. And nowhere to I make
a copy of s1. Again, the compiler does this for me in the ByVal() case.
The calls to ByVal() and ByRef() are identical. You cannot tell from the
calls which thing is going to happen.

-----------------------
class MyClass {public: int a;} s1, s2;

void ByVal(MyClass obj) {obj.a=42;}
void ByRef(MyClass &obj) {obj.a=43;}

int main()
{
s1.a = 1;
s2 = s1;

printf("s1.a %2d s2.a %2d\n", s1.a, s2.a);

s1.a = 99;
printf("s1.a %2d s2.a %2d\n", s1.a, s2.a);

ByVal(s1);
printf("s1.a %2d\n", s1.a);

ByRef(s1);
printf("s1.a %2d\n", s1.a);
}
--------------
class mystruct:
pass

def ByObject(obj): obj.a=42

s1 = mystruct()
s1.a = 1
s2 = s1

print "s1.a %2d s2.a %2d" % (s1.a,s2.a)
s1.a = 99
print "s1.a %2d s2.a %2d" % (s1.a,s2.a)

ByObject(s1)
print "s1.a %2d" % (s1.a)

--------------
C++ OUTPUT
s1.a 1 s2.a 1
s1.a 99 s2.a 1 # note s2.a does not change when s1.a is modified
s1.a 99 # contents of s1.a does not change when passed by val
s1.a 43 # it does change when passed by ref

Python OUTPUT
s1.a 1 s2.a 1
s1.a 99 s2.a 99 # s2.a "changes" because it's the same object as s1
s1.a 42 # Contents of object does change with function call.

....and in Python, of course, as you say, what you call the "value" of
s1, the address of the object, id(val), does not change.
>>
>> [skipping lots of stuff we agree on!]
>>
In C/C++, by contrast, variable names correspond directly to memory
locations and objects, and you can easily find the addresses of
variables.

Hmm, no, the C++ equivalent of an object reference would be:

SomeClass* foo;

Whoah, nonsequitor there. Let's back up. I did not use the word
"reference", and that is the point. And I am correct that C variable
names correspond to memory locations (or sometimes CPU registers -
optimizers can really mangle things). And you can get the addresses of
the variables themselves using the & operator.

Have a look at my example code again. s1 and s2 ARE OBJECTS THEMSELVES,
they are not references to objects. You can do this in C++ (not in
Python). You can pass whole objects, by value (they are copied by the
compiler), on the stack. And we both understand that you can't do that
in Python. That is why we differentiate between "pass by reference" and
"pass by value" in C++, and people generally understand what that means.
> ...
void NiftyMethod2(SomeClassPtr &arg) {...}
...
NiftyMethod2(foo);

Now, arg here is passed by reference (just like a ByRef parameter in RB
or .NET). That means that NiftyMethod2 could very well change the value
that is passed in. It could actually make foo point to something else.

No Python method can do that,

Yes, absolutely agreed. A Python method can never change *which* object
the caller points to, it can only change the *contents* of that object.

But the same is true in C++ as well. In your example, the address of the
foo variable itself can never be changed. You can change the *value* of
foo (so it points to a different object), or you can change the contents
of the object foo points to. foo is a variable with an address which you
can usually see with a printf("%x", &foo), and that is different from
the address of the object which you get when you say printf("%x", foo).

The value &foo cannot be changed.
because Python arguments are ALWAYS passed
by value. There is no call by reference in Python. Period, end of
story, nothing to see here.

Yea, BUT... If you tell this to a C++ programer without any further
explanation, they will be thoroughly confused and misinformed, unless
you point them to this thread or amend that statement with your version
of what "pass by value" means. I know what you mean, and you know what I
mean, but that's because we've both read through this thread ;-)

Look at my example. The ByVal() routine behaves how C++ programmers
expect "pass by value" to work. The contents of the caller's object
cannot be modified.

So, then, what to tell a C++ programmer about how Python passes
arguments? You say: tell them Python only passes by value. I disagree,
because I think that would confuse them. Rather than try to map C++
conventions onto Python, I think it is more useful to just tell them how
it really works. Maybe a few statements like this:

All values in Python are objects, from simple integers up to complex
user-defined classes.

An assignment in Python binds a variable name to an object. The
internal "value" of the variable is the memory address of an object,
and can be seen with id(var), but is rarely needed in practice.

The "value" that gets passed in a Python function call is the address
of an object (the id()).

When making a function call, myfunc(var), the value of id(var) can
never be changed by the function.

Not sure if these are the best. To get into much more detail, you have
to start explaining mutable and immutable objects and such.

dale
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,773
Messages
2,569,594
Members
45,125
Latest member
VinayKumar Nevatia_
Top