Is 'everything' a refrence or isn't it?

[KraftDiner]

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
if lst == 2:
lst=4
print lst

Have I misunderstood something?

 

[Christian Tismer]

I was under the assumption that everything in python was a refrence...

This is true.

so if I code this:
lst = [1,2,3]

lst is a reference to a list that holds references to 1, 2, and 3

for i in lst:

i is a reference to every element in the list, one ofter the other.
But this is not a ref to lst any longer, the list gets dereferenced
before getting at its contents, and only references to that are
returned.

if i==2:
i = 4

You now have i as a reference to 4, during this cycle.

ciao - chris

--
Christian Tismer :^) <mailto:[email protected]>
tismerysoft GmbH : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/

 

[Grant Edwards]

I was under the assumption that everything in python was a refrence...

It is.

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified..

Nope. "i = 4" doesn't modify the object. It changes "i" to
point to a different object (one that is an integer value 4).

(After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
if lst == 2:
lst=4
print lst

Have I misunderstood something?

Sort of.

You've misunderstood what the statement "i = 4" means.

 

[Duncan Booth]

I was under the assumption that everything in python was a refrence...

It is, although it is better to think in terms of names and bindings.

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

During execution of this code, the name 'i' is bound to 1, 2, 3 and 4. the
list elements are bound to 1, 2, and 3. Rebinding 'i' such that instead of
referencing 2 it now references 4 doesn't affect the fact that the second
element of the list references 2.

so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
if lst == 2:
lst=4
print lst

Have I misunderstood something?

Evidently.

 

[Scott David Daniels]

Have I misunderstood something?

yes.
There is a reason people talk about names and bindings.

 

[Simon Brunning]

I was under the assumption that everything in python was a refrence... (snip)
Have I misunderstood something?

Yup. The concept if a reference is, I find, sometimes an unhelpful one
in Python. Reset your brain -
<http://effbot.org/zone/python-objects.htm>.

Armed with that, let's go through your code:

lst = [1,2,3]

Here, you've created a list object, and bound it to the name "lst".
The list contains references to three integer objects. (OK I said
"reference" there didn't I? Well, I only said that it's *sometimes* an
unhelpful concept. ;-) I do tend to think of collections as refering
to objects. It's when dealing with you might call variables that it's
misleading - there it's better to think of names, objects, and
bindings.)

for i in lst:

Here, you are iterating through the list that's currently bount to the
name "lst". You are binding each element of the list in turn to the
name "i".

if i==2:

If the object to which the name "i" is currently bound compares equal to 2....

i = 4

Rebind the name "i" to the integer object 4. Note that this has no
effect on the object that used to be bound to "i" at this point, nor
any effect on the list object.

print lst

Print your old list.

Clear now?

 

[Claudio Grondi]

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
if lst == 2:
lst=4
print lst

Have I misunderstood something?

To give a short answer: yes.

And now into the details assuming, that your misunderstanding comes from
programming in C before evaluating Python:

If you write
i = 4
following happens:
an integer literal 4 gets into the scope of Python script as an object
of type integer and can be from now on reached using the identifier i .
So i is in this context a _kind of_ pointer/reference to the value
'4', but ... in Python sense not in C sense.

if you write
for i in lst:
then the identifier i 'points' to values in lst, but else as in C
where you can use a pointer to change the value it points to by
assigning to *i or where you can use a reference to change a value it
references, in Python you don't change a value of i because i is an
identifier and therefore has no value in sense of C - it is only a name
used to reach a value.

When you write within the 'for i in lst:' loop
i = 4
you just use the identifier i as a 'pointer' to another Python object
as it was pointing to before.

Let's consider a list.
The 'identifier'(i.e. construct) lst pointing to a list element
behaves in Python like what you understand in C as a reference.

This is the main difference between the meaning of = assignment in
Python and in C and it usually results in problems with understanding it
when someone has programmed C and expects from Python to be like C.

The core of the trouble is probably different understanding of the word
'reference', which used in different contexts (e.g. Python and C) means
different things. Therefore as Scott David Daniels says
"There is a reason people talk about names and bindings."

Claudio

 

[Steven D'Aprano]

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

It isn't "call by value" either -- Python never copies objects unless you
explicitly tell it to.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

In the above code, i is a name bound one at a time to the ints [1,2,3].
When you re-assign i to 4, that doesn't change the object 2 into the
object 4, because ints are immutable. Only the name i is rebound to a new
object 4. That doesn't change objects like lst which include 2 inside them.

See this for more detail:

http://effbot.org/zone/call-by-object.htm

 

[Steven D'Aprano]

This is true.

No it is not.

so if I code this:
lst = [1,2,3]

lst is a reference to a list that holds references to 1, 2, and 3

for i in lst:

i is a reference to every element in the list, one ofter the other.
But this is not a ref to lst any longer, the list gets dereferenced
before getting at its contents, and only references to that are
returned.

See what confusing, complicated hoops people have to jump through to
hammer the round peg of Python's behaviour into the square peg of "call by
reference"?

http://effbot.org/zone/call-by-object.htm

Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

 

[Claudio Grondi]

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

It isn't "call by value" either -- Python never copies objects unless you
explicitly tell it to.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

In the above code, i is a name bound one at a time to the ints [1,2,3].
When you re-assign i to 4, that doesn't change the object 2 into the
object 4, because ints are immutable. Only the name i is rebound to a new
object 4. That doesn't change objects like lst which include 2 inside them.

See this for more detail:

http://effbot.org/zone/call-by-object.htm

It seems to be hard to explain it all in a straighforward way without
using pictures showing what happens when an assignment is processed
demonstrating what is what and how it is called. Probably a small nice
movie could be here the right medium to be handed over to Python newbies
coming from C to give some hints towards proper understanding. Replacing
one word with another is not sufficient to avoid confusion as the
concept is probably not always easy to grasp (I have still problems to
get the idea how it works 'inside' with this local and global
dictionaries and so on).
Lack of knowledge about a good reference was the reason why I decided to
give a reply here inspite of the fact, that so many other postings were
already there, but none (including mine) which covered all aspects, so
that I could say: "Wow! It is an excelent explanation and there is
nothing which must be added to it!".
Is there anywhere an ultimative enlightening explanation of the concept
of identifiers and variables in Python for use in case the next posting
of that kind hits comp.lang.python and it sure will as they are coming
in quite regular.
http://effbot.org/zone/call-by-object.htm is sure an attempt of a good
explanation, but in my eyes still nothing what immediately enlightens.

Claudio

 

[Peter Hansen]

Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation of
binding (i.e. usually what assignment does).

-Peter

 

[Mike Meyer]

I was under the assumption that everything in python was a refrence...
so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst
I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

Except this doesn't have *anything at all* to do with python being (or
not being) call by reference. This is a confusion about name binding
vs. assignment to a variable. The proper people to slap around for
this case are the ones who talk about assignment to a variable.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

No, you get *exactly* that behavior from call by reference when you
start passing objects around by reference. If I declare a C object as
"struct foo bar" and do the C "call-by-reference" hack of passing
&bar, I get the exact same behavior I get when I pass an object
referenced by bar to a Python subroutine.

<mike

 

[Mike Meyer]

I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

Correct. What's stored in a list is a reference.

What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation
of binding (i.e. usually what assignment does).

To clarify, assignement statements in python are bindings, and don't
do assign values to a variables. Assignment in other languages do that
- which is why people get confused when exposed to what Python does.

<mike

 

[Steven D'Aprano]

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

On reading back over my post, I realise that it might
sound like I was mad at KraftDiner. My apologies -- I'm
not, I feel (s)he is the victim of incorrect
information here, not the culprit.

After all, as a Python newbie, how is KraftDiner
supposed to know that when people say "Python is call
by reference", what they actually mean is "Um, except
that it doesn't behave like any call by reference
language you're likely to have used before, and
sometimes it behaves more like call by value"?

 

[Donn Cave]

See what confusing, complicated hoops people have to jump through to
hammer the round peg of Python's behaviour into the square peg of "call by
reference"?

http://effbot.org/zone/call-by-object.htm

Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

I agree that it can be simple and straightforward. The particular
terms we use have nothing to do with it, though. Most of us need
a word or two to attach to the concept, because that's how our
brains work, but there are no natural words in the English language
that precisely express Python concepts, and we just have to make
do with one or another of several possible. It's silly to insist
that one particular word resolves all confusions, and another word
creates them. It's sort of unfortunate to let this crusade get to
the point where you're upbraiding people who know a great deal about
the matter because they don't use your favorite word.

As other followups have pointed out, the problem is simply not
understanding what "=" does. While in some obscure way this
may be related to "call by reference" etc., that's certainly not
a helpful digression. To explain what i = 4 does, we can say

creates a named reference "i" to 4
binds the name "i" to 4
etc.

Any of these is delightfully clear if you already understand it,
which is where most of the born again believers are coming from.
People who don't already understand what's being explained, need
to have a little patience and make sure to test their understanding
with a few experiments.

Donn Cave, (e-mail address removed)

 

[Steven D'Aprano]

Claudio Grondi wrote:

[snip]

....

It seems to be hard to explain it all in a straighforward way without
using pictures showing what happens when an assignment is processed
demonstrating what is what and how it is called.

That's no different from call by reference or call by
value. If you think back to when you were first
learning about pointers, that was confusing too.

Probably a small nice
movie could be here the right medium to be handed over to Python newbies
coming from C to give some hints towards proper understanding. Replacing
one word with another is not sufficient to avoid confusion as the
concept is probably not always easy to grasp (I have still problems to
get the idea how it works 'inside' with this local and global
dictionaries and so on).

No, just saying "call by object" is not sufficient, but
it is vital. If you say "call by reference", the term
comes with a lot of mental baggage -- people think they
understand what call by reference means, how it
behaves, and as soon as they hear CBR they think they
understand how Python behaves. They think "Oh, Python
is CBR just like language Foo, so it must behave just
like language Foo, and this idiom works in language
Foo, so it must work in Python too."

And sometimes it does appear to work, which just makes
it all the more puzzling when it doesn't.

When they hear CBO the term is new and different and
doesn't come with any mental frame, so instead of
thinking they understand Python's behaviour, they think
"Call by what now? How does that work?" instead of
making incorrect assumptions.

 

[Steven D'Aprano]

I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation of
binding (i.e. usually what assignment does).

I just call it an item of the list. The fact that
CPython uses an array of pointers to objects for lists
is an implementation detail. Unfortunately, while
"reference" is a nice generic English word, in this
context it has too much mental baggage to safely use
around newbies. (Experience Pythonistas is another
story of course.)

 

[Steven D'Aprano]

I was under the assumption that everything in python was a refrence...
so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst
I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

Except this doesn't have *anything at all* to do with python being (or
not being) call by reference. This is a confusion about name binding
vs. assignment to a variable. The proper people to slap around for
this case are the ones who talk about assignment to a variable.

Mike, you are wrong. Not about talk about "assigning to
variables" being harmful, I agree with that. But this
confusion is *exactly* about the call by reference
misunderstanding. How do I know this?

Because the Original Poster said so! He said, to
paraphrase, "Hey, I thought Python was call by
reference, but I tried this, and it didn't work, what
gives???"

This is not an isolated case either.

You can't get any clearer than that. Talk about call by
reference leads to confusion and incorrect assumptions
about Python's behaviour. End of story.

No, you get *exactly* that behavior from call by reference when you
start passing objects around by reference.

Who cares what Python does internally? It is strictly
irrelevant whether Python internally passes around
pointers, or copies values, or sometimes one or the other.

If I declare a C object as
"struct foo bar" and do the C "call-by-reference" hack of passing
&bar, I get the exact same behavior I get when I pass an object
referenced by bar to a Python subroutine.

It is not a question of whether Python's behaviour is
*sometimes* the same as call by reference. The question
whether Python's behaviour is *always* the same as CBR,
and it is not. Regardless of the reasons, Python does
not behave like CBR in other languages, and it is
harmful to imply that it does.

Likewise, you shouldn't argue that Python is obviously
call by value just because *sometimes* it has the same
behaviour as CBV.

I mean, come on! A whole bunch of people argue black
and blue that Python "obviously" is call by reference,
and another vocal bunch argue that it is "obviously"
call by value. Isn't that the tiniest hint that Python
-- not the underlying C/Java/Python/whatever
implementation, but Python the language --- does not
fall into either camp, but is doing something that
sometimes looks like one and sometimes like the other?

Just google on "Python is call by value" and "Python is
call by reference" -- including quotes -- to see this
argument come up time and time again, year after year
after year. And all because of some false dichotomy
that CBR and CBV are the only two valid descriptions of
a computer language.

 

[Steven D'Aprano]

Correct. What's stored in a list is a reference.

Nonsense. What is stored in the list is an object.
Python doesn't have pointers. You are confusing the
underlying C implementation, which implements lists as
an array of pointers, with the high-level Python code,
which doesn't even know what a pointer is. It only has
objects.

Since we are discussing the perspective from a Python
programmer's point of view, it is irrelevant what the C
implementation does. If lists held references, you
could do this:

x = 5
L = [1, 2, x]
# the third item points to the same memory location
# as x; changing it will also change x
L[2] = 7
assert x == 7

which is the behaviour that "call by reference"
implies. But it doesn't work that way in high-level
Python code.

The *reason* it doesn't work is that the object 5 is
immutable. If you only ever passed mutable objects
around, you would think Python was call by reference.
And if you only ever passed immutable objects around,
you would think Python was call by value. But since
Python has both sorts of objects, it displays both
sorts of behaviour.

 

[Aahz]

Nonsense. What is stored in the list is an object. Python doesn't have
pointers. You are confusing the underlying C implementation, which
implements lists as an array of pointers, with the high-level Python
code, which doesn't even know what a pointer is. It only has objects.

Wrong. Python has objects and targets. Names are a subset of targets.
A target is anything that can contain a binding (or reference, whichever
term you prefer) to an object; lists are arrays of targets.

No matter how you slice it ;-), a list does not actually hold objects, it
only holds references to objects.