When is it a pointer (aka reference) - when is it a copy?

[John Henry]

Hi list,

Just to make sure I understand this.

Since there is no "pointer" type in Python, I like to know how I do
that.

For instance, if I do:

...some_huge_list is a huge list...
some_huge_list[0]=1
aref = some_huge_list
aref[0]=0
print some_huge_list[0]

we know that the answere will be 0. In this case, aref is really a
reference.

But what if the right hand side is a simple variable (say an int)? Can
I "reference" it somehow? Should I assume that:

aref = _any_type_other_than_simple_one

be a reference, and not a copy?

Thanks,

 

[Laszlo Nagy]

John Henry írta:

Hi list,

Just to make sure I understand this.

Since there is no "pointer" type in Python, I like to know how I do
that.

For instance, if I do:

...some_huge_list is a huge list...
some_huge_list[0]=1
aref = some_huge_list
aref[0]=0
print some_huge_list[0]

we know that the answere will be 0. In this case, aref is really a
reference.

But what if the right hand side is a simple variable (say an int)? Can
I "reference" it somehow? Should I assume that:

aref = _any_type_other_than_simple_one

be a reference, and not a copy?

The short answer is that you need to keep the immutable value inside a
mutable object.

The long answer:

a.) You can reference the immutable object just like any object, but you
cannot change the immutable object. For instance, when you do

a = 1
a += 2

then you are rebinding the variable 'a' to a different object (namely,
the 2 int object.)

b.) You can however, keep a reference to your current immutable object
inside a mutable object. In many cases, the mutable object will be a
namespace dictionary.

A module is a very simple example:

a = 1
def f1():
global a
a += 1

def f2():
global a
a += 10

f1()
f2()
print a # prints 12

Notice that the "a+=10" will actually rebind the variable to a different
object.

In other cases, you will be using an object or a class:

class A(object):
a = 5

def inc_id(obj):
obj.id += 1

a = A() # a.id is 5 here
a.id = 4 # makes a.id a reference to 4
inc_id(a) # makes a.id a reference to 5 again


Best,

Laszlo

 

[Steve Holden]

Hi list,

Just to make sure I understand this.

Since there is no "pointer" type in Python, I like to know how I do
that.

For instance, if I do:

...some_huge_list is a huge list...
some_huge_list[0]=1
aref = some_huge_list
aref[0]=0
print some_huge_list[0]

we know that the answere will be 0. In this case, aref is really a
reference.

But what if the right hand side is a simple variable (say an int)? Can
I "reference" it somehow? Should I assume that:

aref = _any_type_other_than_simple_one

be a reference, and not a copy?

Yes. Attributes are always object references. The assignment is actually
the binding of a specific object to a name in some namespace, (r to an
element of a sequence or other container object.

This applies *whatever* the type of the RHS experession: the expression
is evaluated to yield an object, and a reference to the object is stored
in the name or container element.

regards
Steve

 

[John Henry]

Thanks for the reply, both to Laszlo and Steve.

Okay, I understand what you're saying.

But what if I need to make a "pointer" to a simple variable.

For instance, in C:

int i=1
int *j=&i

*j = 2
print i

and you get 2 printed.

In Python,

i=1
j=i
j=2
print i

and you get 1 printed.

So, if I understand you correctly, I must make the reference to a more
elaborate representation. Like:

i=[1,]
j=i
j[0]=2
print i

in order to get 2 printed.

Correct?

Hi list,

Just to make sure I understand this.

Since there is no "pointer" type in Python, I like to know how I do
that.

For instance, if I do:

...some_huge_list is a huge list...
some_huge_list[0]=1
aref = some_huge_list
aref[0]=0
print some_huge_list[0]

we know that the answere will be 0. In this case, aref is really a
reference.

But what if the right hand side is a simple variable (say an int)? Can
I "reference" it somehow? Should I assume that:

aref = _any_type_other_than_simple_one

be a reference, and not a copy?

Yes. Attributes are always object references. The assignment is actually
the binding of a specific object to a name in some namespace, (r to an
element of a sequence or other container object.

This applies *whatever* the type of the RHS experession: the expression
is evaluated to yield an object, and a reference to the object is stored
in the name or container element.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

 

[Grant Edwards]

Thanks for the reply, both to Laszlo and Steve.

Okay, I understand what you're saying.

But what if I need to make a "pointer" to a simple variable.

There's no such thing as a "simple variable". There are
mutable objects and immutable objects. Names are bound to
objects.

x = 3

The name "x" is bound to an immutable integer object who's
value is 3.

For instance, in C:

int i=1
int *j=&i

*j = 2
print i

and you get 2 printed.

In Python,

i=1

The name "i" is bound to an immutable integer object who's value is 1.

j=i

The name "j" is bound to an immutable integer object who's
value is 1. That may or may not be the same object to which
"i" is bound.

j=2

Now the name "j" is bound to an immutable integer object who's
value is 2. Rebinding the name "j" to a different object has
no effect on the object to which "i" is bound.

print i

and you get 1 printed.

Because you've changed neither the object to which "i" is bound
nor the value of that object (you can't change the values of
integer objects).

So, if I understand you correctly, I must make the reference
to a more elaborate representation. Like:

i=[1,]
j=i
j[0]=2
print i

in order to get 2 printed.

Correct?

I suppose, for some values of "correct". You've bound the
names "i" and "j" to the same mutable object, then mutated that
object. Afterwards "i" and "i" still refer to that mutated
object.

That'll work as a rather clumsy imitation of the C code, but I
don't really see what it is you're trying to accomplish. Trying
to write C code using Python isn't going to be fun or productive[1].

When using Python, you should write Python code.

If you'll explain the actual problem you're trying solve for
which you think you need C-style "pointers", then somebody will
be happy to show you how that problem is solved using Python.

[1] There are people here who probably think it fun, but only
as a brain-teaser.

 

[John Henry]

Thanks for the reply, Grant.

I am not doing things like that - I am just trying to clear up in my
mind the Python concepts.

I understand it now.

Thanks for the reply, both to Laszlo and Steve.

Okay, I understand what you're saying.

But what if I need to make a "pointer" to a simple variable.

There's no such thing as a "simple variable". There are
mutable objects and immutable objects. Names are bound to
objects.

x = 3

The name "x" is bound to an immutable integer object who's
value is 3.

For instance, in C:

int i=1
int *j=&i

*j = 2
print i

and you get 2 printed.

In Python,

i=1

The name "i" is bound to an immutable integer object who's value is 1.

j=i

The name "j" is bound to an immutable integer object who's
value is 1. That may or may not be the same object to which
"i" is bound.

j=2

Now the name "j" is bound to an immutable integer object who's
value is 2. Rebinding the name "j" to a different object has
no effect on the object to which "i" is bound.

print i

and you get 1 printed.

Because you've changed neither the object to which "i" is bound
nor the value of that object (you can't change the values of
integer objects).

So, if I understand you correctly, I must make the reference
to a more elaborate representation. Like:

i=[1,]
j=i
j[0]=2
print i

in order to get 2 printed.

Correct?

I suppose, for some values of "correct". You've bound the
names "i" and "j" to the same mutable object, then mutated that
object. Afterwards "i" and "i" still refer to that mutated
object.

That'll work as a rather clumsy imitation of the C code, but I
don't really see what it is you're trying to accomplish. Trying
to write C code using Python isn't going to be fun or productive[1].

When using Python, you should write Python code.

If you'll explain the actual problem you're trying solve for
which you think you need C-style "pointers", then somebody will
be happy to show you how that problem is solved using Python.

[1] There are people here who probably think it fun, but only
as a brain-teaser.

 

[George Sakkis]

So, if I understand you correctly, I must make the reference
to a more elaborate representation. Like:

i=[1,]
j=i
j[0]=2
print i

in order to get 2 printed.

Correct?

I suppose, for some values of "correct". You've bound the
names "i" and "j" to the same mutable object, then mutated that
object. Afterwards "i" and "i" still refer to that mutated
object.

Another way to explain why this is so, without fuzzy terms like "a more
elaborate representation", is that although the statements "x = 2" and
"x[0] = 2" look both as "assignments" syntactically, they work quite
differently under the hood. The former binds a name ("x") to an object
("2"). The latter is syntactic sugar for a method call:
x.__setitem__(0,2). If x happens to be a list, this is equivalent to
calling the unbound method list.__setitem__(x,0,2) which, as you
already know, mutates the list x. The important thing to remember
though is that the effect of something like "x[0] = 2" depends on the
type of x. Any class can define a __setitem__(index,value) method with
arbitrary semantics; it is not (and cannot be) forced to mutate x. The
bottom line is that you can't tell in advance what "x[0] = 2" will do
without knowing the type of x. A binding OTOH like "x=2" has always the
same semantics: make the name "x" refer to the object "2".

Similarly to "x[0] = 2", something like "x.foo = 2" looks like an
assignment but it's again syntactic sugar for a (different) method
call: x.__setattr__('foo',2). All the above about __setitem__ hold for
__setattr__ too.


HTH,
George

 

[Dennis Lee Bieber]

For instance, in C:

int i=1
int *j=&i

*j = 2
print i

and you get 2 printed.

In Python,

i=1
j=i
j=2
print i

and you get 1 printed.

The Python is ALWAYS a "reference"...

Your C example is closer to (this may not be valid C)

int *i = &1
int *j = i

j = &2
print i /* obviously an extension <G> */

References to lists, dictionaries, and class instances (which are,
in a way, just an expanded dictionary) are "mutable" -- you are "going
inside" the object that is referenced, and changing something inside it
-- but the reference stays the same (the references /inside/ the object
get changed)

Scalars (numeric values), strings, and tuples (deliberately, so they
can be dictionary keys) are "immutable" -- while you can retrieve
subparts (of strings and tuples) you are prevented from changing the
insides.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Fredrik Lundh]

References to lists, dictionaries, and class instances (which are,
in a way, just an expanded dictionary) are "mutable"

careful: it's not the *reference* that's mutable, it's the object.

the *only* difference between mutable and immutable objects is that
the latter don't provide any methods that you could use to modify their
contents.

plain assignment (name=) *never* modifies objects, and *never* copies
objects.

this article

http://effbot.org/zone/python-objects.htm

may be useful for those who haven't already seen it.

</F>

 

[Christophe]

John Henry a écrit :

Hi list,

Just to make sure I understand this.

Since there is no "pointer" type in Python, I like to know how I do
that.

For instance, if I do:

...some_huge_list is a huge list...
some_huge_list[0]=1
aref = some_huge_list
aref[0]=0
print some_huge_list[0]

we know that the answere will be 0. In this case, aref is really a
reference.

But what if the right hand side is a simple variable (say an int)? Can
I "reference" it somehow? Should I assume that:

aref = _any_type_other_than_simple_one

be a reference, and not a copy?

Thanks,

That's easy. In Python, every variable is a depth one pointer. Every
variable is of the type (PyObject*)

Of course, since numbers and strings are immutable, that pointer is
useless to push back the modifications you've done.

You need a PyObject** ? Use a one element list instead and manipulate it
like that : number_ref[0] = new_value
instead of that : number_ref = [new_value]

 

[Bruno Desthuilliers]

Hi list,

Just to make sure I understand this.

Since there is no "pointer" type in Python, I like to know how I do
that.

For instance, if I do:

...some_huge_list is a huge list...
some_huge_list[0]=1
aref = some_huge_list
aref[0]=0
print some_huge_list[0]

we know that the answere will be 0. In this case, aref is really a
reference.

But what if the right hand side is a simple variable (say an int)? Can
I "reference" it somehow? Should I assume that:

aref = _any_type_other_than_simple_one

be a reference, and not a copy?

short answer : Python won't copy anything until explicitely asked to.

Longer answer:

First, there's nothing like "simple" type or var in Python. All that you
have are names and objects. The statement 'some_name = some_obj' "binds"
together 'some_name' and 'some_obj' - IOW, once this statement is
executed, 'some_name' refers to ('points to') 'some_obj' (think of it as
an equivalent of 'globals['some_name'] = some_obj', and you won't be far
from truth). This is how it works for all and any type.

What you really need to understand is that in Python, a 'variable' is
*only* a name. It's *not* the object itself.

Now we have mutable and immutable types. Immutable types are (mainly)
numerics, strings and tuples. As the qualificative implies, one cannot
change the state (ie value) of an object of immutable type. Also, note
that mutating (modifying the state of an object) and assignment (binding
a name to an object) are two very different things. Rebinding a name
just make it points to another object, it doesn't impact the object
previously bound to that name (not directly at least, cf memory management).

To come back to your code snippet:

# binds name "some_huge_list" to an empty list
some_huge_list = []

# mutate the list object bound to name 'some_huge_list'
some_huge_list[0]=1

# binds name "aref" to the list object
# already bound to name 'some_huge_list'
aref = some_huge_list

# you can verify that they are the same object:
assert aref is some_huge_list
# nb : in CPython, id(obj) returns the memory address of obj
# FWIW, identity test (obj1 is obj2) is the same as
# equality test on objects id (ie id(obj1) == id(obj2))
print id(aref)
print id(some_huge_list)

# mutate the list object bound to names 'aref' and 'some_huge_list'
aref[0]=0

# Now lets go a bit further and see what happens when we rebind
# some_huge_list:

some_huge_list = []

# does this impact aref ?
print aref
aref is some_huge_list

# well, obviously not.
# name 'aref' is still pointing to the same object:
print id(aref)
# but name 'some_huge_list' now points to another object:
print id(some_huge_list)

To answer your question : it works *exactly* the same way for immutable
objects:

a = 333000000000000
b = a
print b is a # True

b = 333000000000000
print b is a # False

The only difference here is that you can not alter the value of (IOW
mutate) an immutable object. So having a reference to it won't buy you
much... If you want to "share" an immutable object, you have to embed it
into a mutable container and share this container:

a = [333000000000000]
b = a
b[0] = 333000000000001
assert a is b
assert a[0] is b[0]
print a[0]

As a side note : when passing arguments to a function, the arguments
themselves are (references to) the original objects, but the names are
local to the function, so mutating an object passed as argument will
effectively impact the object (if it's mutable of course !-), but
rebinding the name inside the function won't change anything outside the
function :

def test(arg)
# really mutates the object passed in
arg[0] = 42
print "in test : arg is %s (%s)" % (arg, id(arg))

# only rebinds the local name 'arg'
arg = []
print "in test : now arg is %s (%s)" % (arg, id(arg))

def runtest():
obj = ["Life, universe and everything"]
print "in runtest : obj is %s (%s)" % (obj, id(obj))
print "calling test with obj:"
test(obj)
print "in runtest: now obj is %s (%s)" % (obj, id(obj))

Here again, if you want your function to alter the value of an immutable
object passed as argument, you have to embed it in a mutable container.
*But* this is usually useless - it's perfectly legal for a Python
function to return multiple values :

def multi(x):
return x+1, x*2

y, z = multi(42)
print "y : %s - z : %s" % (y, z)

HTH

 

[Dennis Lee Bieber]

careful: it's not the *reference* that's mutable, it's the object.

Apologies... I was, and am now, still half asleep <G>
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Magnus Lycka]

this article

http://effbot.org/zone/python-objects.htm

may be useful for those who haven't already seen it.

I don't know how many times I've referred to, or paraphrased,
that article. Shouldn't it be incorporated into the standard
tutorial? I think it's very helpful for people who are used
to the way C etc handles variables.

 

[GHUM]

Shouldn't it be incorporated into the standard tutorial?
I think it's very helpful for people who are used
to the way C etc handles variables.

That would also be a good headline. "information for those coming from
C", and it should be possible to grow similiar sections for those
comming from Java or PERL.

At least it belongs into the FAQ

Harald

 

[Simon Brunning]

I don't know how many times I've referred to, or paraphrased,
that article. Shouldn't it be incorporated into the standard
tutorial? I think it's very helpful for people who are used
to the way C etc handles variables.

I agree. These two, also:

http://effbot.org/zone/unicode-objects.htm
http://effbot.org/zone/import-confusion.htm

And probably more...