Is behavior of += intentional for int?

R

Rhodri James

Very expressive.

I use python many years. And many years I just took python int as they
are.
I am also not think about names as reference to objects and so on.

Then you are doomed to surprises such as this.
 
D

Derek Martin

Python binds values to names. Always.

No, actually, it doesn't. It binds *objects* to names. This
distinction is subtle, but important, as it is the crux of why this is
confusing to people. If Python is to say that objects have values,
then the object can not *be* the value that it has, because that is a
paradoxical self-reference. It's an object, not a value.
Is it any odder that 3 is an object than that the string literal
"Hello, World!" is an object?

Yes. Because 3 is a fundamental bit of data that the hardware knows
how to deal with, requiring no higher level abstractions for the
programmer to use it (though certainly, a programming language can
provide them, if it is convenient). "Hello, World!" is not. They are
fundamentally different in that way.
For a Python long-timer like Mr. D'Aprano, I don't think he even
consciously thinks about this kind of thing any more; his intuition
has aligned with the Python stars, so he extrapolates from the OP's
suggestion to the resulting aberrant behavior, as he posted it.

I'm sure that's the case. But it's been explained to him before, and
yet he still can't seem to comprehend that not everyone immediately
gets this behavior, and that this is not without good reason.

So, since it *has* been explained to him before, it's somewhat
astonishing that he would reply to zaur's post, saying that the
behavior zaur described would necessarily lead to the insane behavior
that Steven described. When he makes such statements, it's tantamount
to calling the OP an idiot. I find that offensive, especially
considering that Steven's post displayed an overwhelming lack of
understanding of what the OP was trying to say.
You can dispute and rail at this core language concept if you like,
but I think the more entrenched you become in the position that "'3 is
an object' is bizarre", the less enjoyable your Python work will be.

While I did genuinely find the behavior bizarre when I encountered it,
and honestly still do, I learned it quickly and moved past it. I'm
not suggesting that it be changed, and I don't feel particularly
strongly that it even should change. It's not so much the language
I'm railing against, but the humans...

--
Derek D. Martin
http://www.pizzashack.org/
GPG Key ID: 0x81CFE75D


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFKmrEtdjdlQoHP510RAoo+AJ4r9uwiiplN6u654FAvbHsf3jzbygCfbcUI
k6lFEODpM1/EUc89ooIuhsw=
=gsPv
-----END PGP SIGNATURE-----
 
D

Derek Martin

Of course, just as I was typing my response, Steve D'Aprano beat me to
the punch.

Intuition means "The power or faculty of attaining to direct knowledge
or cognition without evident rational thought and inference." Very
naturally, things which behave in a familiar manner are intuitive.
Familiar and intuitive are very closely tied. Correspondingly, when
things look like something familiar, but behave differently, they are
naturally unintuitive.

--
Derek D. Martin
http://www.pizzashack.org/
GPG Key ID: 0x81CFE75D


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFKmraddjdlQoHP510RAiflAJsGdFBpUqGAobb54Uo0fEg2nUgAEgCfU3bU
kA3D0Dgv8wMd3Hl+aP47BPw=
=uW36
-----END PGP SIGNATURE-----
 
D

Derek Martin

[snip rant]

I was not ranting. I was explaining a perspective.
You mean it's different from how you first learned it.

I mean exactly that I find it "strikingly out of the ordinary; odd,
extravagant, or eccentric in style or mode" as Webster's defines the
word. Whether it is so because it is different from how I first
learned it, or for some other reason, it is so nonetheless. I have
elsewhere gone into great detail about why I find it so. If you need
it to be simple, then feel free to simplify it.

--
Derek D. Martin
http://www.pizzashack.org/
GPG Key ID: 0x81CFE75D


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFKmriRdjdlQoHP510RAu+fAJ42GYwuCZSMKvLIlaeVcDr714SeYgCeLJgI
9ttbM3oLxhT9QWpMtq99nqM=
=D+0b
-----END PGP SIGNATURE-----
 
O

OKB (not okblacke)

Derek said:
If Python is to say that objects have values,
then the object can not *be* the value that it has, because that is a
paradoxical self-reference. It's an object, not a value.

But does it say that objects have values? I don't see where you
get this idea. Consider this code:

class A(object):
pass

class B(object):
x = 0

a = A()
b = B()
b2 = B()
b2.x = a

What is the "value" of the object now bound to the name "a"? What
about the "value" of the object bound to b, or b2?

I would say that in Python, objects do not have values. Objects
are values.

--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown
 
D

Derek Martin

But does it say that objects have values? I don't see where you
get this idea.

Yes, it does say that. Read the docs. :)

http://docs.python.org/reference/datamodel.html

(paragraph 2)

class A(object):
pass
a = A()

What is the "value" of the object now bound to the name "a"

In Python, the value of objects depends on the context in which it is
evaluated. But when you do that, you're not getting a value that is
equivalent to object, but of some property of the object. The object
has no intrinsic value until it is evaluated. In that sense, and as
used by the python docs, I would say that the value of the object a is
"true" -- you can use it in boolean expressions, and it will evaluate
as such.
I would say that in Python, objects do not have values.
Objects are values.

You can say that, but if you do you're using some definition of
"value" that's only applicable in Python and programming languages
which behave the same way. It would be more correct to say that an
object is a collection of arbitrary data, which has a type and an
identity, and that the data in that collection has a value that
evaluates in context. An object is an abstract collection of data,
and abstractions have no value. You can not measure them in any
meaningful way. The data contained in the collection does, however,
have a value. When you reference an object in an expression, what you
get is not the value of the object, but the value of some peice of
data about, or contained in, that object.

It is this property of objects, that the value evaluated depends on
the context, that I think demonstrates that an object is *not* a
value. Values never change, as we've said in this thread: 3 is always
3. 'a' is always 'a'. But an object x can evaluate to many different
values, depending on how it is used. The definition of the object
would need to allow for it to do so, but Python allows that, and even
encourages it.

--
Derek D. Martin
http://www.pizzashack.org/
GPG Key ID: 0x81CFE75D


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFKmsMVdjdlQoHP510RArDPAJ9FMUXoVhUUQ5l+OdHbZZFcBkHOhgCdH+3m
qpfkexljGlO6QxZQRJeKsRo=
=B/bL
-----END PGP SIGNATURE-----
 
S

Steven D'Aprano

No, actually, it doesn't. It binds *objects* to names. This
distinction is subtle, but important, as it is the crux of why this is
confusing to people. If Python is to say that objects have values, then
the object can not *be* the value that it has, because that is a
paradoxical self-reference. It's an object, not a value.

You're thinking about this too hard and tying yourself in knots trying to
philosophise about it. In context of OO programming, the distinction
between objects and values is fuzzy, and it depends on the context: e.g.
if I have this:

class MyInt(int):
pass

five = MyInt(5)
five.thingy = 23

is thingy part of the value of the object or not?

For most objects, at least for built-ins, the object *is* the value. (For
custom classes you create yourself, you are free to do anything you
like.) There's no need to try to distinguish between the object 3 and the
value of the object 3: you're looking for a distinction that simply
doesn't matter.

Yes. Because 3 is a fundamental bit of data that the hardware knows how
to deal with, requiring no higher level abstractions for the programmer
to use it (though certainly, a programming language can provide them, if
it is convenient). "Hello, World!" is not. They are fundamentally
different in that way.

Nonsense on two levels.

Firstly, in Python, *both* 3 and "Hello World" are complex objects, and
neither are even close to the fundamental bits of data that the hardware
can deal with.

Secondly, in low level languages, both are nothing but a sequence of
bytes, and hardware knows how to deal with bytes regardless of whether
they are interpreted by the human reader as 3 or "Hello World". The
compiler might stop you from adding 2371 to "Hell" or "o Wor" but the
hardware would be perfectly happy to do so if asked.


I'm sure that's the case. But it's been explained to him before, and
yet he still can't seem to comprehend that not everyone immediately gets
this behavior, and that this is not without good reason.

Oh, it's obvious that not everybody gets this behaviour. I understand
full well that it's different to some other languages, but not all, and
so some people have their expectations violated.

So, since it *has* been explained to him before, it's somewhat
astonishing that he would reply to zaur's post, saying that the behavior
zaur described would necessarily lead to the insane behavior that Steven
described. When he makes such statements, it's tantamount to calling
the OP an idiot.

Given Python's programming model and implementation, the behaviour asked
for *would* lead to the crazy behaviour I described.

(For the record, some early implementations of Fortran allowed the user
to redefine literals like that, and I'm told that Lisp will do so too.)

So what Zaur presumably saw as a little tiny difference is in fact the
tip of the iceberg of a fairly major difference. I don't know how smart
he is, but I'd be willing to bet he hadn't thought through the full
consequences of the behaviour he'd prefer, given Python's execution
model. You could make a language that behaved as he wants, and I wouldn't
be surprised if Java was it, but whatever it is, it isn't Python.

I find that offensive,

It's moments like this that I am reminded of a quote from Stephen Fry:

"You're offended? So f***ing what?"

Taking offense at an intellectual disagreement over the consequences of
changes to a programming model is a good sign that you've got no rational
argument to make and so have to resort to (real or pretend) outrage to
win points.

especially considering that
Steven's post displayed an overwhelming lack of understanding of what
the OP was trying to say.

I'm pretty sure I do understand what the OP was trying to say. He
actually managed to communicate it very well. I think he expects to be
able to do this:
123456

Do you disagree? What do *you* think he wants?
 
D

Dennis Lee Bieber

No, actually, it doesn't. It binds *objects* to names. This

Personally, I don't even see it that way... I see Python binding
/names/ to objects (the name is being attached to the object, and the
object can have more than one name attached at any time. "binding
objects to names" seems to imply that multiple objects could be attached
to a single name at any given moment.
 
C

Carl Banks

Intuition means "The power or faculty of attaining to direct knowledge
or cognition without evident rational thought and inference."  Very
naturally, things which behave in a familiar manner are intuitive.
Familiar and intuitive are very closely tied.  Correspondingly, when
things look like something familiar, but behave differently, they are
naturally unintuitive.

*You* find something unfamiliar, and by your logic that means it's
unintuitive for everyone?

Nice logic there, chief. Presumptuous much?


Carl Banks
 
P

Piet van Oostrum

Derek Martin said:
DM> On Sun, Aug 30, 2009 at 03:42:06AM -0700, Paul McGuire wrote:
DM> Yes. Because 3 is a fundamental bit of data that the hardware knows
DM> how to deal with, requiring no higher level abstractions for the
DM> programmer to use it (though certainly, a programming language can
DM> provide them, if it is convenient). "Hello, World!" is not. They are
DM> fundamentally different in that way.

How the semantics of Python is defined is not dependent of the hardware.
You could imagine a computer where strings are as much built into the
hardware as ints are. On the other hand you could also imagine a
computer that only knows about bits and where int arithmetic has to be
done in software. In fact early microprocessors only could operate
directly on bytes and arithmetic of larger ints was in software. And
even nowadays there are processors in use that don't have built-in
floating point hardware. Would you say that considering whether 3.14 is
an object in Python or whether that is to be considered strange should
depend on the availability of a floating point unit in the hardware
where the program runs? Would that make floating point numbers
"fundamentally different" from ints in the sense described above?
 
Z

zaur

Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "copyright", "credits" or "license()" for more information.>>> a=1
x=[a]
id(a)==id(x[0]) True
a+=1
a 2
x[0]

1

I thought that += should only change the value of the int object. But
+= create new.
Is this intentional?

As a result of this debate is not whether we should conclude that
there should be two types of integers in python: 1) immutable numbers,
which behave as constant value; 2) mutable numbers, which behave as
variable value?
 
C

Carl Banks

Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "copyright", "credits" or "license()" for more information.>>> a=1
x=[a]
id(a)==id(x[0]) True
2

I thought that += should only change the value of the int object. But
+= create new.
Is this intentional?

As a result of this debate is not whether we should conclude that
there should be two types of integers in python: 1) immutable numbers,
which behave as constant value; 2) mutable numbers, which behave as
variable value?

You are free to use third-party modules (such as numpy) which provide
mutable numbers.

I see no reason to include any mutable number type standard library,
as not many people will require the performance benefits and/or
indirection of mutable numbers, and those who do are free to use third-
party modules that provide them.


Carl Banks
 
S

Steven D'Aprano

As a result of this debate is not whether we should conclude that there
should be two types of integers in python: 1) immutable numbers, which
behave as constant value; 2) mutable numbers, which behave as variable
value?

What can you do with mutable numbers that you can't do with immutable
ones, and why do you want to do it?
 
Z

zaur

What can you do with mutable numbers that you can't do with immutable
ones, and why do you want to do it?

Mutable numbers acts as variable quantity. So when augmented
assignment is used there is no need to create a new number object in
every binary operation.

But when I looked now into source of python int (longobject.c) I
realized that direct implementation of mutable int will not give any
benefit against defining proxy int class, which supports mutability.
 
P

Piet van Oostrum

zaur said:
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "copyright", "credits" or "license()" for more information.>>> a=1
x=[a]
id(a)==id(x[0])
True
a+=1
a
2
x[0]

1

I thought that += should only change the value of the int object. But
+= create new.
Is this intentional?
z> As a result of this debate is not whether we should conclude that
z> there should be two types of integers in python: 1) immutable numbers,
z> which behave as constant value; 2) mutable numbers, which behave as
z> variable value?

Numbers are immutable by nature (math). The number 3.14 remains 3.14
whatever you try to do with it. What you call an immutable number is in
fact a container that contains a number. You can change the contents of
the container, not by modifying the number in it but by replacing it
with a different number. Python has sufficient mechanisms for creating
these containers: lists, dictionaries, objects. If you think they are
not good enough then write a new one in C.
 
S

Steven D'Aprano

Mutable numbers acts as variable quantity.

So do immutable numbers bound to a name.

So when augmented assignment
is used there is no need to create a new number object in every binary
operation.

"No need", sure, but there's no *need* to use object oriented code in the
first place, or garbage collectors, or high level languages, or even
functions. People got by with GOTO and assembly for years :) We use all
these things because they make *programming* easier, even if it adds
runtime overhead.

I'm asking what *problem* you are trying to solve with mutable numbers,
where immutable numbers are not satisfactory. The only answer I can
imagine is that you're worried about the overhead of creating new integer
objects instead of just flipping a few bits in an existing integer
variable.
 
C

Carl Banks

Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "copyright", "credits" or "license()" for more information.>>> a=1
x=[a]
id(a)==id(x[0])
True
a+=1
a
2
x[0]
1
I thought that += should only change the value of the int object. But
+= create new.
Is this intentional?
z> As a result of this debate is not whether we should conclude that
z> there should be two types of integers in python: 1) immutable numbers,
z> which behave as constant value; 2) mutable numbers, which behave as
z> variable value?

Numbers are immutable by nature (math). The number 3.14 remains 3.14
whatever you try to do with it. What you call an immutable number is in
fact a container that contains a number.

I wouldn't agree with that terminology or logic.

First of all "mutable number" is really just a short way to say
"mutable number object". A number object in Python is not a number,
it's just a representation of a number. Even if numbers are immutable
by nature, an object representing a number need not be.

And if your number object is mutable, it does not make that object a
container, at least not what I would call a container. A container
you have to dereference somehow to get at the object inside, whereas a
mutable number object you don't dereference: it acts like number as-
is.

IOW, the first example below is a container, the second is not:

num = [2]
num[0] += 3

num = mutable_int(2)
num += 3

(If you want to call the mutable number a container anyway, fine with
me, I am not here to bicker.) A container is sufficient to get a
layer of indirection if that's what you want the mutable number for.

However, the mutable number has a performance advantage over using a
container: it avoids the overhead of creating a new object. If you
were using += in a loop like this, it could turn out to be significant
savings. But since that's not common in Python I'd have to agree that
this optimization opportunity is best done with a third-party C-
extension, and not the standard library.


Carl Banks
 
T

Terry Reedy

Steven said:
I'm asking what *problem* you are trying to solve with mutable numbers,
where immutable numbers are not satisfactory. The only answer I can
imagine is that you're worried about the overhead of creating new integer
objects instead of just flipping a few bits in an existing integer
variable.

Of course, *because ints are immutable*, an implementation can avoid the
overhead of object creation for common cases by creating an array of
small integers.

CPython currently does this for -10 (or -5?) to about 256 or 257. I
would not be surprised if this coevers at least 80% of int.__new__ requests.

tjr
 
S

Steven D'Aprano

I wouldn't agree with that terminology or logic.

Abstract numbers aren't really "things", they're concepts, and as such I
don't think mutable or immutable has any meaning. But if it does, then
abstract numbers are immutable, because you can't change one into two
without losing the value one, and that certainly doesn't happen.

First of all "mutable number" is really just a short way to say "mutable
number object".

Yes, but "object" in the above doesn't necessarily mean object in the
sense of object-oriented programming. Perhaps a better word is "mutable
number thing", where "thing" could be an object in the OOP sense, or a
variable in the memory-location sense, or on a stack, in a register, or
chalk marks on a whiteboard.

A number object in Python is not a number, it's just a
representation of a number. Even if numbers are immutable by nature, an
object representing a number need not be.

/s/number/abstract number/


Given that, then I agree, but given Python's implementation, you would
need more than one change to make number's mutable, and what you would
have wouldn't be Python any more.

And if your number object is mutable, it does not make that object a
container, at least not what I would call a container. A container you
have to dereference somehow to get at the object inside, whereas a
mutable number object you don't dereference: it acts like number as- is.

Agreed, so long as we're talking from the programmer's perspective. From
the compiler's perspective, or the Virtual Machine's perspective,
variables of the type "value stored at memory location" are a kind of
container: you don't dereference anything, but the compiler does.

There are multiple perspectives which are valid at once, depending on
where you're looking from. A *name* is kind of a container (but not in
the same sense that lists and dicts are containers), in that the VM has
to dereference the name to get to the object.

IOW, the first example below is a container, the second is not:

Do you mean the name `num` is a container, or the *contents* of `num` is
a container? I assume you mean the contents.

num = [2]
num[0] += 3

num = mutable_int(2)
num += 3

In the sense of container that you give, I agree. But there's nothing
magic about mutable_int() in the above -- num could be bound to a regular
int, and it still wouldn't be a container. Obviously.


(If you want to call the mutable number a container anyway, fine with
me, I am not here to bicker.) A container is sufficient to get a layer
of indirection if that's what you want the mutable number for.

However, the mutable number has a performance advantage over using a
container: it avoids the overhead of creating a new object.

You're assuming that the overhead of flipping bits is less than the
overhead of creating a new object, and that's almost certainly valid if
the VM is implemented in (say) C. But it may not be the case if the VM is
implemented in (say) Python, or Javascript, in which case "create a new
object" might be less expensive than "flip a bunch of bits". If you can
get the CPU to flip your bits, that will be fast, but perhaps your
implementation needs to convert the int to a string of ones and zeroes,
then perform bit-operations on those, then convert back to a real int.

(I mention this as a theoretical issue, not a serious objection to the
existing CPython implementation. But perhaps the PyPy people have to deal
with it?)
 
P

Piet van Oostrum

CB> I wouldn't agree with that terminology or logic.
CB> First of all "mutable number" is really just a short way to say
CB> "mutable number object". A number object in Python is not a number,
CB> it's just a representation of a number. Even if numbers are immutable
CB> by nature, an object representing a number need not be.

Yes, that remark I made was more on the meta level.
CB> And if your number object is mutable, it does not make that object a
CB> container, at least not what I would call a container. A container
CB> you have to dereference somehow to get at the object inside, whereas a
CB> mutable number object you don't dereference: it acts like number as-
CB> is.

If you read the OP, that was exactly what the OP (human) expected. That
the list would contain some kind of reference to the number. And then of
course you would have to dereference it to get the number.
CB> IOW, the first example below is a container, the second is not:
CB> num = [2]
CB> num[0] += 3
CB> num = mutable_int(2)
CB> num += 3
CB> (If you want to call the mutable number a container anyway, fine with
CB> me, I am not here to bicker.) A container is sufficient to get a
CB> layer of indirection if that's what you want the mutable number for.
CB> However, the mutable number has a performance advantage over using a
CB> container: it avoids the overhead of creating a new object. If you
CB> were using += in a loop like this, it could turn out to be significant
CB> savings. But since that's not common in Python I'd have to agree that
CB> this optimization opportunity is best done with a third-party C-
CB> extension, and not the standard library.

That's an implementation detail.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,779
Messages
2,569,606
Members
45,239
Latest member
Alex Young

Latest Threads

Top