Is this a bug?

[Michael Sparks]

Hi,


I've hit a corner case that I can explain to myself *why* it happens, both
under python 2.3 and python 2.4, but the following inconsistency makes me
wonder if I should log it as a bug:

First the behaviour that isn't unexpected:

a=["hello"]
a = a + "world"

Traceback (most recent call last):

Which is pretty what I'd expect.

However if we do this just slightly differently:

a = ["hello"]
a += "world"
a

['hello', 'w', 'o', 'r', 'l', 'd']

We get completely different behaviour. This strikes me as a bug - should I
log it as one, or is there a good reason for this behaviour?

Regards,


Michael.

 

[Robert Kern]

Hi,


I've hit a corner case that I can explain to myself *why* it happens, both
under python 2.3 and python 2.4, but the following inconsistency makes me
wonder if I should log it as a bug:

First the behaviour that isn't unexpected:

a=["hello"]
a = a + "world"

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: can only concatenate list (not "str") to list


Which is pretty what I'd expect.

However if we do this just slightly differently:

a = ["hello"]
a += "world"
a

['hello', 'w', 'o', 'r', 'l', 'd']

We get completely different behaviour. This strikes me as a bug - should I
log it as one, or is there a good reason for this behaviour?

It's consistent with using a.extend("world") which is what the += is
sugar for.

In [1]:a = ['hello']

In [2]:a.extend("world")

In [3]:a
Out[3]:['hello', 'w', 'o', 'r', 'l', 'd']

It's a *good* thing that .extend() takes any iterable without explicit
conversion to a list. I think that it's just a minor annoyance that the
behavior passes on to +=.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Grant Edwards]

a=["hello"]
a = a + "world"

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: can only concatenate list (not "str") to list

However if we do this just slightly differently:

a = ["hello"]
a += "world"
a

['hello', 'w', 'o', 'r', 'l', 'd']

We get completely different behaviour. This strikes me as a bug - should I
log it as one, or is there a good reason for this behaviour?

I think it's a bug regardless of the reason for the behavior:

1) It doesn't do what a reasonable user expects.

2) It doesn't do what the documentation says it will.
According to the language reference,

An augmented assignment expression like x += 1 can be
rewritten as x = x + 1 to achieve a similar, but not
exactly equal effect. In the augmented version, x is only
evaluated once.

I don't consider the two results you posted "similar".

 

[ajikoe]

when you use a = a + 'world' python sees it as an error because of
different type.

But when you use a += 'world'
python will change the right into list (because a is a list). So when
you're code become:
a += 'world' # a += list('world')

It really helpfull if you stick to use append instead of += when you
operate list

Pujo

 

[ajikoe]

check this site:
http://mail.python.org/pipermail/python-bugs-list/2003-November/021201.html

pujo

 

[Fredrik Lundh]

when you use a = a + 'world' python sees it as an error because of
different type.

But when you use a += 'world'
python will change the right into list (because a is a list).

"changing into a list" is a bit misleadning; to bit a bit more precise,
you may want to change that to

python will treat the right as a sequence

</F>

 

[Fredrik Lundh]

It's consistent with using a.extend("world") which is what the += is
sugar for.

In [1]:a = ['hello']

In [2]:a.extend("world")

In [3]:a
Out[3]:['hello', 'w', 'o', 'r', 'l', 'd']

It's a *good* thing that .extend() takes any iterable without explicit
conversion to a list. I think that it's just a minor annoyance that the
behavior passes on to +=.

mapping += to extend is a design mistake (I guess someone got a
little carried away).

</F>

 

[Michael Sparks]

when you use a = a + 'world' python sees it as an error because of
different type.

But when you use a += 'world'
python will change the right into list (because a is a list). So when
you're code become:
a += 'world' # a += list('world')

It really helpful if you stick to use append instead of += when you
operate list

This was part of a test case I expected to fail. I was surprised when it
passed. Passing caused a latent bug, which I fixed, and I'm preventing it's
re-occurrence by changing the way the test is written.

The value that was being updated is expected to be a _string_. However it
had the possibility of being a list of strings , *if* the code was not
making appropriate checks. The test was intended to detect that failure in
the code. The fact that f += foo succeeded where f = f + foo would fail
masked a bug. It was rewritten around moments after I discovered this, but
it struck me as rather odd.

Based on this comment from the language reference posted by Grant I'd
personally consider it to be a bug...

      "An augmented assignment expression like x += 1 can be
      rewritten as x = x + 1 to achieve a similar, but not
      exactly equal effect. In the augmented version, x is only
      evaluated once."

(I don't consider one operation succeeding and other other throwing an
exception to be similar effects)

Whilst I understand the explanations given (expected them as my post
indicated), the above comment is the one that clinches it for me.

(I also agree for example that it's good that extend takes any iterable for
example, but a PITA that += maps to extend)

That said, the bug reference mentioned above was closed with the leading
comment:

"""I think Guido said if he had it to do over, += would not be
able to take any iterable. However, that behavior has been
set in stone for several years now and it would be hard to
take away."""

https://sourceforge.net/tracker/?func=detail&atid=105470&aid=848812&group_id=5470

I'm now wondering whether it should be posted as a bug, if it's not likely
to be solved short of Python 3000. (ie whether I should just consider it a
wart instead...

(If that's really the case I'd happily consider writing a doc patch as a
warning about the behaviour)


Michael.

 

[Terry Reedy]

According to the language reference,

An augmented assignment expression like x += 1 can be
rewritten as x = x + 1 to achieve a similar, but not
exactly equal effect. In the augmented version, x is only
evaluated once.

I don't consider the two results you posted "similar".

It continues
"Also, when possible, the actual operation is performed in-place, meaning
that rather than creating a new object and assigning that to the target,
the old object is modified instead. ...Similarly, with the exception of the
possible in-place behavior, the binary operation performed by augmented
assignment is the same as the normal binary operations.
"
I take the behavior observed to be the exceptional in-place behavior
referred to. But this could certainly be clearer.

Also, Lib Ref 2.3.6.4 Mutable Sequence Types could have a line added to
the table specifying the the operation 's+=x' is the same as s.extend(x).

Terry J. Reedy

 

[Grant Edwards]

It continues
"Also, when possible, the actual operation is performed in-place, meaning
that rather than creating a new object and assigning that to the target,
the old object is modified instead. ...Similarly, with the exception of the
possible in-place behavior, the binary operation performed by augmented
assignment is the same as the normal binary operations.
"
I take the behavior observed to be the exceptional in-place behavior
referred to.

I certainly don't see how. Strings are immutable. The old
object can't be modified in-place, so the "in-place" behavior
is moot.

In any case, the only difference is supposed to be whether a
new object is created or an existing object is modified. The
two results shouldn't be completely different as shown by the
OP.

Your quote states quite clearly that the binary operation *is
the same* whether it's spelt a = a + b or a += b. That is
simply not true for the example we're discussing.

But this could certainly be clearer.

I don't see how that statement has anything to do with the bug
at hand.

 

[Robert Kern]

I certainly don't see how. Strings are immutable. The old
object can't be modified in-place, so the "in-place" behavior
is moot.

It's the left-hand-side, in this case a list, that gets modified
in-place. Whether the right-hand-side is mutable or not is irrelevant.

In any case, the only difference is supposed to be whether a
new object is created or an existing object is modified. The
two results shouldn't be completely different as shown by the
OP.

Your quote states quite clearly that the binary operation *is
the same* whether it's spelt a = a + b or a += b. That is
simply not true for the example we're discussing.

No, the quote says "with the exception of the possible in-place
behavior, the binary operation performed by augmented assignment is the
same as the normal binary operations." This is "in-place" behavior.
Badly designed "in-place" behavior, yes.

I don't see how that statement has anything to do with the bug
at hand.

It's a mistake, but it's been in the wild too long to be changed. Thus,
it should be documented.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Grant Edwards]

It's the left-hand-side, in this case a list, that gets modified
in-place. Whether the right-hand-side is mutable or not is irrelevant.

You're right. I had things backwards in my head.

No, the quote says "with the exception of the possible
in-place behavior, the binary operation performed by augmented
assignment is the same as the normal binary operations." This
is "in-place" behavior. Badly designed "in-place" behavior,
yes.

Right. The binary operation performed should be the same
regardless of whether it is done in place or not. In this
case, the "in-place" behavior is completely different than the
other spelling.

It's a mistake, but it's been in the wild too long to be changed. Thus,
it should be documented.

Life's like that, unfortunately.

 

[Matthew Dixon Cowles]

mapping += to extend is a design mistake (I guess someone got a
little carried away).

I agree. I think that + and += should be the same except for the target
of the assignment. But I discussed that with the Timbot a little while
ago and he disagrees.

Regards,
Matt

 

[Aahz]

mapping += to extend is a design mistake (I guess someone got a
little carried away).

There were two use cases that drove augmented assignment (I know you know
this -- but other people probably do not):

reallylongvariablename = reallylongvariablename + 1
hugearray = hugearray + tinyarray

The latter was particularly coming from the Numeric types. The docs
probably should clarify that augmented assignment is *NOT* necessarily
the same as ``foo = foo + bar`` when ``foo`` is a mutable type. You can
argue that you disagree with mapping ``+=`` to ``extend()``, but I don't
think it's fair for you to flatly claim that it's a design mistake.