list in a tuple

[montyphyton]

Recently, I got into a debate on programming.reddit.com about
what should happen in the following case:

a = ([1], 2)
a[0] += [3]

Currently, Python raises an error *and* changes the first element of
the tuple. Now, this seems like something one would want to
change - why raise an error *and* execute the thing it
was complaining about? The discussion seems to have no end, and
that is why I'm posting here. I would like to know what is the opinion
of the people on this group... Am I really mistaking for thinking that
this is strange and unwanted behavior? Btw I understand *why* is this
happening, I just think it should change...
And here is the post that started this discussion:
http://filoxus.blogspot.com/2007/12/python-3000-how-mutable-is-immutable.html#links

Thanks for your replies

 

[Arnaud Delobelle]

Recently, I got into a debate on programming.reddit.com about
what should happen in the following case:

a = ([1], 2)
a[0] += [3]

Currently, Python raises an error *and* changes the first element of
the tuple. Now, this seems like something one would want to
change - why raise an error *and* execute the thing it
was complaining about? The discussion seems to have no end, and
that is why I'm posting here. I would like to know what is the opinion
of the people on this group... Am I really mistaking for thinking that
this is strange and unwanted behavior? Btw I understand *why* is this
happening, I just think it should change...
And here is the post that started this discussion:http://filoxus.blogspot.com/2007/12/python-3000-how-mutable-is-immuta...

Thanks for your replies

For the sake of clarity, say you have typed:

L = [1]
a = (L, 2)
a[0] += [3]

Traceback (most recent call last):
([1, 3], 2)


I think that this is what happens when a[0] += [3] is executed:

1. list.__iadd__(L, [3]) is called, modifying L in place and returning
L.
2. tuple.__setitem__(a, L) is tried, raising the TypeError.

Notice that this problem doesn't arise when replacing lists with
tuples:

a = ( (1,), 2)
a[0] += (3, )

Traceback (most recent call last):
((1,), 2)

This is because tuple object don't have an __iadd__ method so a new
object tuple.__add__((1,),(3,)) is created and returned.

AFAICS, the only way to avoid the error message would be to check if
the new object is the same as the old one before raising the
TypeError:

class mytuple(tuple):
"It's ok to do t = obj as long as t was already obj"
def __setitem__(self, i, val):
if self is not val:
raise TypeError("'mytuple' object is immutable")

So:

a = mytuple(([1], 2))
a[0] += [3] # This now works
a[1] += 4 # This won't work as ints are immutable

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tuple.py", line 4, in __setitem__
raise TypeError("'mytuple' object is immutable")
TypeError: 'mytuple' object is immutable([1, 3], 2)

It wouldn't slow anything down I suppose, just make some currently
failing code succeed.

 

[montyphyton]

Like I said, it is clear *why* this happens, what I
am concerned is if this what we *want* to happen, i.e.,
if the current situation is satisfying. Your mytuple class
would be something that resembles a solution, my question
is what the people on this group think about it.

Recently, I got into a debate on programming.reddit.com about
what should happen in the following case:

a = ([1], 2)
a[0] += [3]

Currently, Python raises an error *and* changes the first element of
the tuple. Now, this seems like something one would want to
change - why raise an error *and* execute the thing it
was complaining about? The discussion seems to have no end, and
that is why I'm posting here. I would like to know what is the opinion
of the people on this group... Am I really mistaking for thinking that
this is strange and unwanted behavior? Btw I understand *why* is this
happening, I just think it should change...
And here is the post that started this discussion:http://filoxus.blogspot.com/2007/12/python-3000-how-mutable-is-immuta...

Thanks for your replies

For the sake of clarity, say you have typed:

L = [1]
a = (L, 2)
a[0] += [3]

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment>>> a

([1, 3], 2)

I think that this is what happens when a[0] += [3] is executed:

1. list.__iadd__(L, [3]) is called, modifying L in place and returning
L.
2. tuple.__setitem__(a, L) is tried, raising the TypeError.

Notice that this problem doesn't arise when replacing lists with
tuples:

a = ( (1,), 2)
a[0] += (3, )

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment>>> a

((1,), 2)

This is because tuple object don't have an __iadd__ method so a new
object tuple.__add__((1,),(3,)) is created and returned.

AFAICS, the only way to avoid the error message would be to check if
the new object is the same as the old one before raising the
TypeError:

class mytuple(tuple):
"It's ok to do t = obj as long as t was already obj"
def __setitem__(self, i, val):
if self is not val:
raise TypeError("'mytuple' object is immutable")

So:

a = mytuple(([1], 2))
a[0] += [3] # This now works
a[1] += 4 # This won't work as ints are immutable

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tuple.py", line 4, in __setitem__
raise TypeError("'mytuple' object is immutable")
TypeError: 'mytuple' object is immutable>>> a

([1, 3], 2)

It wouldn't slow anything down I suppose, just make some currently
failing code succeed.

 

[Arnaud Delobelle]

class mytuple(tuple):
    "It's ok to do t = obj as long as t was already obj"
    def __setitem__(self, i, val):
        if self is not val:
            raise TypeError("'mytuple' object is immutable")

So:

a = mytuple(([1], 2))
a[0] += [3] # This now works
a[1] += 4   # This won't work as ints are immutable

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "tuple.py", line 4, in __setitem__
    raise TypeError("'mytuple' object is immutable")
TypeError: 'mytuple' object is immutable>>> a

([1, 3], 2)

It wouldn't slow anything down I suppose, just make some currently
failing code succeed.

Damn! I forgot to finish my post. I wanted to add that when replacing
the inner list with a tuple, the 'mytuple' will still raise an error
for the reasons explained above:

a = mytuple(((1,), 2))
a[0] += (3,)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tuple.py", line 4, in __setitem__
raise TypeError("'mytuple' object is immutable")
TypeError: 'mytuple' object is immutable

 

[Arnaud Delobelle]

Like I said, it is clear *why* this happens, what I
am concerned is if this what we *want* to happen, i.e.,
if the current situation is satisfying. Your mytuple class
would be something that resembles a solution, my question
is what the people on this group think about it.

I understand you said that in your original post, but you didn't
explain what you thought the reason was. I've provided an explanation
(which might not be perfect) for two reasons:
* I thought it would be useful to other people reading this thread.
* It gave a motivation for the 'mytuple' class.

I'm not sure what I think about it yet

 

[Raymond Hettinger]

Recently, I got into a debate on programming.reddit.com about
what should happen in the following case:

a = ([1], 2)
a[0] += [3]

Currently, Python raises an error *and* changes the first element of
the tuple. Now, this seems like something one would want to
change - why raise an error *and* execute the thing it
was complaining about?

Yawn. Multiple actions have been combined into one line. The first
succeeds and the second fails.

If you need commit-rollback behaviors, specify them explicitly in a
try/except. My bet is that you'll figure-out that you didn't really
need that behavior to begin with. No use cluttering and slowing the
language for something like this -- Python is not SQL.


Raymond

 

[Steven D'Aprano]

Yawn. Multiple actions have been combined into one line.

And this is a good idea?

The first succeeds and the second fails.

And this is a good idea?

Shouldn't the tuple assignment raise the exception BEFORE calling
__iadd__ on the item, instead of after?

If you need commit-rollback behaviors, specify them explicitly in a
try/except. My bet is that you'll figure-out that you didn't really
need that behavior to begin with. No use cluttering and slowing the
language for something like this -- Python is not SQL.

Who said anything about commit-rollbacks?

But even if Python is not SQL, operations that half-succeed are a PITA
whenever they can occur, because you end up having to code around them in
all sorts of complicated and ugly ways. Either that, or you end up with
odd corner cases hiding bugs.

I was never a big fan of augmented assignments. I think it goes against
the Python grain: it's an implied operation, using punctuation, for the
sole (?) benefit of saving a keystroke or three.

But I think this behaviour counts as a wart on the language, rather than
a bug.

 

[Arnaud Delobelle]

And this is a good idea?

Shouldn't the tuple assignment raise the exception BEFORE calling
__iadd__ on the item, instead of after?

If you look at the bytecode generated, this doesn't seem possible:
... a = ([1],)
... a[0] += [2]
... 2 0 LOAD_CONST 1 (1)
3 BUILD_LIST 1
6 BUILD_TUPLE 1
9 STORE_FAST 0 (a)

3 12 LOAD_FAST 0 (a)
15 LOAD_CONST 2 (0)
18 DUP_TOPX 2
21 BINARY_SUBSCR
22 LOAD_CONST 3 (2)
25 BUILD_LIST 1
28 INPLACE_ADD
29 ROT_THREE
30 STORE_SUBSCR
31 LOAD_CONST 0 (None)
34 RETURN_VALUE

BINARY_SUBSCR puts a[0] on the stack, it has no way to know that a[0]
will be changed in place. To allow an exception to be thrown before
the in-place modification of a[0], there should be a new bytecode
instruction, say BINARY_SUBSCR_WITH_A_VIEW_TO_CHANGE_IN_PLACE, which
checks that the subscriptable object supports STORE_SUBSCR (;-).

[...]

I was never a big fan of augmented assignments. I think it goes against
the Python grain: it's an implied operation, using punctuation, for the
sole (?) benefit of saving a keystroke or three.

But I think this behaviour counts as a wart on the language, rather than
a bug.

Yes. I didn't realise this before you mentioned it, but the culprit
here seems to be the augmented assignment which acts differently on
mutable and immutable objects:

b = a # say a is immutable
a += c # equivalent to a = a + c
b is a # -> False

b = a # Now say a is mutable
a += c # equivalent to a.__iadd__(c)
b is a # -> True

OTOH augmented assignent are a slight optimisation:

a += 1

will look for the value of a and i only once and duplicate them on the
stack, whereas

a = a + 1

will need to resolve a and i twice (which can be costly if a and i are
globals)

 

[montyphyton]

After some tought I must agree that this is a wart more than
a bug and that it will probably be best not to mess with it.
However, what do you guys think about the print wart in Py3k
described at http://filoxus.blogspot.com/2007/12/python-3000-how-mutable-is-immutable.html#links
(im not trying to advertise my blog, I just don't feel like
typing the whole problem all over again)?

And this is a good idea?

Shouldn't the tuple assignment raise the exception BEFORE calling
__iadd__ on the item, instead of after?

If you look at the bytecode generated, this doesn't seem possible:

...     a = ([1],)
...     a[0] += [2]
...>>> import dis
  2           0 LOAD_CONST               1 (1)
              3 BUILD_LIST               1
              6 BUILD_TUPLE              1
              9 STORE_FAST               0 (a)

  3          12 LOAD_FAST                0 (a)
             15 LOAD_CONST               2 (0)
             18 DUP_TOPX                 2
             21 BINARY_SUBSCR
             22 LOAD_CONST               3 (2)
             25 BUILD_LIST               1
             28 INPLACE_ADD
             29 ROT_THREE
             30 STORE_SUBSCR
             31 LOAD_CONST               0 (None)
             34 RETURN_VALUE

BINARY_SUBSCR puts a[0] on the stack, it has no way to know that a[0]
will be changed in place.  To allow an exception to be thrown before
the in-place modification of a[0], there should be a new bytecode
instruction, say BINARY_SUBSCR_WITH_A_VIEW_TO_CHANGE_IN_PLACE, which
checks that the subscriptable object supports STORE_SUBSCR (;-).

[...]

I was never a big fan of augmented assignments. I think it goes against
the Python grain: it's an implied operation, using punctuation, for the
sole (?) benefit of saving a keystroke or three.

But I think this behaviour counts as a wart on the language, rather than
a bug.

Yes.  I didn't realise this before you mentioned it, but the culprit
here seems to be the augmented assignment which acts differently on
mutable and immutable objects:

b = a  # say a is immutable
a += c # equivalent to a = a + c
b is a # -> False

b = a  # Now say a is mutable
a += c # equivalent to a.__iadd__(c)
b is a # -> True

OTOH augmented assignent are a slight optimisation:

a += 1

will look for the value of a and i only once and duplicate them on the
stack, whereas

a = a + 1

will need to resolve a and i twice (which can be costly if a and i are
globals)

 

[Wildemar Wildenburger]

Subject: Re: list in a tuple
To:
Cc:
Bcc:
Reply-To:
Newsgroup: comp.lang.python
-=-=-=-=-=-=-=-=-=# Don't remove this line #=-=-=-=-=-=-=-=-=-

> After some tought I must agree that this is a wart more than
> a bug and that it will probably be best not to mess with it.
> However, what do you guys think about the print wart in Py3k
> described at http://filoxus.blogspot.com/2007/12/python-3000-how-mutable-is-immutable.html#links
> (im not trying to advertise my blog, I just don't feel like
> typing the whole problem all over again)?
>

From that post:

> Ok, I do admit that doing
>
> a = ([1], 2)
> a[0].append(2)
>
> also doesn't throw an error, but this only confuses me more.
>

Why? You mutate the list, but the tuple does not change. It is still a
tuple of a list and an int. At least that's how I think about it, and I
seem to recall reading that beavior justified like this (don't ask me
where though (might have been "Dive Into Python", but maybe not)).

/W

 

[montyphyton]

From that post:

Ok, I do admit that doing

a = ([1], 2)
a[0].append(2)

also doesn't throw an error, but this only confuses me more.

Why? You mutate thelist, but thetupledoes not change. It is still atupleof alistand an int. At least that's how I think about it, and I
seem to recall reading that beavior justified like this (don't ask me
where though (might have been "Dive Into Python", but maybe not)).

That part is ok, I mean it doesn't confuse me I just wanted to say
that this is somewhat confusing behavior.
I agree that its not best put... But I was thinking about the last
part of the post, the part
that talks about trying to print a tuple and getting an error.

 

[Carl Banks]

After some tought I must agree that this is a wart more than
a bug and that it will probably be best not to mess with it.
However, what do you guys think about the print wart in Py3k
described athttp://filoxus.blogspot.com/2007/12/python-3000-how-mutable-is-immuta...
(im not trying to advertise my blog, I just don't feel like
typing the whole problem all over again)?

1. Tuples are immutable. None of the tuples in your example were
modified.

The behavior you want (which is not immutability of tuples, which
Python already has, but *recursive* immutability of all objects
contained within the tuple) is not an unreasonable thing to ask for,
but the design of Python and common usage of tuples makes it all but
impossible at this point.

There is no general way to determine whether an object is mutable or
not. (Python would have to add this capability, an extremely
substantial change, to grant your wish. It won't happen.)

Tuples are used internally to represent the arguments of a function,
which are often mutable.

Tuples are sometimes used to return multiple values from a function,
which could include mutable values.

Tuples are used to specify multiple arguments to a format string, some
of which could be mutable, though I guess this is going away in Python
3.


2. The issue with print in your example is a bug, not a wart. It'll
be fixed.

(This is just a guess, but I think it might have something to do with
the flux of the new bytes type. The behavior manifested itself when
trying to print a self-referencing structure probably only because
that section of code was lagging behind.)


3. You're still top posting, which goes against this group's
conventions and annoys quite a few people. When you reply to a
message, please move your cursor to below the quoted message before
you begin typing. Thank you


Carl Banks

 

[montyphyton]

1. Tuples are immutable. None of the tuples in your example were
modified.

The behavior you want (which is not immutability of tuples, which
Python already has, but *recursive* immutability of all objects
contained within the tuple) is not an unreasonable thing to ask for,
but the design of Python and common usage of tuples makes it all but
impossible at this point.

There is no general way to determine whether an object is mutable or
not. (Python would have to add this capability, an extremely
substantial change, to grant your wish. It won't happen.)

like I said, I don't think that this behavior should be changed...
therefore, no wish-granting is needed, thank you

Tuples are used internally to represent the arguments of a function,
which are often mutable.

Tuples are sometimes used to return multiple values from a function,
which could include mutable values.

Tuples are used to specify multiple arguments to a format string, some
of which could be mutable, though I guess this is going away in Python
3.


2. The issue with print in your example is a bug, not a wart. It'll
be fixed.

(This is just a guess, but I think it might have something to do with
the flux of the new bytes type. The behavior manifested itself when
trying to print a self-referencing structure probably only because
that section of code was lagging behind.)


3. You're still top posting, which goes against this group's
conventions and annoys quite a few people. When you reply to a
message, please move your cursor to below the quoted message before
you begin typing. Thank you

sorry for top posting...

 

[Gabriel Genellina]

From that post:

Ok, I do admit that doing

a = ([1], 2)
a[0].append(2)

also doesn't throw an error, but this only confuses me more.

Why? You mutate thelist, but thetupledoes not change. It is still
atupleof alistand an int. At least that's how I think about it, and I
seem to recall reading that beavior justified like this (don't ask me
where though (might have been "Dive Into Python", but maybe not)).

That part is ok, I mean it doesn't confuse me I just wanted to say
that this is somewhat confusing behavior.
I agree that its not best put... But I was thinking about the last
part of the post, the part
that talks about trying to print a tuple and getting an error.

Instead of trying to explain it myself, I'll refer you to this little
essay [1] by Michael Hudson including some nice ASCII art, and a long
reply from Alex Martelli from which I'll quote just a few memorable
paragraphs. (Just replace "dictionary" with "tuple" in your example)

"""There is [...] a huge difference
between changing an object, and changing (mutating) some
OTHER object to which the first refers.

In Bologna over 100 years ago we had a statue of a local hero
depicted pointing forwards with his finger -- presumably to
the future, but given where exactly it was placed, the locals
soon identified it as "the statue that points to Hotel
Belfiore". The one day some enterprising developer bought
the hotel's building and restructured it -- in particular,
where the hotel used to be was now a restaurant, Da Carlo.

So, "the statue that points to Hotel Belfiore" had suddenly
become "the statue that points to Da Carlo"...! Amazing
isn't it? Considering that marble isn't very fluid and the
statue had not been moved or disturbed in any way...?

This is a real anecdote, by the way (except that I'm not
sure of the names of the hotel and restaurant involved --
I could be wrong on those), but I think it can still help
here. The dictionary, or statue, has not changed at all,
even though the objects it refers/points to may have been
mutated beyond recognition, and the name people know it
by (the dictionary's string-representation) may therefore
change. That name or representation was and is referring
to a non-intrinsic, non-persistent, "happenstance"
characteristic of the statue, or dictionary...
"""

[1] http://python.net/crew/mwh/hacks/objectthink.html

 

[vjktm]

En Thu, 27 Dec 2007 16:38:07 -0300, <[email protected]> escribió:

On Dec 27, 8:20 pm, Wildemar Wildenburger

From that post:
Ok, I do admit that doing
a = ([1], 2)
a[0].append(2)
also doesn't throw an error, but this only confuses me more.
Why? You mutate thelist, but thetupledoes not change. It is still
atupleof alistand an int. At least that's how I think about it, and I
seem to recall reading that beavior justified like this (don't ask me
where though (might have been "Dive Into Python", but maybe not)).

That part is ok, I mean it doesn't confuse me I just wanted to say
that this is somewhat confusing behavior.
I agree that its not best put... But I was thinking about the last
part of the post, the part
that talks about trying to print a tuple and getting an error.

Instead of trying to explain it myself, I'll refer you to this little
essay [1] by Michael Hudson including some nice ASCII art, and a long
reply from Alex Martelli from which I'll quote just a few memorable
paragraphs. (Just replace "dictionary" with "tuple" in your example)

"""There is [...] a huge difference
between changing an object, and changing (mutating) some
OTHER object to which the first refers.

In Bologna over 100 years ago we had a statue of a local hero
depicted pointing forwards with his finger -- presumably to
the future, but given where exactly it was placed, the locals
soon identified it as "the statue that points to Hotel
Belfiore". The one day some enterprising developer bought
the hotel's building and restructured it -- in particular,
where the hotel used to be was now a restaurant, Da Carlo.

So, "the statue that points to Hotel Belfiore" had suddenly
become "the statue that points to Da Carlo"...! Amazing
isn't it? Considering that marble isn't very fluid and the
statue had not been moved or disturbed in any way...?

This is a real anecdote, by the way (except that I'm not
sure of the names of the hotel and restaurant involved --
I could be wrong on those), but I think it can still help
here. The dictionary, or statue, has not changed at all,
even though the objects it refers/points to may have been
mutated beyond recognition, and the name people know it
by (the dictionary's string-representation) may therefore
change. That name or representation was and is referring
to a non-intrinsic, non-persistent, "happenstance"
characteristic of the statue, or dictionary...
"""

[1]http://python.net/crew/mwh/hacks/objectthink.html

Thank you very much for this discussion and reference [1] and other
pointers in [1]. This has definitely helped my understanding of such
matters.