Puzzled by list-appending behavior

[Uncle Ben]

In playing with lists of lists, I found the following:

(In 3.1, but the same happens also in 2.7)

list = [1,2,3]
list.append ( [4,5,6] )
x = list
x ->
[1,2,3,[4,5,6]]
as expected.

But the shortcut fails:

list=[1,2,3]
x = list.append( [4,5,6] )
x ->
nothing

Can someone explain this to me?

Uncle Ben

 

[Chris Rebert]

In playing with lists of lists, I found the following:

(In 3.1, but the same happens also in 2.7)

list = [1,2,3]
list.append ( [4,5,6] )

Note the lack of output after this line. This indicates that
list.append([4,5,6]) returned None. Contrast this with, say,
list.pop().

x = list
x   ->
   [1,2,3,[4,5,6]]
as expected.

But the shortcut fails:

list=[1,2,3]
x = list.append( [4,5,6] )
x   ->
  nothing

Can someone explain this to me?

The append() method does *not* return the now-appended-to list. It is
a mutator method that modifies the list object in-place; per
convention, it therefore returns None to reinforce its side-effecting
nature to the user (the interactive interpreter by default does not
display None expression results); analogous methods in other languages
return void.

list.remove(), list.sort(), and list.extend() similarly return None
rather than the now-modified list.

Cheers,
Chris

 

[Chris Angelico]

In playing with lists of lists, I found the following:

(In 3.1, but the same happens also in 2.7)

list = [1,2,3]

Ben Finney has already answered the main question, but as a side
point, I would generally avoid creating a variable called 'list'.
That's the name of the type (Python 2) or class (Python 3) of all
lists, so it might result in confusion if you have an actual list with
that name.

If you want the behaviour of joining two lists (or adding something to
a list) and saving the result elsewhere, you can use the plain
addition:

a=[1,2,3]
b=a+[[4,5,6]]

Note that you have to add a list, and it will append the contents of that list.

Chris Angelico

 

[Chris Angelico]

Ben Finney has already answered the main question

Giving credit where credit's due, it was more Chris Rebert's post that
answered the question. Sorry Chris!

Chris Angelico

 

[Uncle Ben]

In playing with lists of lists, I found the following:

(In 3.1, but the same happens also in 2.7)

list = [1,2,3]
list.append ( [4,5,6] )
x = list
x   ->
    [1,2,3,[4,5,6]]
as expected.

But the shortcut fails:

list=[1,2,3]
x = list.append( [4,5,6] )
x   ->
   nothing

Can someone explain this to me?

Uncle Ben

Thank you all. It is wonderful to have this community as a resource.

Uncle Ben

 

[Chris Rebert]

Giving credit where credit's due, it was more Chris Rebert's post that
answered the question. Sorry Chris!

Eh, one can't fight chronology!

I'm just surprised that the docstring doesn't explicitly state
"Returns None." by this point, given that this is such a common point
of newbie confusion.

Cheers,
Chris

 

[MRAB]

In playing with lists of lists, I found the following:

(In 3.1, but the same happens also in 2.7)

list = [1,2,3]
list.append ( [4,5,6] )

Note the lack of output after this line. This indicates that
list.append([4,5,6]) returned None. Contrast this with, say,
list.pop().

x = list
x ->
[1,2,3,[4,5,6]]
as expected.

But the shortcut fails:

list=[1,2,3]
x = list.append( [4,5,6] )
x ->
nothing

Can someone explain this to me?

The append() method does *not* return the now-appended-to list. It is
a mutator method that modifies the list object in-place; per
convention, it therefore returns None to reinforce its side-effecting
nature to the user (the interactive interpreter by default does not
display None expression results); analogous methods in other languages
return void.

list.remove(), list.sort(), and list.extend() similarly return None
rather than the now-modified list.

I'd just like to point out that it's a convention, not a rigid rule.
Sometimes it's not followed, for example, dict.setdefault.

 

[Chris Angelico]

I'd just like to point out that it's a convention, not a rigid rule.
Sometimes it's not followed, for example, dict.setdefault.

dict.setdefault is more like dict.get but it also stores the result.
It's probably more a name issue than a protocol issue.

Chris Angelico

 

[John Ladasky]

list = [1,2,3]

Somewhat unrelated, but... is it a good idea to name your list
"list"? Isn't that the name of Python's built-in list constructor
method?

Shadowing a built-in has contributed to more than one subtle bug in my
code, and I've learned to avoid it.

 

[Terry Reedy]

And why do you insist on calling an instance of list, "list"? Even a
human reader will confuse which is which. What you are showing is an
example how confusing things become when a keyword (list) is
over-written (with list instance).

(Minor note: 'list' is not a keyword (if it were, it could not be
over-ridden) but it is a builtin.) You are correct, it is confusing.
Such usage will also lead to bugs if one ever tries to access the class
as 'list' later in the program.

Here is a legitimate usage of builtins masking:

import builtins
def list(iterable):
print('building list from {}: {}'.format(type(iterable),iterable))
return builtins.list(iterable)

a = list((1,2,3))
b = list({1,2,3})
###
building list from <class 'tuple'>: (1, 2, 3)
building list from <class 'set'>: {1, 2, 3}

 

[Prasad, Ramit]

And why do you insist on calling an instance of list, "list"? Even a

(Minor note: 'list'is not a keyword (if it were, it could not be
over-ridden) but it is a builtin.) You are correct, it is confusing.
Such usage will also lead to bugs if one ever tries to access the >class
as 'list' later in the program.

An example of overriding built-ins you *really* do not want to happen (python 2.6.4):
False

From what I can tell, this does not work in Python 3.x (at leastnot in Python 3.1.1)



Ramit



Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423


This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it isthe responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.

 

[Terry Reedy]

I'd just like to point out that it's a convention, not a rigid rule.
Sometimes it's not followed, for example, dict.setdefault.

The rule for builtin collections is that mutation methods do not return
the collection. If there is a member of the collection to return, they
return that. Otherwise they return None.

list/set.pop and dict.popitem are other mutation methods that have a
(former) member to return.

The rule applies to special methods like __getitem__ (returns an item)
and __setitem__ (returns None). Since a.append(item) is *conceptually*
equivalent to a.__setitem(len(a), item) (I know, it raises) and
*actually* defined as a.__setitem(len(a):len(a), item), it should not be
surprising that all three return None.

I think the above should be better documented.
http://bugs.python.org/issue12192
has some proposals. Comments there welcome.

In another post

I'm just surprised that the docstring doesn't explicitly state
"Returns None." by this point, given that this is such a common point
of newbie confusion.

I never noticed. After reading the above, I added this to the proposal
above.

 

[Steven D'Aprano]

list = [1,2,3]

Somewhat unrelated, but... is it a good idea to name your list "list"?
Isn't that the name of Python's built-in list constructor method?

Shadowing a built-in has contributed to more than one subtle bug in my
code, and I've learned to avoid it.

Agreed. However, there are good reasons for sometimes shadowing built-
ins, and namespaces make it safe to do so if you are sufficiently careful.

E.g. I have a module stats.sum() which shadows the built-in, but only in
that module, which is exactly the behaviour I want.

(If you do "from stats import sum", then you're responsible for whatever
happens next.)

Or you might isolate the shadow to a function small enough that you can
be sure that it won't cause any problems, e.g.:

def get(list, object):
"""Append object to a copy of list and return it."""
return list + [object]

For one or two line functions, I think that's perfectly reasonable.
Anything more than that, I'd be getting nervous.

 

[Chris Angelico]

def get(list, object):
   """Append object to a copy of list and return it."""
   return list + [object]

For one or two line functions, I think that's perfectly reasonable.
Anything more than that, I'd be getting nervous.

But even for something that short, why do it? Why not call the
parameter 'lst' or something? Shadowing with something completely
different is seldom going to give major advantage, and has the
potential to be quite confusing.

Chris Angelico

 

[Steven D'Aprano]

def get(list, object):
   """Append object to a copy of list and return it.""" return list +
   [object]

For one or two line functions, I think that's perfectly reasonable.
Anything more than that, I'd be getting nervous.

But even for something that short, why do it? Why not call the parameter
'lst' or something? Shadowing with something completely different is
seldom going to give major advantage, and has the potential to be quite
confusing.

Because "lst" is not a real word. To native readers of languages derived
from Latin or Germany, such as English, it is quite a strange "word"
because it has no vowel. In addition, it looks like 1st (first).

We use naming conventions like my_list list_ lst alist etc. to avoid
shadowing the built-in list, not because they are good to use in and of
themselves. (E.g. we don't write "my_tree" because there's no "tree" to
shadow.) All of these are ugly to some extent, which is to say, they
cause some small, or not so small, additional cognitive load to the
reader. We don't use the name "list_" because we like trailing
underscores! We do it because it avoids the cost of shadowing a built-in.

But in a small function, there's no real cost to shadowing, so why
bother? Any hypothetical confusion caused is no greater than, and
probably less than, the increased difficulty in reading any of the
alternatives.

Either way, we're talking micro-optimizations in readability here. Don't
get hung up over the choice of a name. If you'd rather never shadow, I'm
fine with that too. I just think "never shadow" is unnecessarily strict.
Sometimes shadowing is harmless, and sometimes it's useful.

If you're writing a tutorial for beginners, shadowing a built-in without
explanation will cause confusion, but for production code, I think it
reasonable to assume the reader knows Python well enough not to stumble
over that choice.

 

[Chris Angelico]

Because "lst" is not a real word. To native readers of languages derived
from Latin or Germany, such as English, it is quite a strange "word"
because it has no vowel. In addition, it looks like 1st (first).

Contrived examples are always awkward; in real examples, there's often
an alternative based on the list's purpose, which can then be used to
offer a name.

Sometimes shadowing is harmless, and sometimes it's useful.

Agreed on both those halves; and obviously, the times where it's
useful are very much important. I have to say, I do like Python's lack
of keywords for these things; the ability to shadow is a flexibility
that means that, for the most part, new builtins can be added to the
language without breaking existing code.

ChrisA

 

[MRAB]

Reminds me of the catch-phrase from the first Pirates of the Caribbean
movie: "It's more of a guideline than a rule."

Much like the Zen of Python.

 

[Prasad, Ramit]

I have to say, I do like Python's lack of keywords for these things

I thought True/False were among the list of keywords in Python 3.x ? Or are those the only keywords?


Ramit



Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423
This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflectthose of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
arehereby notified that any disclosure, copying, distribution, or
use ofthe information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affectany computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiariesand affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, pleaseimmediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.

 

[Ethan Furman]

I thought True/False were among the list of keywords in Python 3.x ? Or are those the only keywords?

http://docs.python.org/py3k/reference/lexical_analysis.html#keywords

False class finally is return
None continue for lambda try
True def from nonlocal while
and del global not with
as elif if or yield
assert else import pass
break except in raise

~Ethan~