Appending a list's elements to another list using a list comprehension

Discussion in 'Python' started by Debajit Adhikary, Oct 17, 2007.

  1. I have two lists:

    a = [1, 2, 3]
    b = [4, 5, 6]

    What I'd like to do is append all of the elements of b at the end of
    a, so that a looks like:

    a = [1, 2, 3, 4, 5, 6]

    I can do this using

    map(a.append, b)

    How do I do this using a list comprehension?

    (In general, is using a list comprehension preferable (or more
    "pythonic") as opposed to using map / filter etc.?)
    Debajit Adhikary, Oct 17, 2007
    #1
    1. Advertising

  2. Re: Appending a list's elements to another list using a listcomprehension

    On Wed, 17 Oct 2007 20:27:14 +0000, Debajit Adhikary wrote:

    > I have two lists:
    >
    > a = [1, 2, 3]
    > b = [4, 5, 6]
    >
    > What I'd like to do is append all of the elements of b at the end of
    > a, so that a looks like:
    >
    > a = [1, 2, 3, 4, 5, 6]
    >
    > I can do this using
    >
    > map(a.append, b)


    This is a bad idea as it creates a useless list of `None`\s, one for each
    element in `b`.

    > How do I do this using a list comprehension?


    Not at all. The obvious solution here is ``a.extend(b)``.

    > (In general, is using a list comprehension preferable (or more
    > "pythonic") as opposed to using map / filter etc.?)


    Some say yes.

    Ciao,
    Marc 'BlackJack' Rintsch
    Marc 'BlackJack' Rintsch, Oct 17, 2007
    #2
    1. Advertising

  3. Re: Appending a list's elements to another list using a listcomprehension

    On Wed, 2007-10-17 at 20:27 +0000, Debajit Adhikary wrote:
    > I have two lists:
    >
    > a = [1, 2, 3]
    > b = [4, 5, 6]
    >
    > What I'd like to do is append all of the elements of b at the end of
    > a, so that a looks like:
    >
    > a = [1, 2, 3, 4, 5, 6]
    >
    > I can do this using
    >
    > map(a.append, b)
    >
    > How do I do this using a list comprehension?


    You don't.

    > (In general, is using a list comprehension preferable (or more
    > "pythonic") as opposed to using map / filter etc.?)


    In general, a list comprehension is more Pythonic nowadays, but in your
    particular case the answer is neither map nor a list comprehension, it's
    this:

    a += b

    HTH,

    --
    Carsten Haese
    http://informixdb.sourceforge.net
    Carsten Haese, Oct 17, 2007
    #3
  4. Debajit Adhikary

    Paul Hankin Guest

    On Oct 17, 9:27 pm, Debajit Adhikary <> wrote:
    > I have two lists:
    >
    > a = [1, 2, 3]
    > b = [4, 5, 6]
    >
    > What I'd like to do is append all of the elements of b at the end of
    > a, so that a looks like:
    >
    > a = [1, 2, 3, 4, 5, 6]
    >
    > I can do this using
    >
    > map(a.append, b)
    >
    > How do I do this using a list comprehension?
    >
    > (In general, is using a list comprehension preferable (or more
    > "pythonic") as opposed to using map / filter etc.?)


    Yes, using a list comprehension is usually more pythonic than using
    map/filter. But here, the right answer is:
    a.extend(b). The first thing you should always do is check the python
    libraries for a function or method that does what you want. Even if
    you think you know the library quite well it's still worth checking:
    I've lost count of the number of times I've discovered a library
    function that does exactly what I wanted.

    Anyway, if extend didn't exist, the pythonic version of map(a.append,
    b) would be
    for x in b:
    a.append(x)

    Rather than
    [a.append(x) for x in b]


    List comprehensions and map produce a new list. That's not what you
    want here: you're using the side-effect of the append method - which
    modifies a. This makes using regular iteration the right idea, because
    by using map or a comprehension, you're also constructing a list of
    the return values of append (which is always None). You can see this
    in the interpreter:

    >>> map(a.append, b)

    [None, None, None]

    >>> a

    [1, 2, 3, 4, 5, 6]

    HTH

    --
    Paul Hankin
    Paul Hankin, Oct 17, 2007
    #4
  5. On Oct 17, 4:41 pm, Carsten Haese <> wrote:
    > On Wed, 2007-10-17 at 20:27 +0000, Debajit Adhikary wrote:
    > > I have two lists:

    >
    > > a = [1, 2, 3]
    > > b = [4, 5, 6]

    >
    > > What I'd like to do is append all of the elements of b at the end of
    > > a, so that a looks like:

    >
    > > a = [1, 2, 3, 4, 5, 6]

    >
    > > I can do this using

    >
    > > map(a.append, b)

    >
    > > How do I do this using a list comprehension?

    >
    > You don't.
    >
    > > (In general, is using a list comprehension preferable (or more
    > > "pythonic") as opposed to using map / filter etc.?)

    >
    > In general, a list comprehension is more Pythonic nowadays, but in your
    > particular case the answer is neither map nor a list comprehension, it's
    > this:
    >
    > a += b
    >
    > HTH,
    >
    > --
    > Carsten Haesehttp://informixdb.sourceforge.net


    Thanks a ton :)

    What in general is a good way to learn about little things like these?
    (I'm fairly new to the language)

    A google search for 'python list methods" did not turn up the +
    operator anywhere for me. Where could I find the official
    documentation for built in structures like the list? (I just noticed
    that the + operator for lists is mentioned in Beazley's Python
    Essential Reference -- in the opening pages, which I didn't look at
    when I was writing the earlier code.)

    How does "a.extend(b)" compare with "a += b" when it comes to
    performance? Does a + b create a completely new list that it assigns
    back to a? If so, a.extend(b) would seem to be faster. How could I
    verify things like these?
    Debajit Adhikary, Oct 17, 2007
    #5
  6. Debajit Adhikary

    Paul Hankin Guest

    On Oct 17, 10:03 pm, Debajit Adhikary <> wrote:
    > How does "a.extend(b)" compare with "a += b" when it comes to
    > performance? Does a + b create a completely new list that it assigns
    > back to a? If so, a.extend(b) would seem to be faster. How could I
    > verify things like these?


    Use a += b rather than a.extend(b): I'm not sure what I was thinking.
    Anyway, look at 'timeit' to see how to measure things like this, but
    my advice would be not to worry and to write the most readable code -
    and only optimise if your code's runnign too slowly.

    To answer your question though: a += b is *not* the same as a = a + b.
    The latter would create a new list and assign it to a, whereas a += b
    updates a in-place.

    --
    Paul Hankin
    Paul Hankin, Oct 17, 2007
    #6
  7. On Oct 17, 5:40 pm, Paul Hankin <> wrote:
    > To answer your question though: a += b is *not* the same as a = a + b.
    > The latter would create a new list and assign it to a, whereas a += b
    > updates a in-place.


    I know I'm being a little finicky here, but how would someone know
    that a+=b is not the same as a=a+b?
    Any documentation or reference that mentions this?


    > Use a += b rather than a.extend(b): I'm not sure what I was thinking.
    > Anyway, look at 'timeit' to see how to measure things like this, but
    > my advice would be not to worry and to write the most readable code -
    > and only optimise if your code's runnign too slowly.


    I understand. Thanks :)
    At the same time, however, as someone new learning the language, I
    feel it always helps to know what the best practices and patterns are
    at the very outset (at least for someone who chooses to become a good
    programmer in that language). I mean, for me it's like this, I just
    don't want to get the work done, I would really want to know why I do
    something a certain way and not some other way :)
    Debajit Adhikary, Oct 18, 2007
    #7
  8. Debajit Adhikary

    Robert Kern Guest

    Re: Appending a list's elements to another list using a listcomprehension

    Debajit Adhikary wrote:
    > On Oct 17, 5:40 pm, Paul Hankin <> wrote:
    >> To answer your question though: a += b is *not* the same as a = a + b.
    >> The latter would create a new list and assign it to a, whereas a += b
    >> updates a in-place.

    >
    > I know I'm being a little finicky here, but how would someone know
    > that a+=b is not the same as a=a+b?
    > Any documentation or reference that mentions this?


    http://docs.python.org/ref/augassign.html

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Oct 18, 2007
    #8
  9. Re: Appending a list's elements to another list using a listcomprehension

    On Wed, 17 Oct 2007 23:46:25 +0000, Debajit Adhikary wrote:

    > On Oct 17, 5:40 pm, Paul Hankin <> wrote:
    >> To answer your question though: a += b is *not* the same as a = a + b.
    >> The latter would create a new list and assign it to a, whereas a += b
    >> updates a in-place.

    >
    > I know I'm being a little finicky here, but how would someone know that
    > a+=b is not the same as a=a+b?
    > Any documentation or reference that mentions this?


    Have you Read The Fine Manual at the Fine Website?

    Try typing "Python documentation" into your favourite search engine, or
    try going to http://www.python.org/ which links directly to
    http://www.python.org/doc/

    Unfortunately, searching for operators is always tricky (try searching
    for "*" some day...) but the section you want is here:

    http://docs.python.org/ref/augassign.html


    Also, it isn't always true that a += b is not the same as a = a+b. It
    depends on what a and b are.


    --
    Steven.
    Steven D'Aprano, Oct 18, 2007
    #9
  10. Re: Appending a list's elements to another list using a listcomprehension

    On Wed, 17 Oct 2007 21:40:40 +0000, Paul Hankin wrote:

    > On Oct 17, 10:03 pm, Debajit Adhikary <> wrote:
    >> How does "a.extend(b)" compare with "a += b" when it comes to
    >> performance? Does a + b create a completely new list that it assigns
    >> back to a? If so, a.extend(b) would seem to be faster. How could I
    >> verify things like these?

    >
    > Use a += b rather than a.extend(b): I'm not sure what I was thinking.


    Neither am I. Why do you say that?


    > Anyway, look at 'timeit' to see how to measure things like this, but my
    > advice would be not to worry and to write the most readable code - and
    > only optimise if your code's runnign too slowly.


    Always good advice, but of course what a person considers "the most
    readable code" changes with their experience.


    > To answer your question though: a += b is *not* the same as a = a + b.



    It might be. It depends on what a and b are.



    --
    Steven
    Steven D'Aprano, Oct 18, 2007
    #10
  11. Re: Appending a list's elements to another list using a listcomprehension

    En Wed, 17 Oct 2007 18:03:56 -0300, Debajit Adhikary <>
    escribió:

    > What in general is a good way to learn about little things like these?
    > (I'm fairly new to the language)
    >
    > A google search for 'python list methods" did not turn up the +
    > operator anywhere for me. Where could I find the official
    > documentation for built in structures like the list?


    Operators are hard to find... For the available list methods, see
    http://docs.python.org/lib/typesseq.html and
    http://docs.python.org/lib/typesseq-mutable.html
    From the Library Reference, you should read at least section 2 (Built-in
    Objects) and section 3 (Built-in types), and the remaining section titles
    so you know they exist at least.

    --
    Gabriel Genellina
    Gabriel Genellina, Oct 18, 2007
    #11
  12. Re: Appending a list's elements to another list using a listcomprehension

    Paul Hankin <> writes:

    > On Oct 17, 10:03 pm, Debajit Adhikary <> wrote:
    >> How does "a.extend(b)" compare with "a += b" when it comes to
    >> performance? Does a + b create a completely new list that it assigns
    >> back to a? If so, a.extend(b) would seem to be faster. How could I
    >> verify things like these?

    >
    > Use a += b rather than a.extend(b): I'm not sure what I was
    > thinking.


    Why? a.extend(b) is at least as readable, and is guaranteed to extend
    the same list. In general, "a += b" can fall back to the slower "a =
    a + b" for sequences that don't support +=.
    Hrvoje Niksic, Oct 18, 2007
    #12
  13. Debajit Adhikary

    Paul Hankin Guest

    On Oct 18, 10:21 am, Hrvoje Niksic <> wrote:
    > Paul Hankin <> writes:
    > > On Oct 17, 10:03 pm, Debajit Adhikary <> wrote:
    > >> How does "a.extend(b)" compare with "a += b" when it comes to
    > >> performance? Does a + b create a completely new list that it assigns
    > >> back to a? If so, a.extend(b) would seem to be faster. How could I
    > >> verify things like these?

    >
    > > Use a += b rather than a.extend(b): I'm not sure what I was
    > > thinking.

    >
    > Why? a.extend(b) is at least as readable, and is guaranteed to extend
    > the same list. In general, "a += b" can fall back to the slower "a =
    > a + b" for sequences that don't support +=.


    Not to me: I can never remember which of a.append and a.extend is
    which. Falling back to a = a + b is exactly what you want. For
    instance:

    a = (1, 2, 3)
    a += (4, 5, 6)

    works, whereas:

    a = (1, 2, 3)
    a.extend((4, 5, 6))

    doesn't. So using += makes your code more general. There's no standard
    sequence type that has extend and not +=, so worrying that += is
    slower isn't a concern unless you're using a user-defined class. Even
    then, it's probably a mistake that should be fixed in the class rather
    than requiring .extend() to be used instead of +=.

    --
    Paul Hankin
    Paul Hankin, Oct 18, 2007
    #13
  14. Debajit Adhikary a écrit :
    > I have two lists:
    >
    > a = [1, 2, 3]
    > b = [4, 5, 6]
    >
    > What I'd like to do is append all of the elements of b at the end of
    > a, so that a looks like:
    >
    > a = [1, 2, 3, 4, 5, 6]
    >
    > I can do this using
    >
    > map(a.append, b)


    And what about a.extend(b) ?

    > How do I do this using a list comprehension?


    Why would you want a list comp here ???

    > (In general, is using a list comprehension preferable (or more
    > "pythonic") as opposed to using map / filter etc.?)


    Depends. Anyway, the pythonic solution here is to use the appropriate
    list method !-)
    Bruno Desthuilliers, Oct 18, 2007
    #14
  15. Re: Appending a list's elements to another list using a listcomprehension

    On Thu, Oct 18, 2007 at 11:57:10AM -0000, Paul Hankin wrote regarding Re: Appending a list's elements to another list using a list comprehension:
    >
    > Not to me: I can never remember which of a.append and a.extend is
    > which. Falling back to a = a + b is exactly what you want. For
    > instance:
    >
    > a = (1, 2, 3)
    > a += (4, 5, 6)
    >
    > works, whereas:
    >
    > a = (1, 2, 3)
    > a.extend((4, 5, 6))
    >
    > doesn't. So using += makes your code more general. There's no standard
    > sequence type that has extend and not +=, so worrying that += is
    > slower isn't a concern unless you're using a user-defined class. Even
    > then, it's probably a mistake that should be fixed in the class rather
    > than requiring .extend() to be used instead of +=.
    >


    I was going to argue that in fact += is not more general, it just covers a different set of use cases, but then I tested my hypothesis...

    >>> a = [1,2,3]
    >>> b = a
    >>> c = [4,5,6]
    >>> d = c
    >>> e = [7,8,9]
    >>> a.extend(e)
    >>> b

    [1, 2, 3, 7, 8, 9]
    >>> c += e
    >>> d # I expected [4, 5, 6]

    [4, 5, 6, 7, 8, 9]
    >>> c = c + e # But += doesn't do the same as this
    >>> c

    [4, 5, 6, 7, 8, 9, 7, 8, 9]
    >>> d

    [4, 5, 6, 7, 8, 9]

    So I learned something new.

    Cheers,
    Cliff
    J. Clifford Dyer, Oct 18, 2007
    #15
  16. Re: Appending a list's elements to another list using a listcomprehension

    Paul Hankin <> writes:

    > Not to me: I can never remember which of a.append and a.extend is
    > which.


    Interesting, with me it's the other way around. Maybe it's because I
    used Python before extend was available.

    > Falling back to a = a + b is exactly what you want.


    Not if you want to mutate the original object, possibly referenced
    from elsewhere.
    Hrvoje Niksic, Oct 18, 2007
    #16
  17. Debajit Adhikary <> wrote:
    ...
    > How does "a.extend(b)" compare with "a += b" when it comes to
    > performance? Does a + b create a completely new list that it assigns
    > back to a? If so, a.extend(b) would seem to be faster. How could I
    > verify things like these?


    That's what the timeit module is for, but make sure that the snippet
    you're timing has no side effects (since it's repeatedly executed).
    E.g.:

    brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]'
    'a=z[:];a.extend(b)'
    1000000 loops, best of 3: 0.769 usec per loop
    brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]' 'a=z[:];a+=b'
    1000000 loops, best of 3: 0.664 usec per loop
    brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]'
    'a=z[:];a.extend(b)'
    1000000 loops, best of 3: 0.769 usec per loop
    brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]' 'a=z[:];a+=b'
    1000000 loops, best of 3: 0.665 usec per loop
    brain:~ alex$

    The repetition of the measurements show them very steady, so now you
    know that += is about 100 nanoseconds faster (on my laptop) than extend
    (the reason is: it saves the tiny cost of looking up 'extend' on a; to
    verify this, use much longer lists and you'll notice that while overall
    times for both approaches increase, the difference between the two
    approaches remains about the same for lists of any length).

    But the key point to retain is: make sure that the snippet is free of
    side effects, so that each of the MANY repetitions that timeit does is
    repeating the SAME operation. If we initialized a in the -s and then
    just extended it in the snippet, we'd be extending a list that keeps
    growing at each repetition -- a very different operation than extending
    a list of a certain fixed starting length (here, serendipitously, we'd
    end up measuring the same difference -- but in the general case, where
    timing difference between approaches DOES depend on the sizes of the
    objects involved, our measurements would instead become meaningless).

    Therefore, we initialize in -s an auxiliary list, and copy it in the
    snippet. That's better than the more natural alternative:

    brain:~ alex$ python -mtimeit 'a=[1,2,3];a+=[4,5,6]'
    1000000 loops, best of 3: 1.01 usec per loop
    brain:~ alex$ python -mtimeit 'a=[1,2,3];a.extend([4,5,6])'
    1000000 loops, best of 3: 1.12 usec per loop
    brain:~ alex$ python -mtimeit 'a=[1,2,3];a+=[4,5,6]'
    1000000 loops, best of 3: 1.02 usec per loop
    brain:~ alex$ python -mtimeit 'a=[1,2,3];a.extend([4,5,6])'
    1000000 loops, best of 3: 1.12 usec per loop

    as in this "more natural alternative" we're also paying each time
    through the snippet the cost of building the literal lists; this
    overhead (which is a lot larger than the difference we're trying to
    measure!) does not DISTORT the measurement but it sure OBSCURES it to
    some extend (losing us about one significant digit worth of difference
    in this case). Remember, the WORST simple operation you can do in
    measurement is gauging a small number delta as the difference of two
    much larger numbers X and X+delta... so, make X as small as feasible to
    reduce the resulting loss of precision!-)

    You can find more details on commandline use of timeit at
    <http://docs.python.org/lib/node808.html> (see adjacent nodes in Python
    docs for examples and details on the more advanced use of timeit inside
    your own code) but I hope these indications may be of help anyway.


    Alex
    Alex Martelli, Oct 18, 2007
    #17
  18. On Oct 18, 9:47 am, (Alex Martelli) wrote:
    > Debajit Adhikary <> wrote:
    >
    > > How does "a.extend(b)" compare with "a += b" when it comes to
    > > performance? Does a + b create a completely new list that it assigns
    > > back to a? If so, a.extend(b) would seem to be faster. How could I
    > > verify things like these?

    >
    > That's what the timeit module is for, but make sure that the snippet
    > you're timing has no side effects (since it's repeatedly executed).
    > E.g.:
    >
    > brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]'
    > 'a=z[:];a.extend(b)'
    > 1000000 loops, best of 3: 0.769 usec per loop
    > brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]' 'a=z[:];a+=b'
    > 1000000 loops, best of 3: 0.664 usec per loop
    > brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]'
    > 'a=z[:];a.extend(b)'
    > 1000000 loops, best of 3: 0.769 usec per loop
    > brain:~ alex$ python -mtimeit -s'z=[1,2,3];b=[4,5,6]' 'a=z[:];a+=b'
    > 1000000 loops, best of 3: 0.665 usec per loop
    > brain:~ alex$
    >
    > The repetition of the measurements show them very steady, so now you
    > know that += is about 100 nanoseconds faster (on my laptop) than extend
    > (the reason is: it saves the tiny cost of looking up 'extend' on a; to
    > verify this, use much longer lists and you'll notice that while overall
    > times for both approaches increase, the difference between the two
    > approaches remains about the same for lists of any length).


    > You can find more details on commandline use of timeit at
    > <http://docs.python.org/lib/node808.html> (see adjacent nodes in Python
    > docs for examples and details on the more advanced use of timeit inside
    > your own code) but I hope these indications may be of help anyway.


    Thanks for the wonderful explanation on timeit.
    Thats one more tool now in my arsenal :p
    Debajit Adhikary, Oct 18, 2007
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    3,701
    Thomas 'PointedEars' Lahn
    Dec 2, 2005
  2. Red Ogden
    Replies:
    0
    Views:
    1,085
    Red Ogden
    Jul 16, 2003
  3. Ron Adam

    Appending Elements in Element Tree

    Ron Adam, Apr 8, 2006, in forum: Python
    Replies:
    1
    Views:
    301
    Ron Adam
    Apr 8, 2006
  4. Adam Hartshorne
    Replies:
    2
    Views:
    364
    Nitin Motgi
    Jan 27, 2006
  5. Vedran Furac(
    Replies:
    4
    Views:
    317
    Marc 'BlackJack' Rintsch
    Dec 19, 2008
Loading...

Share This Page