removing list comprehensions in Python 3.0

Discussion in 'Python' started by Steven Bethard, Jul 8, 2005.

  1. George Sakkis wrote:
    > "Steven Bethard" <> wrote:
    >> Dict comprehensions were recently rejected:
    >> http://www.python.org/peps/pep-0274.html
    >> The reason, of course, is that dict comprehensions don't gain you
    >> much at all over the dict() constructor plus a generator expression,
    >> e.g.:
    >> dict((i, chr(65+i)) for i in range(4))

    >
    > Sure, but the same holds for list comprehensions: list(i*i for i in
    > xrange(10)). The difference is historic I guess; list comprehensions
    > preceded generator expressions and so they cannot be removed, at least
    > not before 3.0. I wonder if they will/should be in the language when
    > the constraint of backwards compatibility is lifted. IMO they should
    > not (TIOOWTDI, uniformity among builtin data structures, not
    > overwhelmingly more useful than set or dict comprehensions), but
    > there's a long way till that day.


    I think the jury's still out on this one:

    * Alex Martelli expects list comprehensions to be removed. [1]
    * Robert Kern wants list comprehensions removed. [2]
    * Raymond Hettinger encourages continued use of list comprehensions [3]
    * Jeremy Bowers thinks list comprehensions should stay. [4]

    I only searched a few relatively recent threads in c.l.py, so there are
    probably more, but it looks to me like the final decision will have to
    be made by a pronouncement from Guido.

    [1]http://groups-beta.google.com/group/comp.lang.python/msg/f5613c00cb8c9539
    [2]http://groups-beta.google.com/group/comp.lang.python/msg/b2cf0cd72d53fbe5
    [3]http://groups-beta.google.com/group/comp.lang.python/msg/781dfab03701dd18
    [4]http://groups-beta.google.com/group/comp.lang.python/msg/771a47d9eb24c863
     
    Steven Bethard, Jul 8, 2005
    #1
    1. Advertising

  2. Steven Bethard

    Kay Schluehr Guest

    Steven Bethard schrieb:

    > I think the jury's still out on this one:
    >
    > * Alex Martelli expects list comprehensions to be removed. [1]
    > * Robert Kern wants list comprehensions removed. [2]
    > * Raymond Hettinger encourages continued use of list comprehensions [3]
    > * Jeremy Bowers thinks list comprehensions should stay. [4]
    >
    > I only searched a few relatively recent threads in c.l.py, so there are
    > probably more, but it looks to me like the final decision will have to
    > be made by a pronouncement from Guido.


    Well, I want to offer a more radical proposal: why not free squared
    braces from the burden of representing lists at all? It should be
    sufficient to write

    >>> list()

    list()

    After being free one can use them for other purposes e.g. replacing the
    ugly @ decorator character by the lovely [ .. ] notation or other
    important features no one never trusted to implement waiting for the
    right syntax sugar. More than this round braces together with lists can
    be considered as a concession to the LISP programmer who was repelled
    from Python by the decision to eliminate functional programming
    features.

    Kay
     
    Kay Schluehr, Jul 8, 2005
    #2
    1. Advertising

  3. Kay Schluehr wrote:
    > Well, I want to offer a more radical proposal: why not free squared
    > braces from the burden of representing lists at all? It should be
    > sufficient to write
    >
    >>>>list()

    >
    > list()


    So then what would the expression list('foo') mean? Would it be
    equivalent to ['foo'] (if so, how would you convert a string or other
    iterable to a list under Py3k?), or would it be equivalent to ['f', 'o',
    'o'] as it is in now (and is so, what gives?)?
     
    Leif K-Brooks, Jul 9, 2005
    #3
  4. Steven Bethard

    Devan L Guest

    List comprehensions are faster than generator comprehensions for
    iterating over smaller sequences.
     
    Devan L, Jul 9, 2005
    #4
  5. Kay Schluehr wrote:
    > Well, I want to offer a more radical proposal: why not free squared
    > braces from the burden of representing lists at all? It should be
    > sufficient to write
    >
    >>>>list()

    > list()
    >
    > After being free one can use them for other purposes e.g. replacing the
    > ugly @ decorator character by the lovely [ .. ] notation or other
    > important features no one never trusted to implement waiting for the
    > right syntax sugar. More than this round braces together with lists can
    > be considered as a concession to the LISP programmer who was repelled
    > from Python by the decision to eliminate functional programming
    > features.


    Heh heh.

    So how do I write a list literal then? Or do we only get tuple literals
    in Python 3.0. ;)

    STeVe
     
    Steven Bethard, Jul 9, 2005
    #5
  6. Devan L wrote:
    > List comprehensions are faster than generator comprehensions for
    > iterating over smaller sequences.


    Could you give me an example? For the simple one below, the generator
    expression was faster:

    $ python -m timeit "for x in (i for i in xrange(10)): y = x"
    100000 loops, best of 3: 4.75 usec per loop

    $ python -m timeit "for x in [i for i in xrange(10)]: y = x"
    100000 loops, best of 3: 5.33 usec per loop


    And for another one, the results are basically indistinguishable:

    $ python -m timeit "for x in (i for i in 'abcdefg'): y = x"
    100000 loops, best of 3: 3.82 usec per loop

    $ python -m timeit "for x in [i for i in 'abcdefg']: y = x"
    100000 loops, best of 3: 3.87 usec per loop


    I vaguely remember that in Python 2.4 conversion to tuples can take
    longer because of the tuple extension code:

    $ python -m timeit "tuple(i for i in xrange(10000))"
    100 loops, best of 3: 2.24 msec per loop

    $ python -m timeit "tuple([i for i in xrange(10000)])"
    100 loops, best of 3: 1.85 msec per loop

    But IIRC, this is supposed to be fixed in Python 2.5.


    I found that for some longer sequences, generator expressions were
    notably faster:

    $ python -m timeit "set(i for i in xrange(10000))"
    100 loops, best of 3: 3.77 msec per loop

    $ python -m timeit "set([i for i in xrange(10000)])"
    100 loops, best of 3: 4.54 msec per loop


    Timings that validate your statement would be appreciated. ;)

    STeVe
     
    Steven Bethard, Jul 9, 2005
    #6
  7. On Fri, 08 Jul 2005 16:07:50 -0600, Steven Bethard
    <> declaimed the following in comp.lang.python:

    >
    > I only searched a few relatively recent threads in c.l.py, so there are
    > probably more, but it looks to me like the final decision will have to
    > be made by a pronouncement from Guido.
    >

    Great... It takes me two releases of Python to get comfortable
    with them, and then they are threatened to be removed again...

    Might as well submit the language to ISO for standardization --
    then I wouldn't be following an erratic target <G>

    --
    > ============================================================== <
    > | Wulfraed Dennis Lee Bieber KD6MOG <
    > | Bestiaria Support Staff <
    > ============================================================== <
    > Home Page: <http://www.dm.net/~wulfraed/> <
    > Overflow Page: <http://wlfraed.home.netcom.com/> <
     
    Dennis Lee Bieber, Jul 9, 2005
    #7
  8. Steven Bethard

    Kay Schluehr Guest

    Leif K-Brooks schrieb:
    > Kay Schluehr wrote:
    > > Well, I want to offer a more radical proposal: why not free squared
    > > braces from the burden of representing lists at all? It should be
    > > sufficient to write
    > >
    > >>>>list()

    > >
    > > list()

    >
    > So then what would the expression list('foo') mean? Would it be
    > equivalent to ['foo'] (if so, how would you convert a string or other
    > iterable to a list under Py3k?), or would it be equivalent to ['f', 'o',
    > 'o'] as it is in now (and is so, what gives?)?


    Spiltting a string and putting the characters into a list could be done
    in method application style:

    >>> "abc".tolist()

    list('a','b','c')

    Or equivalent from lists point of view:

    >>> list.from_str("abc")

    list("a", "b", "c" )

    Kay
     
    Kay Schluehr, Jul 9, 2005
    #8
  9. Dennis Lee Bieber wrote:
    > On Fri, 08 Jul 2005 16:07:50 -0600, Steven Bethard
    > <> declaimed the following in comp.lang.python:
    >
    >>I only searched a few relatively recent threads in c.l.py, so there are
    >>probably more, but it looks to me like the final decision will have to
    >>be made by a pronouncement from Guido.

    >
    > Great... It takes me two releases of Python to get comfortable
    > with them, and then they are threatened to be removed again...
    >
    > Might as well submit the language to ISO for standardization --
    > then I wouldn't be following an erratic target <G>


    Two points:

    (1) There's no reason to get uncomfortable even if they're removed.
    You'd just replace [] with list().

    (2) *IMPORTANT* If this happens *at all*, it won't happen until Python
    3.0, which is probably at least 5 years away. And the Python 2.X branch
    will still be available then, so if you don't like Python 3.0, you don't
    have to use it.

    STeVe
     
    Steven Bethard, Jul 9, 2005
    #9
  10. Steven Bethard

    Ron Adam Guest

    Kay Schluehr wrote:
    >
    > Leif K-Brooks schrieb:
    >
    >>Kay Schluehr wrote:
    >>
    >>>Well, I want to offer a more radical proposal: why not free squared
    >>>braces from the burden of representing lists at all? It should be
    >>>sufficient to write
    >>>
    >>>
    >>>>>>list()
    >>>
    >>>list()

    >>
    >>So then what would the expression list('foo') mean? Would it be
    >>equivalent to ['foo'] (if so, how would you convert a string or other
    >>iterable to a list under Py3k?), or would it be equivalent to ['f', 'o',
    >>'o'] as it is in now (and is so, what gives?)?

    >
    >
    > Spiltting a string and putting the characters into a list could be done
    > in method application style:
    >
    >
    >>>>"abc".tolist()

    >
    > list('a','b','c')


    "abc".splitchrs()

    There's already a str.split() to create a list of words,
    and a str.splitline() to get a list of lines, so it would group related
    methods together.

    I don't thin adding sting methods to lists is a good idea.

    Cheers,
    Ron
     
    Ron Adam, Jul 9, 2005
    #10
  11. Kay Schluehr wrote:
    >>>>list.from_str("abc")

    >
    > list("a", "b", "c" )



    I assume we'll also have list.from_list, list.from_tuple,
    list.from_genexp, list.from_xrange, etc.?
     
    Leif K-Brooks, Jul 9, 2005
    #11
  12. Steven Bethard

    Devan L Guest

    >>> import timeit
    >>> t1 = timeit.Timer('list(i for i in xrange(10))')
    >>> t1.timeit()

    27.267753024476576
    >>> t2 = timeit.Timer('[i for i in xrange(10)]')
    >>> t2.timeit()

    15.050426800054197
    >>> t3 = timeit.Timer('list(i for i in xrange(100))')
    >>> t3.timeit()

    117.61078097914682
    >>> t4 = timeit.Timer('[i for i in xrange(100)]')
    >>> t4.timeit()

    83.502424470149151

    Hrm, okay, so generators are generally faster for iteration, but not
    for making lists(for small sequences), so list comprehensions stay.
     
    Devan L, Jul 9, 2005
    #12
  13. On Fri, 08 Jul 2005 22:29:30 -0600, Steven Bethard <> wrote:

    >Dennis Lee Bieber wrote:
    >> On Fri, 08 Jul 2005 16:07:50 -0600, Steven Bethard
    >> <> declaimed the following in comp.lang.python:
    >>
    >>>I only searched a few relatively recent threads in c.l.py, so there are
    >>>probably more, but it looks to me like the final decision will have to
    >>>be made by a pronouncement from Guido.

    >>
    >> Great... It takes me two releases of Python to get comfortable
    >> with them, and then they are threatened to be removed again...
    >>
    >> Might as well submit the language to ISO for standardization --
    >> then I wouldn't be following an erratic target <G>

    >
    >Two points:
    >
    >(1) There's no reason to get uncomfortable even if they're removed.
    >You'd just replace [] with list().


    So list(1, 2, 3) will be the same as [1, 2, 3] ??

    Right now,
    >>> list(1,2,3)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: list() takes at most 1 argument (3 given)

    have fun ;-)

    >
    >(2) *IMPORTANT* If this happens *at all*, it won't happen until Python
    >3.0, which is probably at least 5 years away. And the Python 2.X branch
    >will still be available then, so if you don't like Python 3.0, you don't
    >have to use it.
    >
    >STeVe


    Regards,
    Bengt Richter
     
    Bengt Richter, Jul 9, 2005
    #13
  14. Steven Bethard

    Ron Adam Guest

    Leif K-Brooks wrote:
    > Kay Schluehr wrote:
    >
    >>>>>list.from_str("abc")

    >>
    >>list("a", "b", "c" )

    >
    >
    >
    > I assume we'll also have list.from_list, list.from_tuple,
    > list.from_genexp, list.from_xrange, etc.?


    List from list isn't needed, nor list from tuple. That's what the * is
    for. And for that matter neither is the str.splitchar() either.


    class mylist(list):
    def __init__(self,*args):
    self[:]=args[:]


    mylist(*[1,2,3]) -> [1,2,3]

    mylist(*(1,2,3)) -> [1,2,3]

    mylist(*"abc") -> ['a','b','c']

    mylist(1,2,3) -> [1,2,3]

    mylist([1],[2]) -> [[1],[2]]

    mylist('hello','world') -> ['hello','world']



    Works for me. ;-)

    I always thought list([1,2,3]) -> [1,2,3] was kind of silly.


    Cheers,
    Ron
     
    Ron Adam, Jul 9, 2005
    #14
  15. Steven Bethard

    Kay Schluehr Guest

    Leif K-Brooks schrieb:
    > Kay Schluehr wrote:
    > >>>>list.from_str("abc")

    > >
    > > list("a", "b", "c" )

    >
    >
    > I assume we'll also have list.from_list, list.from_tuple,
    > list.from_genexp, list.from_xrange, etc.?


    One might unify all those factory functions into a single
    list.from_iter that dispatches to the right constructor that still
    lives under the hood. More conceptually: there is some abstract iter
    base class providing a from_iter method which may be overwritten in
    concrete subclasses like tuple, str or list.

    I would further suggest a lazy iterator used to evaluate objects when
    they get accessed the first time:

    >>> l = lazy( math.exp(100) , 27 )

    <lazy-iterator object at 0x4945f0>
    >>> l[1]

    27

    The first element won't ever be evaluated if it does not get accessed
    explicitely. This is some very special kind of partial
    evaluation/specialization.

    Kay
     
    Kay Schluehr, Jul 9, 2005
    #15
  16. Steven Bethard

    Peter Hansen Guest

    Bengt Richter wrote:
    > On Fri, 08 Jul 2005 22:29:30 -0600, Steven Bethard <> wrote:
    >>(1) There's no reason to get uncomfortable even if they're removed.
    >>You'd just replace [] with list().

    >
    > So list(1, 2, 3) will be the same as [1, 2, 3] ??


    No, the discussion is about list comprehensions. [1,2,3] is not a list
    comprehension, as you know.

    -Peter
     
    Peter Hansen, Jul 9, 2005
    #16
  17. In all probability, both list comprehensions and generator expressions
    will be around in perpetuity. List comps have been a very successful
    language feature.

    The root of this discussion has been the observation that a list
    comprehension can be expressed in terms of list() and a generator
    expression. However, the former is faster when you actually want a
    list result and many people (including Guido) like the square brackets.

    After the advent of generators, it seemed for a while that all
    functions and methods that returned lists would eventually return
    iterators instead. What we are learning is that there is a place for
    both. It is darned inconvenient to get an iterator when you really
    need a list, when you want to slice the result, when you want to see a
    few elements through repr(), and when you need to loop over the
    contents more than once.


    Raymond Hettinger
     
    Raymond Hettinger, Jul 9, 2005
    #17
  18. On Sat, 09 Jul 2005 10:16:17 -0400, Peter Hansen <> wrote:

    >Bengt Richter wrote:
    >> On Fri, 08 Jul 2005 22:29:30 -0600, Steven Bethard <> wrote:
    >>>(1) There's no reason to get uncomfortable even if they're removed.
    >>>You'd just replace [] with list().

    >>
    >> So list(1, 2, 3) will be the same as [1, 2, 3] ??

    >
    >No, the discussion is about list comprehensions. [1,2,3] is not a list
    >comprehension, as you know.
    >

    D'oh. Sorry to have come in from contextual outer space ;-/

    Regards,
    Bengt Richter
     
    Bengt Richter, Jul 9, 2005
    #18
  19. Steven Bethard

    John Roth Guest

    "Raymond Hettinger" <> wrote in message
    news:...
    > In all probability, both list comprehensions and generator expressions
    > will be around in perpetuity. List comps have been a very successful
    > language feature.
    >
    > The root of this discussion has been the observation that a list
    > comprehension can be expressed in terms of list() and a generator
    > expression. However, the former is faster when you actually want a
    > list result and many people (including Guido) like the square brackets.
    >
    > After the advent of generators, it seemed for a while that all
    > functions and methods that returned lists would eventually return
    > iterators instead. What we are learning is that there is a place for
    > both. It is darned inconvenient to get an iterator when you really
    > need a list, when you want to slice the result, when you want to see a
    > few elements through repr(), and when you need to loop over the
    > contents more than once.


    I was wondering about what seemed like an ill-concieved rush to
    make everything an iterator. Iterators are, of course, useful but there
    are times when you really did want a list.

    John Roth
    >
    >
    > Raymond Hettinger
    >
     
    John Roth, Jul 9, 2005
    #19
  20. "Raymond Hettinger" <> wrote:

    > In all probability, both list comprehensions and generator expressions
    > will be around in perpetuity. List comps have been a very successful
    > language feature.
    >
    > The root of this discussion has been the observation that a list
    > comprehension can be expressed in terms of list() and a generator
    > expression.


    No, the root of the discussion, in this thread at least, was the answer
    to "why not dict comprehensions ?", which was along the lines of "well,
    you can do it in one line by dict(gen_expression)".

    > However, the former is faster when you actually want a
    > list result and many people (including Guido) like the square brackets.


    Also dict comprehensions are faster if you actually want a dict result,
    set comprehensions for set result, and so on and so forth.

    > After the advent of generators, it seemed for a while that all
    > functions and methods that returned lists would eventually return
    > iterators instead. What we are learning is that there is a place for
    > both.


    Altering the result type of existing functions and methods is not
    exactly the same with the discussion on the future of list
    comprehensions; the latter affects only whether listcomps are special
    enough to be granted special syntax support, when there is an
    equivalent way to express the same thing. It's funny how one of the
    arguments for removing lambda -- you can do the same by defining a
    named function -- does not apply for list comprehensions.

    > It is darned inconvenient to get an iterator when you really
    > need a list, when you want to slice the result, when you want to see a
    > few elements through repr(), and when you need to loop over the
    > contents more than once.
    >
    > Raymond Hettinger


    Similar arguments can be given for dict comprehensions as well.

    George
     
    George Sakkis, Jul 9, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. seguso
    Replies:
    9
    Views:
    392
    seguso
    Dec 22, 2004
  2. Dave Kuhlman

    Re: Style in list comprehensions

    Dave Kuhlman, Aug 15, 2003, in forum: Python
    Replies:
    1
    Views:
    329
    Alex Martelli
    Aug 16, 2003
  3. Frank Millman

    Simple db using list comprehensions

    Frank Millman, Apr 5, 2004, in forum: Python
    Replies:
    1
    Views:
    288
    Paddy McCarthy
    Apr 16, 2004
  4. Elaine Jackson

    list comprehensions

    Elaine Jackson, Apr 7, 2004, in forum: Python
    Replies:
    10
    Views:
    612
  5. Steven Bethard
    Replies:
    7
    Views:
    401
    Rocco Moretti
    Jan 20, 2006
Loading...

Share This Page