Re: Style in list comprehensions

Discussion in 'Python' started by Dave Kuhlman, Aug 15, 2003.

  1. Dave Kuhlman

    Dave Kuhlman Guest

    Tim Lesher wrote:

    > Suppose I have a list of objects and I want to call a method on
    > each. I can do the simple:
    >
    > for i in objs:
    > i.meth(arg1, arg2)
    >
    > or using list comprehensions:
    >
    > [i.meth(arg1, arg2) for i in objs]
    >
    > The second feels more Pythonic, but do I incur any overhead for
    > creating the list of results when I'm not going to use it?
    >


    Play fair. Your two scripts do not do the same thing. The script
    using list comprehension forms a list containing each of the
    results of applying meth to the items in the original list. If you
    do not need the resulting list, then use the for-loop. If you do
    need that list, then to make the two lists equivalent, you might
    modify the for-loop as follows:

    result = []
    for i in objs:
    result.append(i.meth(arg1, arg2))

    Now ask which is preferred.

    The script using list comprehension has the advantage of of
    possibly mystifying Perl programmers. They do enough of that to us
    Pythonista's, so it serves them right, don't you agree.

    Come to think about it, if you think it might mystify anyone, then
    it is un-Pythonic to write it that way.

    As to speed, my eyeballs could detect no difference with the
    following:

    def func1(x):
    return x*3
    def t1(lst):
    return [func1(x) for x in lst]
    def t2(lst):
    result = []
    for x in lst:
    result.append(func1(x))
    return result
    a = t1(range(100000))
    a = t2(range(100000))

    Dave

    --
    Dave Kuhlman
    http://www.rexx.com/~dkuhlman
     
    Dave Kuhlman, Aug 15, 2003
    #1
    1. Advertising

  2. Dave Kuhlman wrote:
    ...
    > def func1(x):
    > return x*3
    > def t1(lst):
    > return [func1(x) for x in lst]
    > def t2(lst):
    > result = []
    > for x in lst:
    > result.append(func1(x))
    > return result


    With Python 2.3's timeit.py, I think the performance differences are
    more easily and reliably measured -- both those stemming directly from
    using list comprehensions rather than loops of .append calls, and the
    ones often enabled by a list comprehension's better suitability for
    "inlining" whatever operations you're performing. To wit:

    bash-2.05b$ cd /usr/lib/python2.3
    bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
    > 'result=[]' 'for x in range(10000): result.append(f1(x))'

    10 loops, best of 3: 3.7e+04 usec per loop

    bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
    > 'result=[f1(x) for x in range(10000)]'

    10 loops, best of 3: 2.94e+04 usec per loop

    bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
    > 'result=[x*3 for x in range(10000)]'

    100 loops, best of 3: 1.79e+04 usec per loop

    In some (rare) bottleneck situations, these performance differences
    might perhaps be significant. Similarly for good old map...:

    bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
    > 'result=map(f1, range(10000))'

    100 loops, best of 3: 1.87e+04 usec per loop

    bash-2.05b$ python2.3 timeit.py -s'def f1(x): return x*3' \
    > 'result=map(lambda x:x*3, range(10000))'

    100 loops, best of 3: 1.84e+04 usec per loop

    No "inlining" (as you can see, pseudo-inlining via lambda does
    not buy you any performance advantage), but if you're stuck in a
    situation where you just can't inline map may well be faster than
    list comprehensions -- perhaps significantly so, if this happens
    to fall right on a performance bottleneck of your program.

    Finally, in some cases, you can "preallocate" the result list to
    the length it will have in the end, and then rebind its items one
    by one, rather than "growing" the result list as you go. When
    this is feasible, it's often the fastest approach you can possibly
    use, to wit:

    bash-2.05b$ python2.3 timeit.py -s'result=10000*[None]' \
    > 'for i in range(10000): result = i*3'

    100 loops, best of 3: 9.75e+03 usec per loop

    ....almost twice as fast than the best map or list comprehension,
    in this specially favourable case.

    Of course, talking of specially favourable cases, when some time
    consuming operation DOES represent such a case one might be well
    advised to keep an eye out for totally different possibilities...:

    bash-2.05b$ python2.3 timeit.py 'result=range(0,30000,3)'
    100 loops, best of 3: 2.22e+03 usec per loop

    Yes, this IS "cheating" -- but it does produce the same 'result'
    list as each of the other approaches, and it does so 4 or 5 times
    faster than the second-best approach (preallocated list, taking
    some advantage of this being a special case), almost 10 times
    faster than the best "generalized" approach. Any time you need
    to produce any kind of arithmetic progression, do consider using
    just a special-purpose 'range' call, if you care about speed!-)


    Alex
     
    Alex Martelli, Aug 16, 2003
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. seguso
    Replies:
    9
    Views:
    401
    seguso
    Dec 22, 2004
  2. Frank Millman

    Simple db using list comprehensions

    Frank Millman, Apr 5, 2004, in forum: Python
    Replies:
    1
    Views:
    300
    Paddy McCarthy
    Apr 16, 2004
  3. Elaine Jackson

    list comprehensions

    Elaine Jackson, Apr 7, 2004, in forum: Python
    Replies:
    10
    Views:
    631
  4. Mahesh Padmanabhan

    Generator expressions v/s list comprehensions

    Mahesh Padmanabhan, Aug 28, 2004, in forum: Python
    Replies:
    24
    Views:
    697
    Raymond Hettinger
    Sep 1, 2004
  5. Steven Bethard
    Replies:
    7
    Views:
    410
    Rocco Moretti
    Jan 20, 2006
Loading...

Share This Page