help with list comprehension

[Yves Dorfsman]

In the following script, m1() and m2() work fine. I am assuming m2() is
faster although I haven't checked that (loops through the list twice instead
of once).

Now what I am trying to do is something like m3(). As currently written it
does not work, and I have tried different ways, but I haven't managed to
make it work.

Is there a possibility ? Or is m2() the optimum ?


Thanks.



#!/usr/bin/python

l = [ { 'colour': 'black', 'num': 0},
{ 'colour': 'brown', 'num': 1},
{ 'colour': 'red', 'num': 2},
{ 'colour': 'orange', 'num': 3},
{ 'colour': 'yellow', 'num': 4},
{ 'colour': 'green', 'num': 5},
{ 'colour': 'blue', 'num': 6},
{ 'colour': 'violet', 'num': 7},
{ 'colour': 'grey', 'num': 8},
{ 'colour': 'white', 'num': 9}
]

def m1():
colours = [ e['colour'] for e in l ]
nums = [ e['num'] for e in l ]

def m2():
colours = []
nums = []
for e in l:
colours.append(e['colour'])
nums.append(e['num'])

#def m3():
# colours, nums = [ e['colour'], e['num'] for e in l ]

 

[Carsten Haese]

In the following script, m1() and m2() work fine. I am assuming m2() is
faster although I haven't checked that (loops through the list twice
instead of once).

Well, let's check it:

$ python -m timeit -s "import x" "x.m1()"
100000 loops, best of 3: 6.43 usec per loop

$ python -m timeit -s "import x" "x.m2()"
100000 loops, best of 3: 8.34 usec per loop

As it turns out, m1 is faster than m2. The reason is that list
comprehensions do the loop in C, whereas the for-append pattern does the
loop on the Python level.

Now what I am trying to do is something like m3(). As currently written
it does not work, and I have tried different ways, but I haven't managed
to make it work.

Is there a possibility ? Or is m2() the optimum ?

[...]
def m1():
colours = [ e['colour'] for e in l ]
nums = [ e['num'] for e in l ]

def m2():
colours = []
nums = []
for e in l:
colours.append(e['colour'])
nums.append(e['num'])

#def m3():
# colours, nums = [ e['colour'], e['num'] for e in l ]

m3 doesn't work because you're building a list of 10 color/number pairs
that you're trying to unpack that into just two names. The working
"derivative" of m3 is m1, which is the most natural, fastest and
clearest solution to your problem.

HTH,

 

[Karthik Gurusamy]

In the following script, m1() and m2() work fine. I am assuming m2() is
faster although I haven't checked that (loops through the list twice instead
of once).

Now what I am trying to do is something like m3(). As currently written it
does not work, and I have tried different ways, but I haven't managed to
make it work.

Is there a possibility ? Or is m2() the optimum ?

Thanks.

#!/usr/bin/python

l = [ { 'colour': 'black', 'num': 0},
{ 'colour': 'brown', 'num': 1},
{ 'colour': 'red', 'num': 2},
{ 'colour': 'orange', 'num': 3},
{ 'colour': 'yellow', 'num': 4},
{ 'colour': 'green', 'num': 5},
{ 'colour': 'blue', 'num': 6},
{ 'colour': 'violet', 'num': 7},
{ 'colour': 'grey', 'num': 8},
{ 'colour': 'white', 'num': 9}
]

def m1():
colours = [ e['colour'] for e in l ]
nums = [ e['num'] for e in l ]

def m2():
colours = []
nums = []
for e in l:
colours.append(e['colour'])
nums.append(e['num'])

#def m3():
# colours, nums = [ e['colour'], e['num'] for e in l ]

Looks like m1 is the cleanest; if you really want to run list-
comprehension once, one possible way:

p = [ (e['colour'], e['num']) for e in l ]
import operator
map(operator.itemgetter(0), p)

['black', 'brown', 'red', 'orange', 'yellow', 'green', 'blue',
'violet', 'grey', 'white']

map(operator.itemgetter(1), p) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

Karthik

 

[George Sakkis]

In the following script, m1() and m2() work fine. I am assuming m2() is
faster although I haven't checked that (loops through the list twice
instead of once).

Well, let's check it:

$ python -m timeit -s "import x" "x.m1()"
100000 loops, best of 3: 6.43 usec per loop

$ python -m timeit -s "import x" "x.m2()"
100000 loops, best of 3: 8.34 usec per loop

As it turns out, m1 is faster than m2. The reason is that list
comprehensions do the loop in C, whereas the for-append pattern does the
loop on the Python level.

Now what I am trying to do is something like m3(). As currently written
it does not work, and I have tried different ways, but I haven't managed
to make it work.

Is there a possibility ? Or is m2() the optimum ?

[...]
def m1():
  colours = [ e['colour'] for e in l ]
  nums    = [ e['num']    for e in l ]

def m2():
  colours = []
  nums    = []
  for e in l:
    colours.append(e['colour'])
    nums.append(e['num'])

#def m3():
#  colours, nums = [ e['colour'], e['num'] for e in l ]

m3 doesn't work because you're building a list of 10 color/number pairs
that you're trying to unpack that into just two names. The working
"derivative" of m3 is m1, which is the most natural, fastest and
clearest solution to your problem.

Another alternative is:

from operator import itemgetter

def m3():
colours, nums = zip(*map(itemgetter('colour','num'), l))

It's slower than m1() but faster than m2(); it's also the most
concise, especially if you extract more than two keys.

George

 

[Matimus]

Yves Dorfsman wrote:

Well, let's check it:

$ python -m timeit -s "import x" "x.m1()"
100000 loops, best of 3: 6.43 usec per loop

$ python -m timeit -s "import x" "x.m2()"
100000 loops, best of 3: 8.34 usec per loop

As it turns out, m1 is faster than m2. The reason is that list
comprehensions do the loop in C, whereas the for-append pattern does the
loop on the Python level.

Now what I am trying to do is something like m3(). As currently written
it does not work, and I have tried different ways, but I haven't managed
to make it work.
Is there a possibility ? Or is m2() the optimum ?
[...]
def m1():
colours = [ e['colour'] for e in l ]
nums = [ e['num'] for e in l ]
def m2():
colours = []
nums = []
for e in l:
colours.append(e['colour'])
nums.append(e['num'])
#def m3():
# colours, nums = [ e['colour'], e['num'] for e in l ]

m3 doesn't work because you're building a list of 10 color/number pairs
that you're trying to unpack that into just two names. The working
"derivative" of m3 is m1, which is the most natural, fastest and
clearest solution to your problem.

Another alternative is:

from operator import itemgetter

def m3():
colours, nums = zip(*map(itemgetter('colour','num'), l))

It's slower than m1() but faster than m2(); it's also the most
concise, especially if you extract more than two keys.

George

Why deal with zip and unpacking? This seems more obvious to me:

colours, nums = map(itemgetter('colour'), l), map(itemgetter('num'),
l)

Matt

 

[George Sakkis]

Yves Dorfsman wrote:
In the following script, m1() and m2() work fine. I am assuming m2() is
faster although I haven't checked that (loops through the list twice
instead of once).
Well, let's check it:
$ python -m timeit -s "import x" "x.m1()"
100000 loops, best of 3: 6.43 usec per loop
$ python -m timeit -s "import x" "x.m2()"
100000 loops, best of 3: 8.34 usec per loop
As it turns out, m1 is faster than m2. The reason is that list
comprehensions do the loop in C, whereas the for-append pattern does the
loop on the Python level.
Now what I am trying to do is something like m3(). As currently written
it does not work, and I have tried different ways, but I haven't managed
to make it work.
Is there a possibility ? Or is m2() the optimum ?
[...]
def m1():
  colours = [ e['colour'] for e in l ]
  nums    = [ e['num']    for e in l ]
def m2():
  colours = []
  nums    = []
  for e in l:
    colours.append(e['colour'])
    nums.append(e['num'])
#def m3():
#  colours, nums = [ e['colour'], e['num'] for e in l ]
m3 doesn't work because you're building a list of 10 color/number pairs
that you're trying to unpack that into just two names. The working
"derivative" of m3 is m1, which is the most natural, fastest and
clearest solution to your problem.

Another alternative is:

from operator import itemgetter

def m3():
    colours, nums = zip(*map(itemgetter('colour','num'), l))

It's slower than m1() but faster than m2(); it's also the most
concise, especially if you extract more than two keys.

George

Why deal with zip and unpacking?

DRY [1].

This seems more obvious to me:

colours, nums = map(itemgetter('colour'), l), map(itemgetter('num'),
l)

Maybe, but now extend it to three keys. And then four. And then... you
see my point.

George

[1] http://en.wikipedia.org/wiki/Don't_repeat_yourself

 

[Yves Dorfsman]

Another alternative is:

from operator import itemgetter

def m3():
colours, nums = zip(*map(itemgetter('colour','num'), l))

It's slower than m1() but faster than m2(); it's also the most
concise, especially if you extract more than two keys.

Good you guys gave me some ideas, I've made m3() work:

def m3()
total = [ [e['colour'], e['num'], e['couleur']] for e in l ]
c,d,e = zip(*total)
return map(list, [c,d,e])

(I have to transform to lists, I need to transform them later on, so tuples
won't work).

Unfortunately the timing is inconsistant, the first time I run
python -m timeit 'import listcomp ; listcomp.m1()'

m1 is the fastest. But then if I run the test several times, they all seem
to be about the same time ; I'll have to try with a large list.

Now here is a question:
Is there any way to change my first line there for the list comp. to return
a list of lists rather than a list of tuples ?