List Comprehension question

[Mark Elston]

I recently stumbled over List Comprehension while reading the
Python Cookbook. I have not kept up with the What's New
sections in the online docs.

Anyway, I thought I was following the discussions of List
Comprehension (LC) until I got to Recipe 1.16. In this recipe
we have the following:

arr = [[1,2,3], [4,5,6], [7,8,9], [10,11,12]]
print [[r[col] for r in arr] for col in range(len(arr[0]))]
-> [[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]]

For all the previous LC examples (and in the What's New
writeup for 2.0) it was stated (or implied) that code of the form:

[ expression for expr in sequence1
for expr2 in sequence2 ...
for exprN in sequenceN
if condition ]

was equivalent to:

for expr1 in sequence1:
for expr2 in sequence2:
...
for exprN in sequenceN:
if (condition):
# Append the value of
# the expression to the
# resulting list.

I thought I understood this until I got to the above recipe. Here
it looks like the order of evaluation is reversed. That is, instead
of translating to:

for r[col] in arr:
for col in range(len(arr[0])):
...

we actually have

for col in range(len(arr[0])):
for r[col] in arr:
...

And all of this due to a placement of '[ ... ]' around the 'inner'
loop.

The reference documentation doesn't explain this either. The grammar
page on List Displays doesn't seem to give any insight.

While I don't really understand it I may have some kind of rationale.
Please let me know if this is correct.

The print... command is a LC with a single expression and a single 'for'
construct. The expression, itself, is also a LC. When the command is
executed 'col' is set to 0 (the first value in the range) and this is
'passed' to the expression for evaluation. This evaluation results in
a list generated by the 'inner' LC which iterates over each row in the
array and, therefore, generates the list: [1, 4, 7, 10].

The next step is to go back to 'col' and extract the next value (1) and
generate the next list, etc.

Well, hmmmmm. OK. Maybe I do understand it. It just wasn't apparent
at first.

Am I close, or did I guess wrong?

Mark

 

[Francis Avila]

Mark Elston wrote in message ...

Anyway, I thought I was following the discussions of List
Comprehension (LC) until I got to Recipe 1.16. In this recipe
we have the following:

arr = [[1,2,3], [4,5,6], [7,8,9], [10,11,12]]
print [[r[col] for r in arr] for col in range(len(arr[0]))]
-> [[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]]

All the second line is saying, is, for every pass of the innermost (here,
rightmost) for-loop, execute this list comprehension: [r[col] for r in arr].

Same as:

res = []
for col in range(3):
res.append([r[col] for r in arr])

Unrolling a bit more:

res = []
for col in range(3):
innerres = []
for r in arr:
innerres.append(r[col])
res.append(innerres)

Note this is NOT the same as [r[col] for col in range(3) for r in arr]!

res = []
for col in range(3):
for r in arr:
res.append(r[col])

-> [1, 4, 7, 10, 2, 5, 8, 11, 3, 6, 9, 12]
See? It's flattened, because the list comp's expression evaluates to an
integer instead of a list.

I thought I understood this until I got to the above recipe. Here
it looks like the order of evaluation is reversed. That is, instead
of translating to:

for r[col] in arr:
for col in range(len(arr[0])):
...

we actually have

for col in range(len(arr[0])):
for r[col] in arr:
...

Nope. See above.

While I don't really understand it I may have some kind of rationale.
Please let me know if this is correct.

You're thinking too much.

The expression, itself, is also a LC.

This is the part that's causing you confusion. This is a nested list comp:
for every iteration in the outer list comp, execute the expression. The
fact that the expression happens to be a list comp itself just means that
the outer list comp will append a new list on each pass.

Wrap the inner list comp in a function call and it will make sense to you:

def innercomp(col):
return [r[col] for r in arr]

[innercomp(col) for col in range(3)]

-> [[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]]

(It's also significantly slower.)

Well, hmmmmm. OK. Maybe I do understand it. It just wasn't apparent
at first.

Am I close, or did I guess wrong?

You got it.

Note, however, that the same thing is far easier with zip():
[(1, 4, 7, 10), (2, 5, 8, 11), (3, 6, 9, 12)]

If you need the items to be lists,

[list(i) for i in zip(*arr)]

[[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]]

or,
[[1, 4, 7, 10], [2, 5, 8, 11], [3, 6, 9, 12]]