question regarding list comprehensions

P

Pat

I have written chunks of Python code that look this:

new_array = []
for a in array:
if not len( a ):
continue
new_array.append( a )

and...

string = ""
for r in results:
if not r.startswith( '#' ):
string =+ r

It seems that a list comprehension could clean up the code, but I seem
to have a mental block on list comprehensions. I've read up a lot on
this subject in my books and on the Internet and for whatever reason,
I'm having problems with this idiom (if that's the correct expression).

I've made a number of attempts to solve this on my own but I keep
getting errors.

Could someone please tell me how I could convert the above code to
something more elegant but readily understandable?

Finally, if someone could point me to a good tutorial or explain list
compressions I would be forever in your debt.
 
D

Diez B. Roggisch

Pat said:
I have written chunks of Python code that look this:

new_array = []
for a in array:
if not len( a ):
continue
new_array.append( a )

new_array = [a for a in array if len(a)]
and...

string = ""
for r in results:
if not r.startswith( '#' ):
string =+ r


"".join(r for r in results if not r.startswith("#"))

Diez
 
S

Steven D'Aprano

Finally, if someone could point me to a good tutorial or explain list
compressions I would be forever in your debt.

Think of a for-loop:

for x in (1, 2, 3):
x

Creates x=1, then x=2, then x=3. It doesn't do anything with the x's, but
just creates them. Let's turn it into a list comp, and collect the x's:
[1, 2, 3]



for x in (1, 2, 3):
2*x+1

Creates x=1, then evaluates 2*x+1 = 3. Then it repeats for x=2, then x=3.
Here it is as a list comp:
[3, 5, 7]



for x in (1, 2, 3):
if x != 2:
2*x+1

Here it is as a list comp:
[2*x+1 for x in (1, 2, 3) if x != 2]
[3, 7]


You can use any sort of sequence inside a list comp, not just a tuple.
['A', 'B', 'C', 'D']


You can nest list comps:
[y+1 for y in [2*x+1 for x in (1, 2, 3)]]
[4, 6, 8]


Advanced: you can use tuple-unpacking:
[(y,x) for (x,y) in [(1,2), (3, 4)]]
[(2, 1), (4, 3)]

and also multiple for-loops:
[(x,c) for x in (1, 2) for c in "abc"]
[(1, 'a'), (1, 'b'), (1, 'c'), (2, 'a'), (2, 'b'), (2, 'c')]



That last one is close to:

for x in (1, 2):
for c in "abc":
(x, c)
 
B

Bruno Desthuilliers

Pat a écrit :
I have written chunks of Python code that look this:

new_array = []
for a in array:
if not len( a ):
continue
new_array.append( a )


# à la lisp
new_array = filter(None, array)

# à la haskell
new_array = [a for a in array if a]

NB : all builtin empty builtin sequence type have a false boolen value,
so 'not len(a)' and 'not a' are usually practicaly equivalent.

<ot>
While we're at it : I assume you mean 'list', not 'array'. That's at
least how the builtin type is named...
and...

string = ""
for r in results:
if not r.startswith( '#' ):
string =+ r


# à la lisp
string = "".join(filter(lambda s: not s.startwith('#'), results)

# à la haskell
string = "".join(r for r in results if not s.startswith('#'))
It seems that a list comprehension could clean up the code, but I seem
to have a mental block on list comprehensions. I've read up a lot on
this subject in my books and on the Internet and for whatever reason,
I'm having problems with this idiom (if that's the correct expression).

I've made a number of attempts to solve this on my own but I keep
getting errors.

Could someone please tell me how I could convert the above code to
something more elegant but readily understandable?

Basically, a list expression is written as (pseudo BNF):

'['<expression> for item in <iterable_expression> [if
<conditional_expression>] ']'


where
- <expression> is any Python expression
- <iterable_expression> is any Python expression that evals to an iterable
- the 'if <conditional_expression>' part is optional

The resulting list will be obtained by evaluating <expression> for each
item of <iterable_expression> for which <conditional_expression> is
verified.

Since the main goal of list expressions is to build, well, lists, one
usually uses an <expression> that evals to something we possibly want to
be part of the new list !-)


Now for something more practical:

First let's rewrite your first snippet to get rid of the continue statement:

new_list = []
for item in source_list:
if len(item):
new_list.append(item)


And let's rewrite the second one to replace string concatenation with
the more idiomatic (and much more flexible) list / join idiom:

new_list = []
for item in results:
if not item.startswith('#'):
new_list.append(item)

string = "".join(new_list)


You may start to see a kind of pattern here:

0. new_list = []
1. for item in sequence:
2. if condition(item):
3. new_list.append(item)

The equivalent list comprehension is:

new_list = [item for item in sequence if condition(item)]


Since you didn't apply any transformation to 'item' in your for loops,
the <expression> part is as simple as it can be : we want the item, period.

The <iterable_expression> is here again quite simple : it's your source
sequence. Ditto for the the <conditional_expression>.

IOW, your for-loop pattern is really:

new_list = []
for item in <iterable_expression>:
if <conditional_expression>:
new_list.append(<expression>)


Once you've identified the different parts, translating this for-loop to
a list comprehension is mostly a matter of reording.

Finally, if someone could point me to a good tutorial or explain list
compressions I would be forever in your debt.

http://en.wikipedia.org/wiki/List_comprehension
http://en.wikipedia.org/wiki/Set-builder_notation


HTH
 
P

Pat

Steven said:
Finally, if someone could point me to a good tutorial or explain list
compressions I would be forever in your debt.

Think of a for-loop:

for x in (1, 2, 3):
x

Creates x=1, then x=2, then x=3. It doesn't do anything with the x's, but
just creates them. Let's turn it into a list comp, and collect the x's:
[x for x in (1, 2, 3)]
[1, 2, 3]



for x in (1, 2, 3):
2*x+1

Creates x=1, then evaluates 2*x+1 = 3. Then it repeats for x=2, then x=3.
Here it is as a list comp:
[2*x+1 for x in (1, 2, 3)]
[3, 5, 7]



for x in (1, 2, 3):
if x != 2:
2*x+1

Here it is as a list comp:
[2*x+1 for x in (1, 2, 3) if x != 2]
[3, 7]


You can use any sort of sequence inside a list comp, not just a tuple.
[c.upper() for c in "abcd"]
['A', 'B', 'C', 'D']


You can nest list comps:
[y+1 for y in [2*x+1 for x in (1, 2, 3)]]
[4, 6, 8]


Advanced: you can use tuple-unpacking:
[(y,x) for (x,y) in [(1,2), (3, 4)]]
[(2, 1), (4, 3)]

and also multiple for-loops:
[(x,c) for x in (1, 2) for c in "abc"]
[(1, 'a'), (1, 'b'), (1, 'c'), (2, 'a'), (2, 'b'), (2, 'c')]



That last one is close to:

for x in (1, 2):
for c in "abc":
(x, c)

Thank you. I think that clears up the mystery a bit. I added your note
to my snippets file. When I have a situation that I can't resolve, I'll
ask with the specifics.

I really do try to learn this as much as possible on my own without just
flinging a question onto the forum.
 
P

Pat

Diez said:
Pat said:
I have written chunks of Python code that look this:

new_array = []
for a in array:
if not len( a ):
continue
new_array.append( a )

new_array = [a for a in array if len(a)]
and...

string = ""
for r in results:
if not r.startswith( '#' ):
string =+ r


"".join(r for r in results if not r.startswith("#"))

Diez


Thank you very much! That's exactly what I was looking for. That's
much cleaner than my code.

This is a great forum and I really appreciate everyone's help.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,055
Latest member
SlimSparkKetoACVReview

Latest Threads

Top