list comprehension question

[Ross]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?

 

[Chris Rebert]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?

Your comprehension is the identity comprehension (i.e. it effectively
just copies the list as-is).
What you're trying to do is difficult if not impossible to do as a
comprehension.

Here's another approach:
b = list(itertools.chain.from_iterable(a))

And without using a library function:
b = []
for pair in a:
for item in pair:
b.append(item)

Cheers,
Chris

 

[Michael Spencer]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?
--
http://mail.python.org/mailman/listinfo/python-list

>>> a = [(1,2), (3,4), (5,6)]
>>> [i for t in a for i in t]

[1, 2, 3, 4, 5, 6]

Or, with different spacing to make the nesting clearer:

>>> [i

... for t in a
... for i in t]
[1, 2, 3, 4, 5, 6]
Michael

 

[Arnaud Delobelle]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?

When writing nested list comprehension, the for loops are in the same
order as you would write a normal nested for loop (which is not
necessarily intuitive when you first find out but is very handy in the
long run I think).

So write:

[j for i in a for j in i]

 

[thor]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?

When writing nested list comprehension, the for loops are in the same
order as you would write a normal nested for loop (which is not
necessarily intuitive when you first find out but is very handy in the
long run I think).

So write:

    [j for i in a for j in i]

--
Arnaud an trick

a [(1, 2), (3, 4), (5, 6)]
sum(a, ()) (1, 2, 3, 4, 5, 6)

you may search the maillist , somebody questioned before

 

[J Kenneth King]

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?

Your comprehension is the identity comprehension (i.e. it effectively
just copies the list as-is).
What you're trying to do is difficult if not impossible to do as a
comprehension.

Here's another approach:
b = list(itertools.chain.from_iterable(a))

And without using a library function:
b = []
for pair in a:
for item in pair:
b.append(item)

This is much more clear than a nested comprehension.

I love comprehensions, but abusing them can lead to really dense and
difficult to read code.

 

[Emile van Sebille]

b = []
for pair in a:
for item in pair:
b.append(item)

This is much more clear than a nested comprehension.

I love comprehensions, but abusing them can lead to really dense and
difficult to read code.[/QUOTE]

I disagree on dense and difficult, although I'll leave open the question
of abuse.

b = [ item for pair in a for item in pair ]

This is exactly the code above expressed in comprehension form.

It's worth knowing that a list comprehension is structured identically
to the equivalent for loop. So it really is neither more dense nor more
difficult to read. Further, you can tell immediately from the start of
the list comprehension what you've got -- in this case a list of item(s).

Here with some slight changes...

>>> a = [(1, 2), (3, 4, 7), (5, 6)]
>>> [ item for j in a if len(j)==2 for item in j if item % 2 ]

[1, 5]

....opposed to...
.... if len(j)==2:
.... for item in j:
.... if item % 2:
.... b.append(item)
....

>>> b [1, 5]
>>>

YMMV,

Emile

 

[Arnaud Delobelle]

b = []
for pair in a:
for item in pair:
b.append(item)

This is much more clear than a nested comprehension.

I love comprehensions, but abusing them can lead to really dense and
difficult to read code.

I disagree on dense and difficult, although I'll leave open the
question of abuse.

b = [ item for pair in a for item in pair ]

This is exactly the code above expressed in comprehension form.[/QUOTE]

If the comprehension above is an abuse, then every nested list
comprehension is an abuse of comprehensions so they might as well not be
in the language...

 

[Shane Geiger]

from goopy.functional import flatten #
http://sourceforge.net/projects/goog-goopy/
b = [(1,2), (3,4), (5,6)]
print flatten(b)



#from goopy.functional import flatten #
http://sourceforge.net/projects/goog-goopy/

def flatten(seq):
"""
Returns a list of the contents of seq with sublists and tuples "exploded".
The resulting list does not contain any sequences, and all inner sequences
are exploded. For example:

>>> flatten([7,(6,[5,4],3),2,1])

[7,6,5,4,3,2,1]
"""
lst = []
for el in seq:
if type(el) == list or type(el) is tuple:
lst.extend(flatten(el))
else:
lst.append(el)
return lst

If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to
return a new list of each individual element in these tuples, I can do
it with a nested for loop but when I try to do it using the list
comprehension b = [j for j in i for i in a], my output is b =
[5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I
doing wrong?

Your comprehension is the identity comprehension (i.e. it effectively
just copies the list as-is).
What you're trying to do is difficult if not impossible to do as a
comprehension.

Here's another approach:
b = list(itertools.chain.from_iterable(a))

And without using a library function:
b = []
for pair in a:
for item in pair:
b.append(item)

Cheers,
Chris

--
Shane Geiger, IT Director
Council For Economic Education / www.councilforeconed.org
(e-mail address removed) / 402-438-8958

Teaching Opportunity

 

[John Posner]

if type(el) == list or type(el) is tuple:

A tiny improvement:

if type(el) in (list, tuple):

 

[Emile van Sebille]

On 5/1/2009 7:31 AM J Kenneth King said...

I love comprehensions, but abusing them can lead to really dense and
difficult to read code.

I disagree on dense and difficult, although I'll leave open the
question of abuse.

b = [ item for pair in a for item in pair ]

This is exactly the code above expressed in comprehension form.

If the comprehension above is an abuse, then every nested list
comprehension is an abuse of comprehensions so they might as well not be
in the language...
[/QUOTE]

I hope you didn't understand me to say that this usage is abuse. That's
certainly not what I mean. I have, however, examples in my code base
that _would_ qualify as abuse (mostly from pre dictionary comprehension
vintage where I would build a dict from within a list comp), and I have
seen enough from this group to know that abuse abounds, with only slight
provocation.

Fondly-remembering-byte-code-hacks-and-introspection-ly y'rs,

Emile

 

[Arnaud Delobelle]

On 5/1/2009 7:31 AM J Kenneth King said...
I love comprehensions, but abusing them can lead to really dense and
difficult to read code.
I disagree on dense and difficult, although I'll leave open the
question of abuse.

b = [ item for pair in a for item in pair ]

This is exactly the code above expressed in comprehension form.

If the comprehension above is an abuse, then every nested list
comprehension is an abuse of comprehensions so they might as well not be
in the language...

I hope you didn't understand me to say that this usage is abuse.[/QUOTE]

Emile: sorry, the way I was quoting earlier posts was unclear. I was
refering to the post you quoted in your message, which describe the
nested list comprehension as abuse. As supporting evidence for my reply
I included your example of nested list comprehension, which is one of
the simplest possible ones one can think of.

 

[Emile van Sebille]

On 5/5/2009 9:15 AM J Kenneth King said...

List comprehensions can make a reader of your code apprehensive
because it can read like a run-on sentence and thus be difficult to
parse. The Python documentation discourages their use and I believe
for good reason.

Can you provide a link for this? I'd like to see specifically what's
being discouraged, as I'd be surprised to find routine usage frowned upon.

Emile

 

[J. Cliff Dyer]

A tiny improvement:

if type(el) in (list, tuple):

Another alternative, which might be useful in some cases:

if hasattr(el, '__iter__'):

This covers all iterables, not just lists and tuples.

So:

flatten([1,2, xrange(3,15), 15, 16])

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]

The downside, of course, is that some iterables might be infinite (such
as count), which would cause hidden breakage. But if you have one of
those in your data structure, why are you trying to flatten it anyway?


Cheers,
Cliff

 

[J. Cliff Dyer]

On 5/1/2009 7:31 AM J Kenneth King said...

b = []
for pair in a:
for item in pair:
b.append(item)

This is much more clear than a nested comprehension.

I love comprehensions, but abusing them can lead to really dense and
difficult to read code.

I disagree on dense and difficult, although I'll leave open the
question of abuse.

Dense and difficult may be subjective to people like you or I, but you
left out the "to read" part of that sentence. I was referring to the
negative effect nested or complex list comprehensions can have on
readability.

b = [ item for pair in a for item in pair ]

It's a clever statement, but use it once and its gone. Why not use the
expanded form in a function definition and get an even shorter, but
clearer statement?

It's also not obvious what it means. I would have thought the proper
incantation would be:

[item for item in pair for pair in a]

but that returns [5,5,5,6,6,6]. I still haven't figured out why. The
way you have to bounce your eyes back and forth in the comprehension
makes it hard to read the logic. With the loop, on the other hand it is
blatantly obvious which way the nesting occurs.

Boom, done.

Empirically, I'd have to disagree with you. The structure is changed by
the fact that "item" gets put at the beginning of the comprehension.
With the loop, each variable gets introduced in a for statement, and
then used. With the list comprehension, "item" gets used at the
beginning, and then assigned to in the for expression at the end.
Meanwhile, pair gets introduced as the loop variable in the first for
expression, and iterated over in the second for expression. So do you
read it left to right or right to left? Neither. You have to hold the
whole expression in your mind at once. Good for short and simple
comprehensions, bad as they get more complicated.

Here with some slight changes...

a = [(1, 2), (3, 4, 7), (5, 6)]
[ item for j in a if len(j)==2 for item in j if item % 2 ]

[1, 5]

This, to me, looks like line noise, approaching perl in its
unreadability.

...opposed to...

for j in a:

... if len(j)==2:
... for item in j:
... if item % 2:
... b.append(item)
...

b [1, 5]

Much nicer. Thank you.

 

[Steven D'Aprano]

List comprehensions can make a reader of your code apprehensive because
it can read like a run-on sentence and thus be difficult to parse. The
Python documentation discourages their use

Do you have a URL for such an astonishing claim?

and I believe for good
reason. It's much easier to explain complex processes with a function
rather than a single nested statement.

Sure, if you treat the function as a black box: "Input goes in, output
comes out, and I trust it does the right thing". But if you can do that
for a function, with no care about its internal complexity, then you can
do the same thing for a list comp.

What's the difference between these?

results = [complicated nested list comp]

versus

def func():
return [complicated nested list comp]

results = func()


The function call isn't *actually* easier to understand, it just allows
you to treat it as a black box and pretend you understand it. Once you
look inside the function, there's no promise that it will be simpler than
a nested list comp. Maybe it will be, and maybe it won't be.

Besides, not all list comprehensions are equivalent to nested loops. Why
label list comps as difficult to parse when only *some* of them are
difficult? That's like insisting that 1+3*x must be hard to read and
scary-looking to the average programmer, just because arithmetic
operators allow one to also write

1/(1+1/(2*x+3/(7*x+x/(5*x**2-3*x+4/(9**x+8/(-3+6/(2*x+1)))))))

 

[Steven D'Aprano]

Another alternative, which might be useful in some cases:

if hasattr(el, '__iter__'):

This covers all iterables, not just lists and tuples.

Except for the ones that it doesn't cover, like strings:
['a', 'b', 'c', 'd']


And classes that follow the sequence protocol:

.... def __getitem__(self, index):
.... if 0 <= index < 5:
.... return "spam"
.... raise IndexError
....['spam', 'spam', 'spam', 'spam', 'spam']

 

[alex23]

The way you have to bounce your eyes back and forth in the comprehension
makes it hard to read the logic.  With the loop, on the other hand it is
blatantly obvious which way the nesting occurs.

[ item for j in a if len(j)==2 for item in j if item % 2 ]
...opposed to...
for j in a:
...     if len(j)==2:
...         for item in j:
...             if item % 2:
...                 b.append(item)
...

Much nicer.  Thank you.

Apart from the presence of 'item' at the beginning of the list
comprehension as opposed to 'b.append(item)' at the end of the for-
loop, how exactly does the listcomp force you to "bounce [..] back and
forth" to follow the logic? The only other difference between the two
is in the layout - the indentation & colons - otherwise they're
structurally identical.

 

[Steven D'Aprano]

The way you have to bounce your eyes back and forth in the
comprehension makes it hard to read the logic.  With the loop, on the
other hand it is blatantly obvious which way the nesting occurs.

[ item for j in a if len(j)==2 for item in j if item % 2 ]
...opposed to...
for j in a:
...     if len(j)==2:
...         for item in j:
...             if item % 2:
...                 b.append(item) ...

Much nicer.  Thank you.

Apart from the presence of 'item' at the beginning of the list
comprehension as opposed to 'b.append(item)' at the end of the for-
loop, how exactly does the listcomp force you to "bounce [..] back and
forth" to follow the logic? The only other difference between the two is
in the layout - the indentation & colons - otherwise they're
structurally identical.

It's precisely the indentation and colons (plus newlines) that makes
nested for-loops easier to read than list-comps with multiple fors.

for a in alist:
for b in blist:
for c in clist:
if c:
parrot(a, b, c)


is much easier to read than the equivalent one-liner:

[parrot(a, b, c) for a in alist for b in blist for c in clist if c]

You can get back *nearly* all the readability by splitting the list comp
into multiple lines:

[parrot(a, b, c)
for a in alist
for b in blist
for c in clist if c
]

 

[Terry Reedy]

False

True in Python3