For loop comprehensions

[Benjamin S Wolf]

It occurred to me as I was writing a for loop that I would like to
write it in generator comprehension syntax, eg.

for a in b if c:

rather than using one of the more verbose but allowable syntaxes:

for a in (x for x in b if c):

for a in b:
if not c: continue

Python 3.1 does not support "for comprehensions", and a few cursory
searches of PEPs and this list don't turn up anything. I like the idea
enough to suggest it though I'm unfamiliar with the PEP/feature
request process (PEP 1 pointed me here). What do other people think?

--Ben

 

[Jon Clements]

It occurred to me as I was writing a for loop that I would like to
write it in generator comprehension syntax, eg.

  for a in b if c:

rather than using one of the more verbose but allowable syntaxes:

  for a in (x for x in b if c):

  for a in b:
    if not c: continue

Python 3.1 does not support "for comprehensions", and a few cursory
searches of PEPs and this list don't turn up anything. I like the idea
enough to suggest it though I'm unfamiliar with the PEP/feature
request process (PEP 1 pointed me here). What do other people think?

--Ben

Can't help with the PEP stuff, but ummm... is anything wrong with
filter()? For some things, I personally find it more readable in some
circumstances than list/gen comps. I don't mind predicates at "the
end", but sometimes they're more obvious at "the front".

Jon.

 

[Westley Martínez]

It occurred to me as I was writing a for loop that I would like to
write it in generator comprehension syntax, eg.

for a in b if c:

rather than using one of the more verbose but allowable syntaxes:

for a in (x for x in b if c):

for a in b:
if not c: continue

Python 3.1 does not support "for comprehensions", and a few cursory
searches of PEPs and this list don't turn up anything. I like the idea
enough to suggest it though I'm unfamiliar with the PEP/feature
request process (PEP 1 pointed me here). What do other people think?

--Ben

No, too confusing. Then people'll want compound loops e.g.:

for a in b if c while d else return x:
print('Ha ha I'm so clever!')

 

[Ian Kelly]

No, too confusing. Then people'll want compound loops e.g.:

for a in b if c while d else return x:
   print('Ha ha I'm so clever!')

I've yet to see anybody arguing for while loops to be nested like
that, but we already allow nested for loops in comprehensions and
generator expressions:

[(x, y) for x in a for y in b if c(y)]

Why not allow the same thing in for-loop conditions? I think that
anything that makes the language syntax more consistent is good.

Cheers,
Ian

 

[Steven D'Aprano]

Why not allow the same thing in for-loop conditions?

Because new syntax and new language features means more work. Somebody
has to write the code, make sure that it doesn't break existing Python
code, test it for bugs, change the documentation, write regression tests
for it... And then, having done it for CPython, somebody else has to do
the same for IronPython, Jython, PyPy, CLPython, Stackless ... And all
that code has to be maintained.

Because new syntax and features increases the size and complexity of the
Python code base, which means more bugs and more tests needed.

Because new syntax and features increases the size and complexity of the
documentation needed, and hence increases the amount of things that need
to be learned.

Because the guiding philosophy of Python is "there should be one, and
preferably only one, obvious way to do it". That doesn't mean that
multiple ways of doing it are forbidden (far from it!) but it does mean
that Python is conservative about adding multiple ways of spelling the
same thing.

New features don't come for free. They have costs too. You need something
more than just "why not" to justify adding features -- you also need a
"why" as well.

So... why add extra syntax to for loops when they're already very simple
and effective?

There's nothing wrong with writing

for x in iterable:
if condition(x):
process(x)

The existing syntax is clear and obvious. There's no clear benefit to
shifting the if clause to the for expression: it complicates the parser,
and any benefit (if any!) only applies to a tiny fraction of for loops.
You save one line, which is trivial. You may save one indentation level,
which might, sometimes, be useful, but more often will also be trivial.
If there's an advantage to the suggestion, it's small.

I think that
anything that makes the language syntax more consistent is good.

"A foolish consistency is the hobgoblin of little minds."

Consistency, in and of itself, is not a virtue.

 

[Terry Reedy]

It occurred to me as I was writing a for loop that I would like to
write it in generator comprehension syntax, eg.

for a in b if c:

Already proposed and rejected. See archives for python-ideas or the
gmane.comp.python.ideas mirror.

 

[Ian Kelly]

Because new syntax and new language features means more work. Somebody
has to write the code, make sure that it doesn't break existing Python
code, test it for bugs, change the documentation, write regression tests
for it... And then, having done it for CPython, somebody else has to do
the same for IronPython, Jython, PyPy, CLPython, Stackless ... And all
that code has to be maintained.

Because new syntax and features increases the size and complexity of the
Python code base, which means more bugs and more tests needed.

Because new syntax and features increases the size and complexity of the
documentation needed, and hence increases the amount of things that need
to be learned.

Because the guiding philosophy of Python is "there should be one, and
preferably only one, obvious way to do it". That doesn't mean that
multiple ways of doing it are forbidden (far from it!) but it does mean
that Python is conservative about adding multiple ways of spelling the
same thing.

These things occurred to me, but they don't seem terribly compelling
in this instance. We're not talking about adding an entirely new
feature. We're talking about adapting an existing syntax feature to
make it more widely useful.

However, I've come up with what I think is a better reason:

for x in a for y in b if condition(a, b):
break
else:
found = False

I can argue about what I think this code should do, but it certainly
would not be clear to somebody unfamiliar with the syntax. Does the
'break' break out of the entire loop or just the inner part? Does the
'else' apply to the loop or to the 'if'?

"A foolish consistency is the hobgoblin of little minds."

If you think that quote is applicable here, then you misapprehend it.
It has nothing whatsoever to do with uniformity as a language design
principle.

 

[Benjamin S Wolf]

No, too confusing. Then people'll want compound loops e.g.:

for a in b if c while d else return x:
    print('Ha ha I'm so clever!')

There's nothing wrong with writing

for x in iterable:
    if condition(x):
        process(x)

The existing syntax is clear and obvious. There's no clear benefit to
shifting the if clause to the for expression: it complicates the parser,
and any benefit (if any!) only applies to a tiny fraction of for loops.
You save one line, which is trivial. You may save one indentation level,
which might, sometimes, be useful, but more often will also be trivial.
If there's an advantage to the suggestion, it's small.

My reasons for suggesting this are more to align for-loop syntax with
generator expression/list comprehension syntax than to add completely
new functionality to for-loops. I had observed that what I had typed
initially before realizing it was incorrect:

for a in b if c:
f(a)

was equivalent (and nearly syntactically so) to

[f(a) for a in b if c]

minus the generation of a list.

I understand there are more verbose ways to accomplish the same goal
(such as filter or multiple lines). But there are also more verbose
ways to construct a list (such as filter or multiple lines). That is
why I decided to suggest it.

Already proposed and rejected. See archives for python-ideas or the
gmane.comp.python.ideas mirror.

Thanks for pointing me in the right direction, and sorry for bringing
it up a third (if not more!) time.

For posterity, http://groups.google.com/group/python-ideas/browse_thread/thread/87eee156ac2c3a24/61621e7779b5b255,
and earlier, http://groups.google.com/group/python-ideas/browse_thread/thread/e2d076fe35ece873/862674672b4de683.

The latter brings up a good point about parsing: how will we be sure
after reading 'for a in b if c' whether this is a comprehension or
whether (b if c) begins a ternary expression (and we should expect a
'else d' to follow). Well, my suggestion is to bring for-loop syntax
in line with comprehension syntax, and:

[a for a in range(10) if False else [99]] SyntaxError: invalid syntax
[a for a in (range(10) if False else [99])] [99]
for a in range(10) if False else [99]:

print(a)
....
99

So as it stands now in 3.1, comprehensions don't permit the ternary
expression without parenthesizing it, but for-loops do. So my
suggestion would have the side effect of requiring parentheses for
that latter expression, as comprehensions do. :/

Thanks again,
--Ben