[ x for x in xrange(10) when p(x) ]

[Alex Martelli]

I use "list" in the name in "english"/general sense(say a list in
haskell is lazily evaluated), it could be a list or it could be a
lazily evaluated iterable.

OK, but the general point is:
[x for x in <whatever>] is best written list(<whatever>)
(x for x in <whatever>) is best written iter(<whatever>)
(and if <whatever> is already a list and doesn't need to be copied, or

The original post is really just about "when" or may be "until" syntax
that makes it a bit shorter to read and hopefuly easier to understand.

This is the first time on this thread in which I'm glimpsing that you
mean 'when' not as in SQL (where it has just the same meaning as the
'if' in Python's genexps/listcomps), but rather with the meaning that
any Pythonista would instinctively spell 'while'. Since AFAIK 'when' is
only used in SQL (out of widespread languages), using it with a
drastically different meaning would be an utter disaster, IMHO.

Right now, listcomps and genexps can be explained very simply as
equivalent to just the same nesting of for and if statement as they have
clauses in sequence. Adding a 'while' clause (or 'until', etc) would
unfortunately break this simple rule, and therefore make the whole
construct harder, not easier, to understand. Unless there are very
compelling and frequent use cases for such an addition, I doubt it's
worth even trying to make a patch in order to time it against
itertools.takewhile...


Alex

 

[Alex Martelli]

takewhile(lambda x: condition(x), some_generator) is not very much
difference than(well, still more things to type)

(x for x in some_generator when condition(x))

So use takewhile(condition, some_generator)

which is LESS to type. When your predicate is a function, there's no
need to wrap a lambda around it, just like there's no need to wrap an
'[x for x in' or '(x for x in' around a list/iterator.

but when I have a number of them in the same expression, the
takewhile/dropwhile becomes to add up.

Complicated expressions often become hard to read; so, break the
expression up into more readable pieces, assigning names to the
intermediate steps. There's negligible price to pay for that, since in
Python assignment is NOT a copy, just a 'naming' of an intermediate
object. For example, instead of:

for x in takefile(foo, takewhile(bar, zlupp)): ...

you may choose to code, e.g.,

zlupps_bars = takewhile(bar, zlupp)
zb_foos = takewhile(foo, zlupps_bars)
for x in zb_foos: ...

Flat is better than nested.
Sparse is better than dense.
Readability counts.


Alex

 

[bonono]

This is the first time on this thread in which I'm glimpsing that you
mean 'when' not as in SQL (where it has just the same meaning as the
'if' in Python's genexps/listcomps), but rather with the meaning that
any Pythonista would instinctively spell 'while'. Since AFAIK 'when' is
only used in SQL (out of widespread languages), using it with a
drastically different meaning would be an utter disaster, IMHO.

The original post was in the context about dropwhile/takewhile. I may
choose the wrong word "when". BTW, what does "when" do it SQL ? I only
know about "where" which I believe is the closest equvialent of "if" in
python generator comprehension/list expression.

To me when/while has a "sequence" element there whereas "where/if"
don't. I didn't choose "while" because I know it is a key word already
in python so I never thought about it being in this kind of construct,
but thinking about it, "if" can be used in list expression then may be
while can be used to.

Right now, listcomps and genexps can be explained very simply as
equivalent to just the same nesting of for and if statement as they have
clauses in sequence. Adding a 'while' clause (or 'until', etc) would
unfortunately break this simple rule, and therefore make the whole
construct harder, not easier, to understand. Unless there are very
compelling and frequent use cases for such an addition, I doubt it's
worth even trying to make a patch in order to time it against
itertools.takewhile...

I have no idea what it is involved or what about rules. I just raised a
simple question because the whole dropping filter/map/reduce discussion
I read gave me the impression that python is siding over in-line
list/generator expression rather than the more traditional
map/reduce/takewhile function call style.

 

[bonono]

So use takewhile(condition, some_generator)

which is LESS to type. When your predicate is a function, there's no
need to wrap a lambda around it, just like there's no need to wrap an
'[x for x in' or '(x for x in' around a list/iterator.

No. my predicate sometimes is not function but inline expression that
needs scoping within nested for.

Complicated expressions often become hard to read; so, break the
expression up into more readable pieces, assigning names to the
intermediate steps. There's negligible price to pay for that, since in
Python assignment is NOT a copy, just a 'naming' of an intermediate
object. For example, instead of:

for x in takefile(foo, takewhile(bar, zlupp)): ...

you may choose to code, e.g.,

zlupps_bars = takewhile(bar, zlupp)
zb_foos = takewhile(foo, zlupps_bars)
for x in zb_foos: ...

Flat is better than nested.
Sparse is better than dense.
Readability counts.

That is one opinion and I think I can decide when to break or not to
break. Sometimes, breaking them in mutiple level for with interim name
assignment is not desirable. In fact, because python allows me to write
in both style, I practiced to write the same thing in both style and
found that the nested list expression way is less error prone(at least
for me ).

 

[Colin J. Williams]

I think the builtin namespace shouldn't get too crowded. But what I
think matters little -- what __GvR__ thinks is more important...!-)


Alex

Are there generally accepted guidelines on what is appropriate for the
builtin namespace?

Colin W.

 

[Bengt Richter]

The reason is simple:

I found it easier to read for me and using list/generator expression
helped me uncover a number of subtle bugs comparing with an imperative
approach.

on its own :

takewhile(lambda x: condition(x), some_generator) is not very much
difference than(well, still more things to type)

(x for x in some_generator when condition(x))

I wish you wouldn't write "when" like that, as if it were legal python python syntax.
(Nor do I like guessing what it's supposed to mean ;-)
File "<stdin>", line 1
list (x for x in xrange(20) when x<5))
^
SyntaxError: invalid syntax

If you want to terminate a generator expression after the first sequence of elements
satisfying a condition, and you don't want to use takewhile, I don't know of a gotcha
to prevent you from just raising StopIteration, using an expression that will do that, e.g.,

>>> list (x for x in xrange(20) if x<5 or iter([]).next())

[0, 1, 2, 3, 4]

Or a bit more readably: [0, 1, 2, 3, 4]

IOW, your "when condition(x)" (IIUIC) can be spelled "if condition(x) or stop()"

but when I have a number of them in the same expression, the
takewhile/dropwhile becomes to add up.

If you don't like Alex'(s?) good advice, you can continue bracketed expressions
on several lines, and indent and group for clarity.

Regards,
Bengt Richter

 

[bonono]

If you want to terminate a generator expression after the first sequence of elements
satisfying a condition, and you don't want to use takewhile, I don't know of a gotcha
to prevent you from just raising StopIteration, using an expression that will do that, e.g.,

list (x for x in xrange(20) if x<5 or iter([]).next())

[0, 1, 2, 3, 4]

Or a bit more readably:[0, 1, 2, 3, 4]

IOW, your "when condition(x)" (IIUIC) can be spelled "if condition(x) or stop()"

If it is a single loop, takewhile/dropwhile is perfectly fine but as I
mentioned in another post, it is nested and the condition needs
surrounding scope so seperate function and StopIteration doesn't work
as it breaks out of the whole thing expression.

takewhile/dropwhile still works in this case(because of the lambda
which sees the surrounding scope).

 

[Alex Martelli]

list (x for x in xrange(20) if x<5 or iter([]).next())

[0, 1, 2, 3, 4]

Or a bit more readably:

def stop(): raise StopIteration ...
list (x for x in xrange(20) if x<5 or stop())

[0, 1, 2, 3, 4]

IOW, your "when condition(x)" (IIUIC) can be spelled "if condition(x) or
stop()"

If it is a single loop, takewhile/dropwhile is perfectly fine but as I
mentioned in another post, it is nested and the condition needs
surrounding scope so seperate function and StopIteration doesn't work
as it breaks out of the whole thing expression.

Can you give one example where this stop() function wouldn't work and
your hypothetical ``when'' would? I don't see how "it breaks out of the
whole thing expression" -- it terminates ONE for-clause (and what else
would your cherished ``when'' do?).


Alex

 

[bonono]

oops, stand corrected. I was under the impression that an exception
would break out of the current expression and forgot that the "for"
would contain it(that StopIteration is a condition to it expects to
stop it).

thanks, this is the functionality I am looking for.

 

[bonono]

IOW, your "when condition(x)" (IIUIC) can be spelled "if condition(x) or stop()"

neat trick.

 

[Bengt Richter]

If you want to terminate a generator expression after the first sequence of elements
satisfying a condition, and you don't want to use takewhile, I don't know of a gotcha
to prevent you from just raising StopIteration, using an expression that will do that, e.g.,

list (x for x in xrange(20) if x<5 or iter([]).next())

[0, 1, 2, 3, 4]

Or a bit more readably:

def stop(): raise StopIteration ...
list (x for x in xrange(20) if x<5 or stop())

[0, 1, 2, 3, 4]

IOW, your "when condition(x)" (IIUIC) can be spelled "if condition(x) or stop()"

If it is a single loop, takewhile/dropwhile is perfectly fine but as I
mentioned in another post, it is nested and the condition needs
surrounding scope so seperate function and StopIteration doesn't work
as it breaks out of the whole thing expression.

How do you mean? How did you write the expression?
E.g., there are two stop() calls in this one, but they are in different scopes:
in xrange(20) if j<4 or stop() ))
[(0, [0, 1, 2, 3, 4]), (1, [0, 1, 2, 3, 4]), (2, [0, 1, 2, 3, 4]), (3, [0, 1, 2, 3, 4])]

Or, (perhaps) more prettily written (same syntax, just spread differently):
... (i,list(xseq)) for i, xseq in enumerate(
... (x for x in xrange(20) if x<5 or stop())
... for j in xrange(20) if j<4 or stop()
... )
... )
[(0, [0, 1, 2, 3, 4]), (1, [0, 1, 2, 3, 4]), (2, [0, 1, 2, 3, 4]), (3, [0, 1, 2, 3, 4])]
I would think takewile would do the same. Might as well see:
... (i,list(twiter)) for i, twiter in enumerate(
... takewhile(lambda x:x<5, xrange(20))
... for j in takewhile(lambda x:x<4, xrange(20))
... )
... )
[(0, [0, 1, 2, 3, 4]), (1, [0, 1, 2, 3, 4]), (2, [0, 1, 2, 3, 4]), (3, [0, 1, 2, 3, 4])]

Seems ok.

takewhile/dropwhile still works in this case(because of the lambda
which sees the surrounding scope).

Could you show your code with the traceback of its failure? I've lost context ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

list (x for x in xrange(20) if x<5 or iter([]).next())
[0, 1, 2, 3, 4]

Or a bit more readably:
def stop(): raise StopIteration
...
list (x for x in xrange(20) if x<5 or stop())
[0, 1, 2, 3, 4]

IOW, your "when condition(x)" (IIUIC) can be spelled "if condition(x) or
stop()"

If it is a single loop, takewhile/dropwhile is perfectly fine but as I
mentioned in another post, it is nested and the condition needs
surrounding scope so seperate function and StopIteration doesn't work
as it breaks out of the whole thing expression.

Can you give one example where this stop() function wouldn't work and
your hypothetical ``when'' would? I don't see how "it breaks out of the
whole thing expression" -- it terminates ONE for-clause (and what else
would your cherished ``when'' do?).

Well, it seems you do have to put them in the scopes of different generators,
not just for-clauses, depending on the semantics you want e.g.,

>>> def stop(): raise StopIteration ...
>>> list( ((x,y) for x in xrange(20) if x<5 or stop() for y in xrange(20) if y<3 or stop())) [(0, 0), (0, 1), (0, 2)]
>>> list( ((x,y) for x in xrange(20) if x<5 or stop() for y in (y for y in xrange(20) if y<3 or stop())))

[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2),
(4, 0), (4, 1), (4, 2)]

Regards,
Bengt Richter

 

[bonono]

Well, it seems you do have to put them in the scopes of different generators,
not just for-clauses, depending on the semantics you want e.g.,

def stop(): raise StopIteration ...
list( ((x,y) for x in xrange(20) if x<5 or stop() for y in xrange(20) if y<3 or stop())) [(0, 0), (0, 1), (0, 2)]
list( ((x,y) for x in xrange(20) if x<5 or stop() for y in (y for y in xrange(20) if y<3 or stop())))

[(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2), (3, 0), (3, 1), (3, 2),
(4, 0), (4, 1), (4, 2)]

thanks. it works and I have explained in another post why I initially
confused and thought it didn't work. stand corrected.

However, I found something interesting which I don't quite understand :

list((x for x in [1,2,3] if x<2 or stop())) works

but

a = [ x for x in [1,2,3] if x <2 or stop() ] doesn't.

 

[Peter Otten]

However, I found something interesting which I don't quite understand :

list((x for x in [1,2,3] if x<2 or stop())) works

but

a = [ x for x in [1,2,3] if x <2 or stop() ] doesn't.

Here's how Carl Banks explained it to me when Bengt came up with this
trick.

http://mail.python.org/pipermail/python-list/2005-April/274051.html

Peter

 

[bonono]

However, I found something interesting which I don't quite understand :

list((x for x in [1,2,3] if x<2 or stop())) works

but

a = [ x for x in [1,2,3] if x <2 or stop() ] doesn't.

Here's how Carl Banks explained it to me when Bengt came up with this
trick.

http://mail.python.org/pipermail/python-list/2005-April/274051.html

Thanks, but that is even more confusing for me. I thought it is the
"for" that trap the StopIteration as in :

for x in iteratables: pass

But it said it is not ?

 

[Bengt Richter]

However, I found something interesting which I don't quite understand :

list((x for x in [1,2,3] if x<2 or stop())) works

but

a = [ x for x in [1,2,3] if x <2 or stop() ] doesn't.

Here's how Carl Banks explained it to me when Bengt came up with this
trick.

http://mail.python.org/pipermail/python-list/2005-April/274051.html

Thanks (I guess, or is guilt being pinned? ;-) for the attribution.
BTW, that thread is easier to follow on google:

http://groups.google.com/group/comp...read/thread/ca7ba6ca9d70d24c/fdcadd1e3eba61cd

But googling shows I wasn't the first to come up with iter([]).next() to raise Stopiteration. Usual suspect:

http://groups.google.com/group/comp...e4f5d3e1c2a/84c158ff18d08ece#84c158ff18d08ece

It probably stuck subconsciously in my mind and just came back when I wanted a StopIteration expression
for terminating a generator expression.

Regards,
Bengt Richter