[ x for x in xrange(10) when p(x) ]

[bonono]

Hi,

I am wondering if there is such a thing, as python is moving away from
FP functions like dropwhile/takewhile etc.

 

[Alex Martelli]

Hi,

I am wondering if there is such a thing, as python is moving away from

This becomes a valid list comprehension by writing 'if' instead of
'when'.

FP functions like dropwhile/takewhile etc.

No way -- the itertools module is and remains a PRECIOUS resource. If
you want an iterator rather than a list, itertools.ifilter is quite
appropriate here.


Alex

 

[George Sakkis]

No way -- the itertools module is and remains a PRECIOUS resource.
If you want an iterator rather than a list, itertools.ifilter is quite
appropriate here.

What about the future of itertools in python 3K ? IIRC, several
functions and methods that currently return lists are going to return
iterators. Could this imply that itertools.(imap/ifilter/izip) will
take the place of map/filter/zip as builtins ?

George

 

[bonono]

This becomes a valid list comprehension by writing 'if' instead of
'when'.

valid, yes. efficient, I am not sure.

[ x for x in xrange(10000000) if p(x) ]

means I need to go through the whole range even if p = lambda x: x < 2.

 

[Alex Martelli]

What about the future of itertools in python 3K ? IIRC, several
functions and methods that currently return lists are going to return
iterators. Could this imply that itertools.(imap/ifilter/izip) will
take the place of map/filter/zip as builtins ?

I think the builtin namespace shouldn't get too crowded. But what I
think matters little -- what __GvR__ thinks is more important...!-)


Alex

 

[Alex Martelli]

This becomes a valid list comprehension by writing 'if' instead of
'when'.

valid, yes. efficient, I am not sure.

[ x for x in xrange(10000000) if p(x) ]

means I need to go through the whole range even if p = lambda x: x < 2.

Of course, barring semantically-very-deep optimizations (ones no
production implementation I know of ANY language would even try). And,
your POINT would be...?


Alex

 

[bonono]

I thought I read some where there these are intended to be dropped(or
at least moved out of built-in) ?

 

[George Sakkis]

This becomes a valid list comprehension by writing 'if' instead of
'when'.

valid, yes. efficient, I am not sure.

[ x for x in xrange(10000000) if p(x) ]

means I need to go through the whole range even if p = lambda x: x < 2

Itertools is your friend in this case:[0, 1]

George

 

[bonono]

I just try to use list/generator expression when possible and found I
need dropwhile/takewhile from time to time and see if there will be a
construct like this.

As I sort of think that the language in general encouraging this
style(like the talk about dropping map/filter/reduce).

 

[Steve Holden]

:

This becomes a valid list comprehension by writing 'if' instead of
'when'.

valid, yes. efficient, I am not sure.

[ x for x in xrange(10000000) if p(x) ]

means I need to go through the whole range even if p = lambda x: x < 2

Itertools is your friend in this case:

[0, 1]

Maybe, but the code also implies an esoteric knowledge that the trught
value of the predicate is monotonically decreasing (in a Boolean sense).
This would not be true if (e.g.) p = lambda x: x % 2 == 0. So while
itertools.takewhile can save you unnecessary computations, such savings
rely on provable conditions of the predicate which are frequently false.

regards
Steve

 

[bonono]

[0, 1]

thanks. that is what I am doing now, in a more generic form :

takewhile(p, (x for x in xrange(100000000)))

 

[George Sakkis]

Itertools is your friend in this case:

from itertools import takewhile
list(takewhile(p, xrange(10000000)))

[0, 1]

Maybe, but the code also implies an esoteric knowledge that the trught
value of the predicate is monotonically decreasing (in a Boolean sense).
This would not be true if (e.g.) p = lambda x: x % 2 == 0. So while
itertools.takewhile can save you unnecessary computations, such savings
rely on provable conditions of the predicate which are frequently false.

Right, it wasn't intended as a general solution for any p. For p =
lambda x: x % 2 == 0, you do have to iterate till the iterator is
consumed anyway, so a list comprehension is not less efficient (unless
you replaced it with xrange(0,10000000,2), which again implies an
esoteric knowledge of p; as usually, there's no free meal).

George

 

[Leif K-Brooks]

list(takewhile(p, xrange(10000000)))

[0, 1]

thanks. that is what I am doing now, in a more generic form :

takewhile(p, (x for x in xrange(100000000)))

How does a useless generator expression make it more generic?

 

[bonono]

list(takewhile(p, xrange(10000000)))

[0, 1]

thanks. that is what I am doing now, in a more generic form :

takewhile(p, (x for x in xrange(100000000)))

How does a useless generator expression make it more generic?

xrange is only picked as an example. I may be newbie on python but not
that dumb if all I want is a list of integer(sorted) that meets certain
criteria.

takewhile(p, (x for x in
some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))

 

[Paul Rubin]

xrange is only picked as an example. I may be newbie on python but not
that dumb if all I want is a list of integer(sorted) that meets certain
criteria.

takewhile(p, (x for x in
some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))

The generator expression is useless in that example too. some_function...
has to return an iterator, not a list, if you don't want it to use a
pile of memory. And if it returns an iterator, the generator
expression is redundant. You can pass the iterator directly to
takewhile.

 

[bonono]

The generator expression is useless in that example too. some_function...
has to return an iterator, not a list, if you don't want it to use a
pile of memory. And if it returns an iterator, the generator
expression is redundant. You can pass the iterator directly to
takewhile.

oops, my original code is much more complex than that. it is not the
pythonic way of doing things but that is what I like. Try again:

takewhile(p, ((exp1(x), exp2(y)) for (x, y) in f()))

 

[Leif K-Brooks]

xrange is only picked as an example. I may be newbie on python but not
that dumb if all I want is a list of integer(sorted) that meets certain
criteria.

takewhile(p, (x for x in
some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))

Wrapping a function in a generator expression doesn't magically make it
lazily evaluated. The whole list has to be generated and returned;
using a generator expression instead of a list comprehension just means
that it doesn't need to be copied in memory.

 

[bonono]

I use "list" in the name in "english"/general sense(say a list in
haskell is lazily evaluated), it could be a list or it could be a
lazily evaluated iterable.

The original post is really just about "when" or may be "until" syntax
that makes it a bit shorter to read and hopefuly easier to understand.

 

[Peter Hansen]

This becomes a valid list comprehension by writing 'if' instead of
'when'.

valid, yes. efficient, I am not sure.

[ x for x in xrange(10000000) if p(x) ]

means I need to go through the whole range even if p = lambda x: x < 2.

If you're looking for efficient, not to mention readable, then this is
obviously far superior:

[x for x in xrange(2)]

or even

[0, 1]

Yes, I know yours was a contrived example, but there's a time when
contrived examples stop showing us anything useful.

Do you have a real use case, where you think that the list comprehension
form is more readable or somehow better, and yet you are concerned about
its failure to magically optimize your case?

(I say "readable or somehow better" since you stated in another post "I
just try to use list/generator expression when possible" but you didn't
explain your reason for doing so. I assume you have some reason other
than arbitrary whim.)

-Peter

 

[bonono]

(I say "readable or somehow better" since you stated in another post "I
just try to use list/generator expression when possible" but you didn't
explain your reason for doing so. I assume you have some reason other
than arbitrary whim.)

The reason is simple:

I found it easier to read for me and using list/generator expression
helped me uncover a number of subtle bugs comparing with an imperative
approach.

on its own :

takewhile(lambda x: condition(x), some_generator) is not very much
difference than(well, still more things to type)

(x for x in some_generator when condition(x))

but when I have a number of them in the same expression, the
takewhile/dropwhile becomes to add up.