[ x for x in xrange(10) when p(x) ]

Discussion in 'Python' started by bonono@gmail.com, Nov 10, 2005.

  1. Guest

    Hi,

    I am wondering if there is such a thing, as python is moving away from
    FP functions like dropwhile/takewhile etc.
     
    , Nov 10, 2005
    #1
    1. Advertising

  2. <> wrote:

    > Hi,
    >
    > I am wondering if there is such a thing, as python is moving away from


    This becomes a valid list comprehension by writing 'if' instead of
    'when'.

    > FP functions like dropwhile/takewhile etc.


    No way -- the itertools module is and remains a PRECIOUS resource. If
    you want an iterator rather than a list, itertools.ifilter is quite
    appropriate here.


    Alex
     
    Alex Martelli, Nov 10, 2005
    #2
    1. Advertising

  3. "Alex Martelli" wrote:

    > <> wrote:
    >
    > > FP functions like dropwhile/takewhile etc.

    >
    > No way -- the itertools module is and remains a PRECIOUS resource.
    > If you want an iterator rather than a list, itertools.ifilter is quite
    > appropriate here.


    What about the future of itertools in python 3K ? IIRC, several
    functions and methods that currently return lists are going to return
    iterators. Could this imply that itertools.(imap/ifilter/izip) will
    take the place of map/filter/zip as builtins ?

    George
     
    George Sakkis, Nov 10, 2005
    #3
  4. Guest

    Alex Martelli wrote:
    > This becomes a valid list comprehension by writing 'if' instead of
    > 'when'.

    valid, yes. efficient, I am not sure.

    [ x for x in xrange(10000000) if p(x) ]

    means I need to go through the whole range even if p = lambda x: x < 2.
     
    , Nov 10, 2005
    #4
  5. George Sakkis <> wrote:
    ...
    > > > FP functions like dropwhile/takewhile etc.

    > >
    > > No way -- the itertools module is and remains a PRECIOUS resource.
    > > If you want an iterator rather than a list, itertools.ifilter is quite
    > > appropriate here.

    >
    > What about the future of itertools in python 3K ? IIRC, several
    > functions and methods that currently return lists are going to return
    > iterators. Could this imply that itertools.(imap/ifilter/izip) will
    > take the place of map/filter/zip as builtins ?


    I think the builtin namespace shouldn't get too crowded. But what I
    think matters little -- what __GvR__ thinks is more important...!-)


    Alex
     
    Alex Martelli, Nov 10, 2005
    #5
  6. <> wrote:

    > Alex Martelli wrote:
    > > This becomes a valid list comprehension by writing 'if' instead of
    > > 'when'.

    > valid, yes. efficient, I am not sure.
    >
    > [ x for x in xrange(10000000) if p(x) ]
    >
    > means I need to go through the whole range even if p = lambda x: x < 2.


    Of course, barring semantically-very-deep optimizations (ones no
    production implementation I know of ANY language would even try). And,
    your POINT would be...?


    Alex
     
    Alex Martelli, Nov 10, 2005
    #6
  7. Guest

    I thought I read some where there these are intended to be dropped(or
    at least moved out of built-in) ?

    George Sakkis wrote:
    > What about the future of itertools in python 3K ? IIRC, several
    > functions and methods that currently return lists are going to return
    > iterators. Could this imply that itertools.(imap/ifilter/izip) will
    > take the place of map/filter/zip as builtins ?
    >
    > George
     
    , Nov 10, 2005
    #7
  8. "" wrote:

    > Alex Martelli wrote:
    > > This becomes a valid list comprehension by writing 'if' instead of
    > > 'when'.

    >
    > valid, yes. efficient, I am not sure.
    >
    > [ x for x in xrange(10000000) if p(x) ]
    >
    > means I need to go through the whole range even if p = lambda x: x < 2


    Itertools is your friend in this case:
    >>> from itertools import takewhile
    >>> list(takewhile(p, xrange(10000000)))

    [0, 1]

    George
     
    George Sakkis, Nov 10, 2005
    #8
  9. Guest

    I just try to use list/generator expression when possible and found I
    need dropwhile/takewhile from time to time and see if there will be a
    construct like this.

    As I sort of think that the language in general encouraging this
    style(like the talk about dropping map/filter/reduce).

    Alex Martelli wrote:
    > Of course, barring semantically-very-deep optimizations (ones no
    > production implementation I know of ANY language would even try). And,
    > your POINT would be...?
    >
    >
    > Alex
     
    , Nov 10, 2005
    #9
  10. Steve Holden Guest

    George Sakkis wrote:
    > "" wrote:
    >
    >
    >>Alex Martelli wrote:
    >>
    >>>This becomes a valid list comprehension by writing 'if' instead of
    >>>'when'.

    >>
    >>valid, yes. efficient, I am not sure.
    >>
    >>[ x for x in xrange(10000000) if p(x) ]
    >>
    >>means I need to go through the whole range even if p = lambda x: x < 2

    >
    >
    > Itertools is your friend in this case:
    >
    >>>>from itertools import takewhile
    >>>>list(takewhile(p, xrange(10000000)))

    >
    > [0, 1]


    Maybe, but the code also implies an esoteric knowledge that the trught
    value of the predicate is monotonically decreasing (in a Boolean sense).
    This would not be true if (e.g.) p = lambda x: x % 2 == 0. So while
    itertools.takewhile can save you unnecessary computations, such savings
    rely on provable conditions of the predicate which are frequently false.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Nov 10, 2005
    #10
  11. Guest

    George Sakkis wrote:
    > >>> list(takewhile(p, xrange(10000000)))

    > [0, 1]

    thanks. that is what I am doing now, in a more generic form :

    takewhile(p, (x for x in xrange(100000000)))
     
    , Nov 10, 2005
    #11
  12. "Steve Holden" <> wrote:

    > George Sakkis wrote:
    > > Itertools is your friend in this case:
    > >
    > >>>>from itertools import takewhile
    > >>>>list(takewhile(p, xrange(10000000)))

    > >
    > > [0, 1]

    >
    > Maybe, but the code also implies an esoteric knowledge that the trught
    > value of the predicate is monotonically decreasing (in a Boolean sense).
    > This would not be true if (e.g.) p = lambda x: x % 2 == 0. So while
    > itertools.takewhile can save you unnecessary computations, such savings
    > rely on provable conditions of the predicate which are frequently false.


    Right, it wasn't intended as a general solution for any p. For p =
    lambda x: x % 2 == 0, you do have to iterate till the iterator is
    consumed anyway, so a list comprehension is not less efficient (unless
    you replaced it with xrange(0,10000000,2), which again implies an
    esoteric knowledge of p; as usually, there's no free meal).

    George
     
    George Sakkis, Nov 10, 2005
    #12
  13. wrote:
    > George Sakkis wrote:
    >
    >>>>>list(takewhile(p, xrange(10000000)))

    >>
    >>[0, 1]

    >
    > thanks. that is what I am doing now, in a more generic form :
    >
    > takewhile(p, (x for x in xrange(100000000)))


    How does a useless generator expression make it more generic?
     
    Leif K-Brooks, Nov 10, 2005
    #13
  14. Guest

    Leif K-Brooks wrote:
    > wrote:
    > > George Sakkis wrote:
    > >
    > >>>>>list(takewhile(p, xrange(10000000)))
    > >>
    > >>[0, 1]

    > >
    > > thanks. that is what I am doing now, in a more generic form :
    > >
    > > takewhile(p, (x for x in xrange(100000000)))

    >
    > How does a useless generator expression make it more generic?


    xrange is only picked as an example. I may be newbie on python but not
    that dumb if all I want is a list of integer(sorted) that meets certain
    criteria.

    takewhile(p, (x for x in
    some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))
     
    , Nov 10, 2005
    #14
  15. Paul Rubin Guest

    "" <> writes:
    > > How does a useless generator expression make it more generic?

    >
    > xrange is only picked as an example. I may be newbie on python but not
    > that dumb if all I want is a list of integer(sorted) that meets certain
    > criteria.
    >
    > takewhile(p, (x for x in
    > some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))


    The generator expression is useless in that example too. some_function...
    has to return an iterator, not a list, if you don't want it to use a
    pile of memory. And if it returns an iterator, the generator
    expression is redundant. You can pass the iterator directly to
    takewhile.
     
    Paul Rubin, Nov 10, 2005
    #15
  16. Guest

    Paul Rubin wrote:
    > "" <> writes:
    > > > How does a useless generator expression make it more generic?

    > >
    > > xrange is only picked as an example. I may be newbie on python but not
    > > that dumb if all I want is a list of integer(sorted) that meets certain
    > > criteria.
    > >
    > > takewhile(p, (x for x in
    > > some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))

    >
    > The generator expression is useless in that example too. some_function...
    > has to return an iterator, not a list, if you don't want it to use a
    > pile of memory. And if it returns an iterator, the generator
    > expression is redundant. You can pass the iterator directly to
    > takewhile.

    oops, my original code is much more complex than that. it is not the
    pythonic way of doing things but that is what I like. Try again:

    takewhile(p, ((exp1(x), exp2(y)) for (x, y) in f()))
     
    , Nov 10, 2005
    #16
  17. wrote:
    > Leif K-Brooks wrote:
    >
    >> wrote:
    >>
    >>>thanks. that is what I am doing now, in a more generic form :
    >>>
    >>>takewhile(p, (x for x in xrange(100000000)))

    >>
    >>How does a useless generator expression make it more generic?

    >
    > xrange is only picked as an example. I may be newbie on python but not
    > that dumb if all I want is a list of integer(sorted) that meets certain
    > criteria.
    >
    > takewhile(p, (x for x in
    > some_function_that_could_potentially_generate_a_long_list_of_elements_but_first_element_that_meets_the_condition_can_come_fast(*args,**kwargs)))


    Wrapping a function in a generator expression doesn't magically make it
    lazily evaluated. The whole list has to be generated and returned;
    using a generator expression instead of a list comprehension just means
    that it doesn't need to be copied in memory.
     
    Leif K-Brooks, Nov 10, 2005
    #17
  18. Guest

    I use "list" in the name in "english"/general sense(say a list in
    haskell is lazily evaluated), it could be a list or it could be a
    lazily evaluated iterable.

    The original post is really just about "when" or may be "until" syntax
    that makes it a bit shorter to read and hopefuly easier to understand.

    Leif K-Brooks wrote:
    > Wrapping a function in a generator expression doesn't magically make it
    > lazily evaluated. The whole list has to be generated and returned;
    > using a generator expression instead of a list comprehension just means
    > that it doesn't need to be copied in memory.
     
    , Nov 10, 2005
    #18
  19. Peter Hansen Guest

    wrote:
    > Alex Martelli wrote:
    >
    >>This becomes a valid list comprehension by writing 'if' instead of
    >>'when'.

    >
    > valid, yes. efficient, I am not sure.
    >
    > [ x for x in xrange(10000000) if p(x) ]
    >
    > means I need to go through the whole range even if p = lambda x: x < 2.


    If you're looking for efficient, not to mention readable, then this is
    obviously far superior:

    [x for x in xrange(2)]

    or even

    [0, 1]

    Yes, I know yours was a contrived example, but there's a time when
    contrived examples stop showing us anything useful.

    Do you have a real use case, where you think that the list comprehension
    form is more readable or somehow better, and yet you are concerned about
    its failure to magically optimize your case?

    (I say "readable or somehow better" since you stated in another post "I
    just try to use list/generator expression when possible" but you didn't
    explain your reason for doing so. I assume you have some reason other
    than arbitrary whim.)

    -Peter
     
    Peter Hansen, Nov 10, 2005
    #19
  20. Guest

    Peter Hansen wrote:
    > (I say "readable or somehow better" since you stated in another post "I
    > just try to use list/generator expression when possible" but you didn't
    > explain your reason for doing so. I assume you have some reason other
    > than arbitrary whim.)

    The reason is simple:

    I found it easier to read for me and using list/generator expression
    helped me uncover a number of subtle bugs comparing with an imperative
    approach.

    on its own :

    takewhile(lambda x: condition(x), some_generator) is not very much
    difference than(well, still more things to type)

    (x for x in some_generator when condition(x))

    but when I have a number of them in the same expression, the
    takewhile/dropwhile becomes to add up.
     
    , Nov 10, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bob Gailer

    Re: xrange() question

    Bob Gailer, Aug 5, 2003, in forum: Python
    Replies:
    2
    Views:
    503
    Peter Abel
    Aug 6, 2003
  2. Christopher T King

    xrange() syntactic sugar

    Christopher T King, Jan 9, 2004, in forum: Python
    Replies:
    0
    Views:
    318
    Christopher T King
    Jan 9, 2004
  3. Gerrit Holl

    xrange not hashable - why not?

    Gerrit Holl, Jan 25, 2004, in forum: Python
    Replies:
    6
    Views:
    330
    Peter Otten
    Jan 25, 2004
  4. Hans Nowak

    Re: xrange not hashable - why not?

    Hans Nowak, Jan 26, 2004, in forum: Python
    Replies:
    1
    Views:
    336
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Jan 26, 2004
  5. Christian Neumann

    Problem with Python xrange

    Christian Neumann, Jun 6, 2004, in forum: Python
    Replies:
    7
    Views:
    2,770
    Terry Reedy
    Jun 8, 2004
Loading...

Share This Page