Re: while expression feature proposal

Discussion in 'Python' started by Ian Kelly, Oct 24, 2012.

  1. Ian Kelly

    Ian Kelly Guest

    On Wed, Oct 24, 2012 at 3:54 PM, Tim Chase
    <> wrote:
    > It may be idiomatic, but that doesn't stop it from being pretty
    > ugly. I must say I really like the parity of Dan's
    >
    > while EXPR as VAR:
    > BLOCK
    >
    > proposal with the "with" statement. It also doesn't fall prey to
    > the "mistaken-assignment vs. intentional-assignment" found in most
    > C-like languages. I could see a pretty reasonable PEP coming from this.


    Often though the while test is not a simple boolean test of VAR. For example:

    j = int(random() * n)
    while j in selected:
    j = int(random() * n)

    It also doesn't flow quite as naturally. "with x as y" is
    grammatically correct English. "while x as y" is not, and I wonder
    how easily it might be confused for "while x is y", which is valid
    Python and means something completely different.
    Ian Kelly, Oct 24, 2012
    #1
    1. Advertising

  2. Ian Kelly

    Paul Rubin Guest

    Ian Kelly <> writes:
    > j = int(random() * n)
    > while j in selected:
    > j = int(random() * n)


    from itertools import dropwhile

    j = dropwhile(lambda j: j in selected,
    iter(lambda: int(random() * n), object()))
    .next()

    kind of ugly, makes me wish for a few more itertools primitives, but I
    think it expresses reasonably directly what you are trying to do.
    Paul Rubin, Oct 25, 2012
    #2
    1. Advertising

  3. Ian Kelly

    Ian Kelly Guest

    On Wed, Oct 24, 2012 at 5:08 PM, Paul Rubin <> wrote:
    > from itertools import dropwhile
    >
    > j = dropwhile(lambda j: j in selected,
    > iter(lambda: int(random() * n), object()))
    > .next()
    >
    > kind of ugly, makes me wish for a few more itertools primitives, but I
    > think it expresses reasonably directly what you are trying to do.


    Nice, although a bit opaque. I think I prefer it as a generator expression:

    j = next(j for j in iter(partial(randrange, n), None) if j not in selected)
    Ian Kelly, Oct 25, 2012
    #3
  4. Am 25.10.2012 01:39 schrieb Ian Kelly:
    > On Wed, Oct 24, 2012 at 5:08 PM, Paul Rubin <> wrote:
    >> from itertools import dropwhile
    >>
    >> j = dropwhile(lambda j: j in selected,
    >> iter(lambda: int(random() * n), object()))
    >> .next()
    >>
    >> kind of ugly, makes me wish for a few more itertools primitives, but I
    >> think it expresses reasonably directly what you are trying to do.

    >
    > Nice, although a bit opaque. I think I prefer it as a generator expression:
    >
    > j = next(j for j in iter(partial(randrange, n), None) if j not in selected)


    This generator never ends. If it meets a non-matching value, it just
    skips it and goes on.

    The dropwhile expression, however, stops as soon as the value is found.

    I think

    # iterate ad inf., because partial never returns None:
    i1 = iter(partial(randrange, n), None)
    # take the next value, make it None for breaking:
    i2 = (j if j in selected else None for j in i1)
    # and now, break on None:
    i3 = iter(lambda: next(i2), None)

    would do the job.


    Thomas
    Thomas Rachel, Oct 25, 2012
    #4
  5. Ian Kelly

    Paul Rudin Guest

    Paul Rubin <> writes:

    > kind of ugly, makes me wish for a few more itertools primitives


    JOOI, do you have specific primitives in mind?
    Paul Rudin, Oct 25, 2012
    #5
  6. Am 25.10.2012 09:21 schrieb Thomas Rachel:

    > I think
    >
    > # iterate ad inf., because partial never returns None:
    > i1 = iter(partial(randrange, n), None)
    > # take the next value, make it None for breaking:
    > i2 = (j if j in selected else None for j in i1)
    > # and now, break on None:
    > i3 = iter(lambda: next(i2), None)
    >
    > would do the job.


    But, as I read it now again, it might be cleaner to create an own
    generator function, such as

    def rand_values(randrange, n, selected):
    # maybe: selected = set(selected) for the "not in"
    while True:
    val = partial(randrange, n)
    if val not in selected: break
    yield val

    for value in rand_values(...):

    or, for the general case proposed some posings ago:

    def while_values(func, *a, **k):
    while True:
    val = func(*a, **k):
    if not val: break
    yield val

    Thomas
    Thomas Rachel, Oct 25, 2012
    #6
  7. Ian Kelly

    Ian Kelly Guest

    On Thu, Oct 25, 2012 at 1:21 AM, Thomas Rachel
    <>
    wrote:
    >> j = next(j for j in iter(partial(randrange, n), None) if j not in
    >> selected)

    >
    >
    > This generator never ends. If it meets a non-matching value, it just skips
    > it and goes on.


    next() only returns one value. After it is returned, the generator is
    discarded, whether it has ended or not. If there were no valid values
    for randrange to select, then it would descend into an infinite loop.
    But then, so would the dropwhile and the original while loop.
    Ian Kelly, Oct 25, 2012
    #7
  8. Ian Kelly

    Ian Kelly Guest

    On Thu, Oct 25, 2012 at 10:36 AM, Ian Kelly <> wrote:
    > On Thu, Oct 25, 2012 at 1:21 AM, Thomas Rachel
    > <>
    > wrote:
    >>> j = next(j for j in iter(partial(randrange, n), None) if j not in
    >>> selected)

    >>
    >>
    >> This generator never ends. If it meets a non-matching value, it just skips
    >> it and goes on.

    >
    > next() only returns one value. After it is returned, the generator is
    > discarded, whether it has ended or not. If there were no valid values
    > for randrange to select, then it would descend into an infinite loop.
    > But then, so would the dropwhile and the original while loop.


    To demonstrate that the code does in fact return:

    >>> selected = set(range(5))
    >>> n = 10
    >>> from functools import partial
    >>> from random import randrange
    >>> j = next(j for j in iter(partial(randrange, n), None) if j not in selected)
    >>> j

    5
    Ian Kelly, Oct 25, 2012
    #8
  9. Am 25.10.2012 18:36 schrieb Ian Kelly:
    > On Thu, Oct 25, 2012 at 1:21 AM, Thomas Rachel
    > <>
    > wrote:
    >>> j = next(j for j in iter(partial(randrange, n), None) if j not in
    >>> selected)

    >>
    >>
    >> This generator never ends. If it meets a non-matching value, it just skips
    >> it and goes on.

    >
    > next() only returns one value. After it is returned, the generator is
    > discarded, whether it has ended or not. If there were no valid values
    > for randrange to select, then it would descend into an infinite loop.
    > But then, so would the dropwhile and the original while loop.


    You are completely right. My solution was right as well, but didn't
    match the problem...

    Yours does indeed return one random value which is guaranteed not to be
    in selected.

    Mine returns random values until the value is not in selected. I just
    misread the intention behind the while loop...


    Thomas
    Thomas Rachel, Oct 25, 2012
    #9
  10. It seems the topic of this thread has changed drastically from the originalmessage.

    1) "while EXPR as VAR" in no way says that EXPR must be a boolean value. Infact, a use case I've run into commonly in web development is popping froma redis set. E.g.

    client = StrictRedis()
    while True:
    profile_id = client.spop("profile_ids")
    if not profile_id:
    break
    print profile_id

    In this case, profile_id is "None" when the loop breaks. It would be much more straightforward (and more Pythonic, IMO), to write:

    client = StrictRedis()
    while client.spop("profile_ids") as profile_id:
    print profile_id

    2) Although not originally intended, I kind of like the "if" statement change proposed later in this thread. It certainly makes sense, since both while and if are "conditional" statements that are commonly followed by an assignment (or vice versa).

    3) I don't think the use case I brought up is solved nicely by wrapping a function / lambda in a generator and using a for loop. E.g.

    def helper(f):
    value = f()
    if value:
    yield value

    for profile_id in helper(lambda: client.spop("profile_ids")):
    print profile_id

    This works too, I guess

    def helper(f, *args, **kwargs):
    value = f(*args, **kwargs)
    if value:
    yield value

    for profile_id in helper(client.spop, "profile_ids"):
    print profile_id

    Either way, it adds too much mental overhead. Every developer on a project has to now insert x lines of code before a for loop or import a helper method from some module, and do this every time this pattern reappears. It's not something I would want to do in one of my projects, since it makes thingsharder to understand. So all in all, it's a net negative from just doing things the canonical way (with the while / assignment pattern).

    Dan
    Dan Loewenherz, Oct 26, 2012
    #10
  11. Ian Kelly

    Paul Rubin Guest

    Dan Loewenherz <> writes:
    > In this case, profile_id is "None" when the loop breaks. It would be
    > much more straightforward (and more Pythonic, IMO), to write:
    >
    > client = StrictRedis()
    > while client.spop("profile_ids") as profile_id:
    > print profile_id


    That is pretty loose, in my opinion. If the loop is supposed to return
    a string until breaking on None, the break test should explicitly check
    for None rather than rely on an implicit bool conversion that will also
    test as false on an empty string. Code that handles strings should do
    the right thing with the empty string. What you posted relies on an
    unstated assumption that the strings that come back are never empty.

    > it's a net negative from just doing things the canonical way (with the
    > while / assignment pattern).


    Yeah, the while/assignment is a bit ugly but it doesn't come up often
    enough to be a bad problem, imho.
    Paul Rubin, Oct 26, 2012
    #11
  12. On Fri, Oct 26, 2012 at 5:06 PM, Paul Rubin <> wrote:
    > Dan Loewenherz <> writes:
    >> In this case, profile_id is "None" when the loop breaks. It would be
    >> much more straightforward (and more Pythonic, IMO), to write:
    >>
    >> client = StrictRedis()
    >> while client.spop("profile_ids") as profile_id:
    >> print profile_id

    >
    > That is pretty loose, in my opinion. If the loop is supposed to return
    > a string until breaking on None, the break test should explicitly check
    > for None rather than rely on an implicit bool conversion that will also
    > test as false on an empty string.


    while (client.spop("profile_ids") as profile_id) is not None:
    print profile_id

    Why is everyone skirting around C-style assignment expressions as
    though they're simultaneously anathema and the goal? :)

    But seriously, this new syntax would probably enhance Python somewhat,
    but you're going to end up with odd edge cases where it's just as
    almost-there as current syntax is for what this will solve. Is it
    worth doing half the job?

    ChrisA
    Chris Angelico, Oct 26, 2012
    #12
  13. On Fri, 26 Oct 2012 17:23:12 +1100, Chris Angelico wrote:

    > Why is everyone skirting around C-style assignment expressions as though
    > they're simultaneously anathema and the goal? :)


    Only if your goal is to introduce an anathema :p


    --
    Steven
    Steven D'Aprano, Oct 26, 2012
    #13
  14. On Thursday, October 25, 2012 11:06:01 PM UTC-7, Paul Rubin wrote:
    > Dan Loewenherz <> writes:
    >
    > > In this case, profile_id is "None" when the loop breaks. It would be

    >
    > > much more straightforward (and more Pythonic, IMO), to write:

    >
    > >

    >
    > > client = StrictRedis()

    >
    > > while client.spop("profile_ids") as profile_id:

    >
    > > print profile_id

    >
    >
    >
    > That is pretty loose, in my opinion. If the loop is supposed to return
    >
    > a string until breaking on None, the break test should explicitly check
    >
    > for None rather than rely on an implicit bool conversion that will also
    >
    > test as false on an empty string. Code that handles strings should do
    >
    > the right thing with the empty string. What you posted relies on an
    >
    > unstated assumption that the strings that come back are never empty.
    >


    I think this is a good point. However, I can't think of any situation whereI'd want to work with an empty string (in the applications I've worked with, at least).

    We also don't special case things like this just because x is an empty string. If this "while EXPR as VAR" thing were to move forward, we shouldn't treat the truth testing any differently than how we already do. IMO we shouldwrite our applications with the understanding that '' will return False and work with that.

    Here's a workaround BTW. Just have that method return a tuple, and do the truth testing yourself if you feel it's necessary.

    while client.spop("profile_ids") as truthy, profile_id:
    if not truthy:
    break

    print profile_id

    Here, client.spop returns a tuple, which will always returns true. We then extract the first element and run a truth test on it. The function we use is in charge of determining the truthiness.

    Dan
    Dan Loewenherz, Oct 26, 2012
    #14
  15. Ian Kelly

    Ian Kelly Guest

    On Fri, Oct 26, 2012 at 9:29 AM, Dan Loewenherz <> wrote:
    > while client.spop("profile_ids") as truthy, profile_id:
    > if not truthy:
    > break
    >
    > print profile_id
    >
    > Here, client.spop returns a tuple, which will always returns true. We then extract the first element and run a truth test on it. The function we use is in charge of determining the truthiness.


    I don't like the idea of testing the first element. There's a large
    element of surprise in doing that, I think. I would expect the truth
    test to be the same with or without the existence of the "as" clause
    there. That is, you should be able to remove the "as" clause and have
    exactly the same behavior, just without the assignments. So it would
    need to test the entire tuple.

    That brings up an interesting additional question in my mind, though.
    Should the while loop syntax attempt to perform the assignment on the
    very last test, when the expression is false? I think there is a good
    argument for doing so, as it will allow additional inspection of the
    false value, if necessary. In the above, though, if the return value
    is false (an empty tuple or None) then the assignment would fail
    during unpacking, raising an exception.
    Ian Kelly, Oct 26, 2012
    #15
  16. Ian Kelly

    Paul Rubin Guest

    Dan Loewenherz <> writes:
    > We also don't special case things like this just because x is an empty
    > string. If this "while EXPR as VAR" thing were to move forward, we
    > shouldn't treat the truth testing any differently than how we already
    > do. IMO we should write our applications with the understanding that
    > '' will return False and work with that.


    We don't "already" treat the truth testing any particular way because we
    don't have this construction in the language at the moment. However,
    it's well-established in Python that converting a string to a bool
    results in False iff the string is empty.

    The empty string is a perfectly good string and code that deals with
    strings should handle the empty string properly, unless it knows the
    string won't be empty. Basic modularity principles say to avoid putting
    such knowledge into more of the code than necessary.

    The conclusion is to not automatically convert the parameter to a bool.
    However, if the "as" can be part of an expression as in Chris Angelico's
    post, Chris's suggestion

    while (client.spop("profile_ids") as profile_id) is not None:
    print profile_id

    looks good to me.

    > while client.spop("profile_ids") as truthy, profile_id:
    > if not truthy:
    > break


    This is ugly on two levels. First of all, if the .spop() still returns
    None at the end of the input, the tuple unpacking will fail. Second,
    the separate test and break defeats the purpose of the "while ... as"
    construction. Might as well use the current style of assignment and
    test inside the loop.
    Paul Rubin, Oct 26, 2012
    #16
  17. On 26Oct2012 09:10, Paul Rubin <> wrote:
    | However, if the "as" can be part of an expression as in Chris Angelico's
    | post, Chris's suggestion
    |
    | while (client.spop("profile_ids") as profile_id) is not None:
    | print profile_id
    |
    | looks good to me.

    Now this pulls me from a -0 to a +0.5.

    Instead of burdening the control constructs with further structure, make
    "as" a binding operation for keeping intermediate results from expressions.

    It will work anywhere an expression is allowed, and superficially
    doesn't break stuff that exists if "as" has the lowest precedence.

    Any doco would need to make it clear that no order of operation is
    implied, so that this:

    x = 1
    y = (2 as x) + x

    does not have a defined answer; might be 2, might be 3. Just like any
    other function call with side effects.

    Speaking for myself (of course!), I definitely prefer this to adding
    "as" as a post expression struction on if/while/etc.

    I'm not +1 because to my mind it still presents a way for
    assignment/binding to not be glaringly obvious at the left hand side of
    an expression.

    It would probably mean folding the except/with "as" uses back into
    expressions and out of the control-structural part of the grammar. I can't
    see that that would actually break any existing code though - anyone else?

    Cheers,
    --
    Cameron Simpson <>

    UNIX was not designed to stop you from doing stupid things, because that
    would also stop you from doing clever things. - Doug Gwyn
    Cameron Simpson, Oct 26, 2012
    #17
  18. Ian Kelly

    Ian Kelly Guest

    On Fri, Oct 26, 2012 at 4:03 PM, Cameron Simpson <> wrote:
    > It will work anywhere an expression is allowed, and superficially
    > doesn't break stuff that exists if "as" has the lowest precedence.


    Please, no. There is no need for it outside of while expressions, and
    anywhere else it's just going to be bad practice. Even if it's
    considered an expression, let's only allow it in while expressions.

    > Any doco would need to make it clear that no order of operation is
    > implied, so that this:
    >
    > x = 1
    > y = (2 as x) + x
    >
    > does not have a defined answer; might be 2, might be 3. Just like any
    > other function call with side effects.


    Actually, the docs are clear that expressions are evaluated left to
    right, so the expected result of the above would be 4.

    > It would probably mean folding the except/with "as" uses back into
    > expressions and out of the control-structural part of the grammar. I can't
    > see that that would actually break any existing code though - anyone else?


    Yes it would, because the meaning is a bit different in both of those
    cases. For except, the result of the expression (an exception class
    or tuple of classes) is not stored in the target; the exception
    *instance* is. Similarly for with, the result of the expression is
    not stored; the result of calling its __enter__ method is, which is
    often but not always the same thing.
    Ian Kelly, Oct 26, 2012
    #18
  19. On Fri, Oct 26, 2012 at 2:23 AM, Chris Angelico <> wrote:
    > while (client.spop("profile_ids") as profile_id) is not None:
    > print profile_id
    >
    > Why is everyone skirting around C-style assignment expressions as
    > though they're simultaneously anathema and the goal? :)


    Why should these two statements behave differently? :(

    with foo() as bar: bar.baz()
    with (foo() as bar): bar.baz()

    I don't understand why everyone is so attached to this "as" syntax.
    It's confusing because it behaves subtly differently than how it works
    in "with", and it puts the variable on the wrong side of the
    assignment operator.

    (I've always been partial to ":=", personally.)

    -- Devin
    Devin Jeanpierre, Oct 27, 2012
    #19
  20. Ian Kelly

    Tim Chase Guest

    On 10/26/12 17:03, Cameron Simpson wrote:
    > On 26Oct2012 09:10, Paul Rubin <> wrote:
    > | while (client.spop("profile_ids") as profile_id) is not None:
    >
    > Now this pulls me from a -0 to a +0.5.
    >
    > Any doco would need to make it clear that no order of operation is
    > implied, so that this:
    >
    > x = 1
    > y = (2 as x) + x
    >
    > does not have a defined answer; might be 2, might be 3. Just like any
    > other function call with side effects.


    I really don't like undefined (or underdefined) specs. If it was to
    be PEP'd out, I'd want to address as many edge cases as possible.
    Such as

    y = (2 as x) + (3 as x) + (4 as x)
    y = (2 as x) + 4 as x
    y = booleanish and (2 as x) or (4 as x)
    y = booleanish and 2 or 4 as x
    y = (2 as x) if booleanish else (3 as x)
    y = (2 as x) if booleanish else (3 as z)

    regardless of how "$PEJORATIVE, that's a dumb thing to do!" it is.

    I hate C for how underdefined a lot of corners are. ("amongst my
    hatreds of C are such diverse elements as: underdefined corners, a
    pitiful standard library, the ease of shooting yourself in the foot,
    ....")

    > I'm not +1 because to my mind it still presents a way for
    > assignment/binding to not be glaringly obvious at the left hand side of
    > an expression.


    I think this is why I like it in the "while" (and could be twisted
    into accepting it for "if") because it also introduces an
    implied/actual scope for which the variable is intended. In an
    arbitrary evaluation/assignment, it's much easier to lose the
    "definition" nature of it at the top of a block.

    -tkc
    Tim Chase, Oct 27, 2012
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dan Loewenherz

    while expression feature proposal

    Dan Loewenherz, Oct 24, 2012, in forum: Python
    Replies:
    1
    Views:
    145
    Paul Rubin
    Oct 24, 2012
  2. Ian Kelly
    Replies:
    0
    Views:
    132
    Ian Kelly
    Oct 24, 2012
  3. Tim Chase
    Replies:
    0
    Views:
    150
    Tim Chase
    Oct 24, 2012
  4. Cameron Simpson

    Re: while expression feature proposal

    Cameron Simpson, Oct 24, 2012, in forum: Python
    Replies:
    6
    Views:
    162
    Thomas Rachel
    Oct 25, 2012
  5. Chris Angelico

    Re: while expression feature proposal

    Chris Angelico, Oct 24, 2012, in forum: Python
    Replies:
    0
    Views:
    178
    Chris Angelico
    Oct 24, 2012
Loading...

Share This Page