python3: 'where' keyword

[=?iso-8859-1?B?QW5kcuk=?=]

Darn space-eater google groups :-( Here is it again, at teh risk of
generating controversy

..def gcd(a, b):
.. where:
.. a: int, b: int
.. return c where:
.. c: int
.. while a:
.. a, b = b%a, a
.. return b
more examples can be found at aroberge.blogspot.com

André

 

[oren]

When I first saw this I thought: "hmmm... this seems as redundant as
adding a repeat/until loop to Python; there's no chance in hell it will
ever be accepted by the community or Guido, but I actually kinda like
it". It's nice to see mostly positive reactions to this idea so far.

I think it's a really ingenious solution to the the anonymous function
problem - don't make it anonymous! A short, throwaway name with a very
localized scope is as good as a truly anonymous function and feels more
Pythonic to me. We thought we wanted a better syntax than lambda for
anonymous functions but Andrey shows that perhaps it wasn't what we
really need. What we need is a solution to quickly and cleanly generate
bits of callable code without polluting the containing namespace,
without having to think too hard about unique names and while making
their temporary and local nature clear from the context. Anonymity
isn't one of the requirements.

I really liked Nick Coghlan's property example. The names 'get' and
'set' are too short and generic to be used without a proper scope but
with this syntax they are just perfect.

Here's another example:

w = Widget(color=Red, onClick=onClick, onMouseOver=onMouseOver) where:
.. def onClick(event): do_this(event.x, event.y, foo)
.. def onMouseOver(event): someotherwidget.do_that()

The "onClick=onClick" part seems a bit redundant, right? So how about
this:

w = Widget(**kw) where:
.. color = Red
.. def onClick(event): do_this(event.x, event.y, blabla)
.. def onMouseOver(event): someotherwidget.do_that()
.. x, y = 100, 200
.. kw = locals()

I'm not really sure myself how much I like this. It has a certain charm
but also feels like abuse of the feature. Note that "w =
Widget(**locals()) where:" would produce the wrong result as it will
include all the values in the containing scope, not just those defined
in the where block.

Oren

 

[Peter Hansen]

print words[3], words[5] where:
words = input.split()

- defining variables in "where" block would restrict their visibility to
one expression

Then your example above doesn't work... print takes a
sequence of expressions, not a tuple as you seem to think.

-Peter

 

[Carlos Ribeiro]

print words[3], words[5] where:
words = input.split()

- defining variables in "where" block would restrict their visibility to
one expression

Then your example above doesn't work... print takes a
sequence of expressions, not a tuple as you seem to think.

I found it strange that he had chosen to make the example with
"print", that is a statement. I'm not sure how could it be made to
work with both expressions and statements, it just seems strange...

Overall, I found the idea interesting. It seems like a obvious
counterpart to "with", in the sense that both act as modifiers to the
scoping rules. I think it can be made to work, and that it would lead
to elegant & readable code, but there are still lots of things to
consider: exception handling, fast name lookup in the "where" block,
access to symbols outside the "where" block, just to name a few.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Carl Banks]

print words[3], words[5] where:
words = input.split()

- defining variables in "where" block would restrict their visibility to
one expression

Then your example above doesn't work... print takes a
sequence of expressions, not a tuple as you seem to think.

You misunderstand. There "where" is not part of the expression but the
statement. The above example would be a modified print statement, a
print...where statement, if you will. Under this suggestion, there
would be modified versions of various simple statements.

This wouldn't be a problem parsing, of course, because "where" would be
a keyword.

 

[Paul Rubin]

You misunderstand. There "where" is not part of the expression but the
statement. The above example would be a modified print statement, a
print...where statement, if you will. Under this suggestion, there
would be modified versions of various simple statements.

You mean I can't say

# compute sqrt(2) + sqrt(3)
x = (sqrt(a) where:
a = 2.) \
+ sqrt (a) where:
a = 3.

Hmmm.

 

[Carl Banks]

And, is the whole thing after the '=' an expression? E.g.,

x = ( foo(x) where:
x = math.pi/4.0
) where:
def foo(x): print 'just for illustration', x

How would that be any improvement over this?

.. x = foo(x) where:
.. x = math.pi/4.0
.. def foo(x): print 'just for illustration', x

Can anyone think of a use case for embedding "where" inside an
expression as opposed to making it part of a simple statement? And, if
so, is the benefit of it worth the massive hit in readability.

or is this legal?

for y in ([foo(x) for x in bar] where:
bar = xrange(5)
): baz(y) where:
def baz(arg): return arg*2

Here, I can only hope not. One reason I proposed a where...do syntax
is so that, if you wanted to localize a variable to a for loop or some
other compound statement, you could do it with a minimum of fuss.

.. where:
.. bar = xrange(5)
.. def baz(arg): return arg*2
.. do:
.. for y in [foo(x) for x in bar]:
.. baz(y)

Not trying to sabotage the idea, really, just looking for

clarification ;-)

That's ok. For it fly, it's got to be able to withstand the flak.

 

[Carl Banks]

BTW, Peter, I guess I should have said "I misunderstand, but it can be
legal if you consider it part of the statements", since it appears the
author did intend it to be part of an expression.

You mean I can't say

# compute sqrt(2) + sqrt(3)
x = (sqrt(a) where:
a = 2.) \
+ sqrt (a) where:
a = 3.

Hmmm.

What would be the advantage of that over this?

.. x = sqrt(a) + sqrt(b) where:
.. a = 2.0
.. b = 3.0

Where would making "where" part of an expression rather than part of
the statement help? Can you think of a place? ("That it makes Python
more like LISP" is not a good enough answer for me, BTW. But feel free
to try.

 

[Paul Rubin]

What would be the advantage of that over this?

. x = sqrt(a) + sqrt(b) where:
. a = 2.0
. b = 3.0

The idea of "where" is to allow re-using variable names instead of
having to keep track of which ones are in use. I just tried to give a
very simple example of how you might do that more than once in a
statement.

 

[Carl Banks]

I have a different suggestion for this.

'as' is used for renaming in import statements. 'as' will be used for exception
naming in Python 3k.

So let's use it for expression naming in 'if' statements, too.

if someregexp.match(s) as m:
# blah using m
elif someotherregexp.match(s) as m:
# blah using m

What if the condition you wanted to test wasn't the same as the thing
you want to save? In other words, how would you convert this?

.. where:
.. m = something()
.. if m > 20:
.. do_something_with(m)

What you propose works for typical regexps idiom but not for the
slightly more general case. However, I could see why some people might
not like the where...if syntax I proposed; it's kind of choppy and not
exactly easy to follow at a first glance.

As a compromise, howabout:

.. if m > 20 where m=something():
.. do_something_with(m)

In this case, the m=something() is NOT an assignment statement, but
merely a syntax resembling it. The "where m=something()" is part of
the if-statement, not the if-expression. It causes m to be visisble in
the if-expression and the if-block.

It (or your suggestion) could work with a while-loop too.

.. while line where line=f.readline():
.. do_something_with(line)


The main problem here (as some would see it) is that you can't do
something this:

.. if m > 20 where (def m(): a(); b()):

 

[Bengt Richter]

How would that be any improvement over this?

Not in any way except that the idea was not to show the best code,
but to show an illustration of possible rules of parsing ;-)

. x = foo(x) where:
. x = math.pi/4.0
. def foo(x): print 'just for illustration', x

Can anyone think of a use case for embedding "where" inside an
expression as opposed to making it part of a simple statement? And, if
so, is the benefit of it worth the massive hit in readability.

I guess that might depend on the local-scope effects of 'where:' --
I think it is too early to jump to conclusions one way or the other.
At this point I just wanted to clarify what the proposal really was,
by asking questions about silly code snippets.

or is this legal?

for y in ([foo(x) for x in bar] where:
bar = xrange(5)
): baz(y) where:
def baz(arg): return arg*2

Here, I can only hope not. One reason I proposed a where...do syntax
is so that, if you wanted to localize a variable to a for loop or some
other compound statement, you could do it with a minimum of fuss.

Just because something contorted is _possible_ is not a good reason for
limiting a general capability IMO. Why not say instead,
"I can only hope no one uses it that way."?

. where:
. bar = xrange(5)
. def baz(arg): return arg*2
. do:
. for y in [foo(x) for x in bar]:
. baz(y)

Not trying to sabotage the idea, really, just looking for

clarification ;-)

That's ok. For it fly, it's got to be able to withstand the flak.

I actually like it quite a bit, in spite of the NIH ego thing ;-)

Regards,
Bengt Richter

 

[Nick Coghlan]

And, is the whole thing after the '=' an expression? E.g.,

x = ( foo(x) where:
x = math.pi/4.0
) where:
def foo(x): print 'just for illustration', x

or is this legal?

for y in ([foo(x) for x in bar] where:
bar = xrange(5)
): baz(y) where:
def baz(arg): return arg*2

Not trying to sabotage the idea, really, just looking for clarification ;-)

Actually, I was conceiving this as an addition to the grammar for the relevant
'simple statements', rather than to the grammar for expressions. Using the
assignment statement as the ongoing example:

Current:
assignment_stmt ::= (target_list "=")+ expression_list
augmented_assignment_stmt ::= target augop expression_list

New:
assignment_stmt ::= (target_list "=")+ expression_list [where_clause]
augmented_assignment_stmt ::= target augop expression_list [where_clause]
where_clause ::= "where" ":" suite

So the expressions in existing compound statements (for, while, if, elif) would
be out of luck. You could conceivably add the 'where' clause to the end of those
as well, to give statement local variables that apply to the whole compound
statement:

for y in [foo(x) for x in bar]:
baz(y)
where:
meaningful_name = xrange(5)
def baz(arg):
return arg * 2

This would only be appropriate for short loops - for long loops, the 'where'
clause gets *too* hidden.

Keeping the grammar simple might favour making the addition higher in the parse
tree:

Current:
statement ::= stmt_list NEWLINE | compound_stmt

New:
statement ::= (stmt_list NEWLINE | compound_stmt) [where_clause]
where_clause ::= "where" ":" suite

However, doing it that way allows nonsense like this:
pass where:
print "This is just plain silly!"

That would be something to be thrashed out in a PEP, though.

The name 'statement local variables' also gave me an idea for a rough
implementatation strategy.

<stmt> where:
<suite>

would be equivalent to:

def stmt_with_locals():
<suite>
<stmt>
stmt_with_locals()

For the assignment versions, the behaviour would be:

def assignment_with_locals():
<suite>
<stmt>
return <name>
<name> = assignment_with_locals()

Cheers,
Nick.

 

[Nick Coghlan]

The idea of "where" is to allow re-using variable names instead of
having to keep track of which ones are in use. I just tried to give a
very simple example of how you might do that more than once in a
statement.

I think having to keep the names unique within the statement you are currently
writing is a reasonable request

Cheers,
Nick

 

[Nick Coghlan]

What if the condition you wanted to test wasn't the same as the thing
you want to save? In other words, how would you convert this?

. where:
. m = something()
. if m > 20:
. do_something_with(m)

Yeah, this problem eventually occurred to me as well. However, I think a little
utility function can help solve it:

def test(val, condition):
if condition(val):
return val
else:
return None

if test(something(), lambda x: x < 10) as m:
print "Case 1:", m
elif test(something(), lambda x: x > 20) as m:
print "Case 2:", m
else:
print "No case at all!"

If we were to use a where clause instead, it looks like:

if test(something(), less_than(10)) as m:
print "Case 1:", m
elif test(something(), more_than(20)) as m:
print "Case 2:", m
else:
print "No case at all!"
where:
def less_than(y):
def lt(x):
return x < y
return lt

def more_than(y):
def gt(x):
return x > y
return lt

This is an example of why I don't think where clauses would completely eliminate
the utility of deferred expressions. Here's a version using my preferred syntax
from the AlternateLambdaSyntax page:

if test(something(), (def x < 10 from x)) as m:
print "Case 1:", m
elif test(something(), (def x > 20 from x)) as m:
print "Case 2:", m
else:
print "No case at all!"

Cheers,
Nick.

 

[Paul Rubin]

I think having to keep the names unique within the statement you are
currently writing is a reasonable request

Um, you could say the same thing about the function, the module, etc.

 

[Carl Banks]

Yeah, this problem eventually occurred to me as well. However, I think a little
utility function can help solve it:

def test(val, condition):
if condition(val):
return val
else:
return None

if test(something(), lambda x: x < 10) as m:
print "Case 1:", m
elif test(something(), lambda x: x > 20) as m:
print "Case 2:", m
else:
print "No case at all!"

I'm sorry, I really can't agree that this helper function "solves" it.
IMO, it's a workaround, not a solution. And, if I may be frank, it's a
pretty ugly one.

Not only that, but it still doesn't work. What if the object itself is
false?

 

[Nick Coghlan]

Heh, even further:

z = C() where:
class C(object):
...

Lets you make anonymous classes and singleton objects.

Here's another nice one if 'where' is added to compound statements as well:

@dbc(pre, post)
def foo():
pass
where:
def pre():
pass
def post():
pass

Cheers,
Nick.

 

[Donn Cave]

Quoth "Carl Banks" <[email protected]>:
....
| As a compromise, howabout:
|
| . if m > 20 where m=something():
| . do_something_with(m)
|
| In this case, the m=something() is NOT an assignment statement, but
| merely a syntax resembling it. The "where m=something()" is part of
| the if-statement, not the if-expression. It causes m to be visisble in
| the if-expression and the if-block.

If m=something() binds the function return to a name "m" for use
in other expressions, it sure IS an assignment. If it isn't an
assignment statement, it's only inasmuch as assignment has become
something other than a statement. Over whose dead body, I wonder.

In case it's of any interest, here's how "where" looks with "if"
in Haskell. It would take longer than you might imagine to explain
what that "return" is doing there, but part of it is that every "if"
must have an "else", and there is no such thing as "elif". Haskell's
layout (indent) structure is more flexible than Python's, there are
other ways this could look.

if a > 10
then putStrLn (show a)
else return ()
where
a = 5 + 6

FYI, I suppose the closest it comes to anything like "assignment as an
expression" is pattern matching -

case (regexp_group "^([^:]*): (.*)" line) of
Nothing -> f1 line
Just [a, v] -> f2 a v

-- This "unwraps" the return value of regexp_group, an imaginary
-- function of type (String -> String -> Maybe [String]). The
-- Maybe type has two values, Maybe a = Nothing | Just a.

| It (or your suggestion) could work with a while-loop too.
|
| . while line where line=f.readline():
| . do_something_with(line)
|
| The main problem here (as some would see it) is that you can't do
| something this:
|
| . if m > 20 where (def m(): a(); b()):

The way it made sense to me, "where" introduces a block. The whole
point is a private scope block. Actually kind of like the reverse
of a function, where instead of binding names to input parameters,
you in effect bind names to the scope for a sort of return-by-reference
effect. But never mind, the point is that you get a private block,
with one or more names exported to the surrounding scope in the left
hand side of the where clause. What you're trying to do here seems
to have almost nothing to do with that.

If Python 3 is going to get assignment-as-expression, it will be
because GvR accepts that as a reasonable idea. You won't bootleg it
in by trying to hide it behind this "where" notion, and you're not
doing "where" any good in trying to twist it this way either.

Donn Cave, (e-mail address removed)

 

[Nick Coghlan]

Um, you could say the same thing about the function, the module, etc.

And, indeed, that is what Python currently says. When writing code, the relevant
namespaces are the builtins, the module, any containing functions and the
current function. Inadvertent conflicts with any of those can have surprising
side effects.

The idea of 'where' is to push that down one level, and allow a namespace to be
associated with a single statement.

Trying to push it a level further (down to expressions) would, IMO, be a lot of
effort for something which would hurt readability a lot.

Compare:

x = sqrt(a) + sqrt(b) where:
a = 2.0
b = 3.0

This brings the operation we care about (add the sqrt's of 2 and 3) right up
front. A folding code editor could actually hide the details quite easily. We
can look inside the statement if we want to know what x & y actually are.

Versus:
x = (sqrt(a) where:
a = 2.) \
+ sqrt (a) where:
a = 3.

We haven't gotten rid of anything here - all the stuff we're interested in
clearing out of the way is still embedded in the middle of our statement.

Also not insignificantly, we're trying to put a suite inside an expression,
which will be rejected for all the reasons that have kept lambda restricted to a
single expression despite numerous complaints over time.

Now, nothing in the idea of a statement local namespace actually *rules out* the
prospect of an expression local namespace, so it could be added at a later date.
However, doing so would require some actual use cases, and an
expression-friendly syntax. Perhaps something that involves providing the
namespace directly, like:

x = (sqrt(a) where (a=2.0)) + (sqrt(b) where (a=3.0))

It seems to make more sense to try for statement local namespaces *first*, and
then see if expression local namespaces are worth it.

Cheers,
Nick.

 

[Paul Rubin]

x = (sqrt(a) where (a=2.0)) + (sqrt(b) where (a=3.0))

Hmm, I like that too.