python3: 'where' keyword

[Nick Coghlan]

I'm sorry, I really can't agree that this helper function "solves" it.
IMO, it's a workaround, not a solution. And, if I may be frank, it's a
pretty ugly one.

Heck, I thought it was ugly and I wrote it

So in reality, I'd continue to use the nested-if approach that works right now
if I wanted access to part of the condition instead of the whole thing.

However, being able to bind a name to the conditions would be handy for many cases.

Cheers,
Nick.

 

[Carl Banks]

If Python 3 is going to get assignment-as-expression, it will be
because GvR accepts that as a reasonable idea. You won't bootleg it
in by trying to hide it behind this "where" notion, and you're not
doing "where" any good in trying to twist it this way either.

I suspect you misunderstood me. When proposed this:

.. if m > 20 where m=something():
.. use(m)

the "where m=something()" part is NOT part of the if-expression. It is
an optional part of the if-statement. A very poor excuse for a BNF
grammar of the if-statment would look like this:

.. "if" expr [ "where" symbol "=" expr ] suite ...

In no way, shape, or form did I ever intend for something like this to
be possible:

.. x = (m > 20 where m=something())

Besides, if I had intended it to be an alternately-spelled assignment
expression, then it wouldn't have worked as I stated. I wanted the
thing being bound to be visible inside the if-expression and the
if-block, and to do that it must have the cooperation of the if-block,
and therefore must be part of the if-statement. If this were an
assigment expression, then m would have to be either be visible within
the whole surrounding scope, or just within that expression.

What I proposed was really nothing more than a convenient way to sneak
an extra binding inside an elif clause. (The real point here is not to
use this on if-clauses, but on elif-clauses.)

 

[Carlos Ribeiro]

Here's another nice one if 'where' is added to compound statements as well:

@dbc(pre, post)
def foo():
pass
where:
def pre():
pass
def post():
pass

Interesting. It solves some of the issues with chosing a clear syntax
for "design by contract" and other similar features that were being
debated lately. I also thought about his one:

! def foo(x=calculate_default_value()):
! pass
! where:
! def calculate_default_value():
! ...
! return <something>


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Andrey Tatarinov]

As a compromise, howabout:

. if m > 20 where m=something():
. do_something_with(m)

That's good, but first idea was about 'where' block that contains any
expressions, that we need, for example function definition. the syntax
you proposed has same problems as 'lambda'.

The main problem here (as some would see it) is that you can't do
something this:

. if m > 20 where (def m(): a(); b()):

exactly

 

[Andrey Tatarinov]

print words[3], words[5] where:
words = input.split()

- defining variables in "where" block would restrict their visibility
to one expression

Then your example above doesn't work... print takes a
sequence of expressions, not a tuple as you seem to think.

sorry, I used "expression" carelessly.

I mean that

>>> print words[3], words[5]

is a single expression
(and that would be in Python 3, when print would be subtituted with
write()/writeln()).

 

[Alex Martelli]

I don't know haskell, but it looks SQL-ish to me (only by loose

Indeed, the fact that many MANY more people are familiar with SQL than
with Haskell may be the strongest practical objection to this choice of
syntax sugar; the WHERE clause in an SQL SELECT has such wildly
different semantics from Haskell's "where" that it might engender huge
amounts of confusion. I.e., reasoning by analogy with SQL only, one
might reasonably expect that minor syntax variations on:

print a, b where a = b

could mean roughly the same as "if a==b: print a, b", rather than
roughly the same as:

a = b
print a, b

I wonder if 'with', which GvR is already on record as wanting to
introduce in 3.0, might not be overloaded instead.


Alex

 

[Andrey Tatarinov]

You mean I can't say

# compute sqrt(2) + sqrt(3)
x = (sqrt(a) where:
a = 2.) \
+ sqrt (a) where:
a = 3.

No, I'd prefer to not embed 'where' into expression.

 

[Andrey Tatarinov]

The idea of "where" is to allow re-using variable names instead of
having to keep track of which ones are in use. I just tried to give a
very simple example of how you might do that more than once in a
statement.

then I'd write something like this:

x = a + b where:
a = sqrt(n) where:
n = 2.
b = sqrt(n) where:
n = 3.

 

[Paul Rubin]

Trying to push it a level further (down to expressions) would, IMO, be
a lot of effort for something which would hurt readability a lot.

I think we should just try to do things in a simple and general way
and not try to enforce readability. For example, the
slightly-overcomplex code that I proposed might have been generated by
a macro, or even by a compiler from some other language. No human
would ever have to look at it, so it doesn't matter whether it's
easily readable. There's no reason to add needless constraints on the
language just to make writing ugly code difficult. The main goal
should be to make writing clear code easy, not to worry about whether
someone might also write ugly code.

 

[Nick Coghlan]

I wonder if 'with', which GvR is already on record as wanting to
introduce in 3.0, might not be overloaded instead.

Perhaps we could steal 'using' from the rejected decorator syntax.

x = [f(x) for x in seq] using:
def f(x):
return x * x

Cheers,
Nick.

 

[Andrey Tatarinov]

Current:
assignment_stmt ::= (target_list "=")+ expression_list
augmented_assignment_stmt ::= target augop expression_list

New:
assignment_stmt ::= (target_list "=")+ expression_list [where_clause]
augmented_assignment_stmt ::= target augop expression_list
[where_clause]
where_clause ::= "where" ":" suite

So the expressions in existing compound statements (for, while, if,
elif) would be out of luck. You could conceivably add the 'where' clause
to the end of those as well, to give statement local variables that
apply to the whole compound statement:

Nick, you're good at formalization, thanks again.

So it seems that people loved the idea of 'where' keyword, may be it's
time to think about PEP draft? I appreciate any help (cause my english
is not that good =)).

 

[Nick Coghlan]

sorry, I used "expression" carelessly.

I mean that

print words[3], words[5]

is a single expression
(and that would be in Python 3, when print would be subtituted with
write()/writeln()).

'statement' is the appropriate word in Python's grammar.

And I don't think we'd actually have to wait for Python 3 for this, we'd just
have to do the __future__ dance in order to introduce the new keyword.

Cheers,
Nick.

 

[Andrey Tatarinov]

sorry, I used "expression" carelessly.

I mean that

print words[3], words[5]

is a single expression
(and that would be in Python 3, when print would be subtituted with
write()/writeln()).

'statement' is the appropriate word in Python's grammar.

thanks )

And I don't think we'd actually have to wait for Python 3 for this, we'd
just have to do the __future__ dance in order to introduce the new keyword.

resonable, I mentioned Python3 as something in not that near future,
that'd be great to think of it earlier

 

[Nick Coghlan]

I think we should just try to do things in a simple and general way
and not try to enforce readability.

Trying to embed a suite in a Python expression is anything but simple, though

Attaching an optional local suite to every statement can actually be done quite
cleanly (so long as the parser is up to the job).

Doing the substitutions on the whole statement using distinct names will work
just as well as doing the substitutions directly on the subexpressions using
duplicate names, and moves the 'unimportant' bits after the whole thing, instead
of sprinkling them throughout the statement.

It seems a statement based approach would be:
1. Easier to read
2. Less work to implement
3. Consistent with the rest of the language
4. Works for any statements, including compound statements

The only disadvantage relative to expression local namespaces would be that you
would still have to watch for namespace conflicts within the statement - and I
have grave doubts about the structure of any statement which has that problem.

If statement local namespaces were pursued, I would see expression local
namespaces as something which hurt readability a lot without providing any new
functionality.

Cheers,
Nick.

 

[Alex Martelli]

I wonder if 'with', which GvR is already on record as wanting to
introduce in 3.0, might not be overloaded instead.

Perhaps we could steal 'using' from the rejected decorator syntax.

x = [f(x) for x in seq] using:
def f(x):
return x * x

If 'with' is going to become a keyword anyway, I suspect that many,
Guido included, would see giving 'with' two distinct syntax roles as a
lower 'price' than introducing yet another keyword, as long of course as
no ambiguity results. 'import', for example, is happily reused for both
'import foo' and 'from foo import bar'.

In terms of readability I see nothing in it, either way, between 'with'
and 'using' (and 'where', were it not for the wildly different semantics
it has in SQL) -- e.g. your example seems to read just fine with any of
these keywords.

If the use Guido intends for 'with' is something like:

with aname:
.x = .y

meaning

aname.x = aname.y

then there would be no ambiguity between this use of with as a leading
keyword in the new statement, always followed by a name; and its use in
the "with clause", where it always comes after an expression and is
immediately followed by a colon. I mean, no ambiguity for either the
parser or a human reader, I believe.

Once the AST-branch is folded back into the CVS head, playing around
with syntax should be much easier than today...


Alex

 

[Nick Coghlan]

So it seems that people loved the idea of 'where' keyword, may be it's
time to think about PEP draft? I appreciate any help (cause my english
is not that good =)).

There's certainly a PEP in the idea. Here's a summary of what we have so far in
this thread in a PEPish format. Any reference implementation should definitely
wait for the compiler upgrade to hit the main development branch in CVS though
(or be based on ast-branch if someone gets inspired in the interim).

Cheers,
Nick.

Abstract
--------
The proposal is to add the capacity for statement local namespaces to Python.
This allows a statement to be placed at the current scope, while the statement's
'setup code' is indented after the statement::

<statement> with:
<suite>

The main benefit is the avoidance of namespace pollution in the outer scope.
Sections of the statement can be extracted and named in the statement's local
namespace without any inadvertent side effects due to name conflicts.
The second benefit is that it can improve readability by allow easy grouping
of the setup code for a given expression (see the examples section)
Thirdly, it provides full-fledged effectively anonymous functions (just
define them in the statement's local namespace).
The idea is inspired by Haskell's where clause, as posted to python-list by
Andrey Tatarinov. Alex Martelli suggested using the 'with' keyword.


Grammar Change
--------------
Current::
statement ::= stmt_list NEWLINE | compound_stmt

New::
statement ::= (stmt_list NEWLINE | compound_stmt) [local_namespace]
local_namespace ::= "with" ":" suite


Semantics
---------
The code::

<statement> with:
<suite>

translates to::

def unique_name():
<suite>
<statement>
unique_name()

Assignment statements (including augmented assignment) require special handling.
The original assignment statement is translated to a return statement in the
inner scope::

<target> <assign-op> <expr> with:
<suite>

translates to::
def unique_name():
<suite>
return <expr>
<target> <assign-op> unique_name()

Function and class definitions will also require special casing, so that the
created function or class is returned from the inner-scope and the name bound
correctly in the outer scope.

Note that calling locals() inside a statement with a local namespace will refer
to the statement's locals, rather than those of the containing function.


Keyword Choice
--------------
The original suggestion on python-list used the 'where' keyword. Alex
Martelli pointed out that this could be misleading for anyone with expectations
based on SQL's WHERE clause.
He suggested 'with' as an alternative spelling, as 'with' is already planned
as a keyword for Python3k. Even for with clauses associated with compound
statements, there should be no ambiguity, given that the with statement will
have an expression between it and the colon, while the with clause does not.


Open Issues
-----------
Is it actually possible to make it work?
Keyword choice
Should the clause be allowed on any statement (as described), or restricted
to ones where it "makes sense"?


Examples
--------

# Statement local functions (from Andrey Tatarinov)
# aka How to cope if lambda goes away
res = [ f(i) for i in objects ] with:
def f(x):
#do something

# Declaring properties (from Nick Coghlan)
class C(object):
x = property(get, set) with:
def get(self):
pass
def set(self, value):
pass

# Design by contract (from Nick Coghlan)
@dbc(pre, post)
def foo():
pass
with:
def pre():
pass
def post():
pass

# Singleton classes (from Paul Rubin)
C = C() with:
class C:
pass

# Complex default values (from Carlos Ribeiro)
def f(x=default()):
pass
with:
def default():
pass

 

[Andrey Tatarinov]

Indeed, the fact that many MANY more people are familiar with SQL than
with Haskell may be the strongest practical objection to this choice of
syntax sugar; the WHERE clause in an SQL SELECT has such wildly
different semantics from Haskell's "where" that it might engender huge
amounts of confusion. I.e., reasoning by analogy with SQL only, one
might reasonably expect that minor syntax variations on:

print a, b where a = b

could mean roughly the same as "if a==b: print a, b", rather than

hmm... SQL-ish associations are good too, if you think a little deeper
then common post-effect of using SQL "where" keyword.

print a, b where a = b

means a==b in statement "print a, b"

roughly the same as:

a = b
print a, b

I wonder if 'with', which GvR is already on record as wanting to
introduce in 3.0, might not be overloaded instead.

using 'with', is some way unnatural for my point of view (may be because
translations of 'with' and 'where' into russian, are way different and
'where' translation reads as natural language while 'with' does not).

I'd like to keep 'where' keyword for proposal, but semantics are more
important, so in case I do not mind for other keyword.

 

[Andrey Tatarinov]

And about examples for usage "where" keyword

reading http://manatee.mojam.com/~skip/python/fastpython.html I
understand that almost every example should use that keyword =)

 

[Steve Holden]

Indeed, the fact that many MANY more people are familiar with SQL than
with Haskell may be the strongest practical objection to this choice of
syntax sugar; the WHERE clause in an SQL SELECT has such wildly
different semantics from Haskell's "where" that it might engender huge
amounts of confusion. I.e., reasoning by analogy with SQL only, one
might reasonably expect that minor syntax variations on:

print a, b where a = b

could mean roughly the same as "if a==b: print a, b", rather than
roughly the same as:

a = b
print a, b

I wonder if 'with', which GvR is already on record as wanting to
introduce in 3.0, might not be overloaded instead.

IIRC the proposed meaning seems to be most closely related (in my own
experience, annyway) to the use of "where" in BCPL (heavens, *that* was
a long time ago). I never found that entirely natural either.

regards
Steve

 

[Steve Holden]

Yeah, this problem eventually occurred to me as well. However, I think a
little utility function can help solve it:

def test(val, condition):
if condition(val):
return val
else:
return None

if test(something(), lambda x: x < 10) as m:
print "Case 1:", m
elif test(something(), lambda x: x > 20) as m:
print "Case 2:", m
else:
print "No case at all!"

If we were to use a where clause instead, it looks like:

if test(something(), less_than(10)) as m:
print "Case 1:", m
elif test(something(), more_than(20)) as m:
print "Case 2:", m
else:
print "No case at all!"
where:
def less_than(y):
def lt(x):
return x < y
return lt

def more_than(y):
def gt(x):
return x > y
return lt

This is an example of why I don't think where clauses would completely
eliminate the utility of deferred expressions. Here's a version using my
preferred syntax from the AlternateLambdaSyntax page:

if test(something(), (def x < 10 from x)) as m:
print "Case 1:", m
elif test(something(), (def x > 20 from x)) as m:
print "Case 2:", m
else:
print "No case at all!"

Excuse me, these are supposed to be IMPROVEMENTS to Python?

regards
Steve