attaching names to subexpressions

[Steve Howell]

I have been reading the thread "while expression feature proposal,"
and one of the interesting outcomes of the thread is the idea that
Python could allow you to attach names to subexpressions, much like C
allows. In C you can say something like this:

tax_next_year = (new_salary = salary * (1 + raise)) * tax_rate

To avoid the "=" pitfall, folks have proposed something like this for
Python:

tax_next_year = ((salary * (1 + raise)) as new_salary) * tax_rate
print new_salary, tax_next_year

The basic rule in Python is that you can only do one assignment per
line of code, which generally forces you to write more readable code
IMHO:

new_salary = salary * (1 + raise)
tax_next_year = new_salary * tax_rate
print new_salary, tax_next_year

The above code is slightly more verbose than the "as" proposal would
permit, but the latter code is arguably easier for a human to parse,
and it's also very amenable to print debugging and/or defensive
coding:

new_salary = salary * (1 + raise)
print new_salary
assert new_salary > salary
tax_next_year = new_salary * tax_rate
print new_salary, tax_next_year

If the problem statement is "How do I name subexpression?", then
Python already has a clear path--break your code up into multiple
lines. I'm wondering where this simple solution really breaks down
from a readability perspective. Perhaps with short-circuited boolean
expressions?

 

[Gelonida N]

I have been reading the thread "while expression feature proposal,"
and one of the interesting outcomes of the thread is the idea that
Python could allow you to attach names to subexpressions, much like C
allows. In C you can say something like this:

tax_next_year = (new_salary = salary * (1 + raise)) * tax_rate

To avoid the "=" pitfall, folks have proposed something like this for
Python:

tax_next_year = ((salary * (1 + raise)) as new_salary) * tax_rate
print new_salary, tax_next_year
.. . .

If the problem statement is "How do I name subexpression?", then
Python already has a clear path--break your code up into multiple
lines. I'm wondering where this simple solution really breaks down
from a readability perspective. Perhaps with short-circuited boolean
expressions?

For me two places where expression assignemts can be useful are while
loops and sometimes elif statements.


While example
----------------

line = complex_exression
while line:
do something with(line)
line = complex_expression

violates clearly the DRY principle and the risk to change one line but
not the other is high.


The 'canonical way'
while True:
line = complex_expression
if not line:
break
do_something_with(line)

avoids this problem, but I was never really convinced about the
beauty / readbility of this construct.

One misses the exit condition of the while loop on the first glance.
In my opinion I shouldn't be obliged to read any of the indented lines
of the while statement on a first 'visual' pass through somebody elses
code and still be able to see what the loop iterates through.


Following looks in my opinion nicer.

while complex_expression as line:
do_something_with(line)

or the even more powerful suggestion:

while (complex_expression as line) is not None:
do_something_with(line)


Elif Example:
--------------

value1 = expression1
if (value1):
do_something
else:
value2 = expression2
if(value2):
do_something_with(value2)
else:
value2 = expression3
if(value3):
do_something_with(value3)

Could be rewritten as

value1= expression1
if(value1):
do_something_with(value1)
elif(expression2 as value2):
do_something_with(value2)
elif(expression3 as value3):
do_something_with(value3)



However in all other cases I really think using this new syntax would
reduce readability andit would be better to just split the statement
into two lines.

for while / elif statements splitting up is not possible without doing
some further acrobatics, which render in my opinion the code less readable.


If the new syntax were adopted, then one open question
would be scoping.

value=expression
if(value):
do_something_with(value)
elif(expression2 as value):
do_something_with(value)
elif(expression3 as value):
do_something_with(value)
print value # will value leak out of the elif statements or not

I never tried to use other 'as-variables' (e.g. from 'with' or 'except'
statements) outside of their indented block,
so I never bothered to check how Python would react.

 

[Steven D'Aprano]

line = complex_exression
while line:
do something with(line)
line = complex_expression

violates clearly the DRY principle and the risk to change one line but
not the other is high.

Only if the change is being made by an incompetent programmer.

DRY is not a law of physics. Not every expression needs to be written
once and exactly once or your code will become an unmaintainable mess. In
the example shown, the two instances of the expression are separated by
TWO lines -- if somebody edits one without editing the other, they are
probably not fit to be editing the code in the first place.

Repeated code is a code-smell, not a crime. You should always be on the
look-out for ways to minimise repeated code, but that doesn't mean every
time you see some trivial chunk of code you need to refactor it out.

If "complex_expression" actually is complex, then it should be factored
out into a function:

line = function(x, y, z)
while line:
do something with(line)
line = function(x, y, z)

so that the actual complexity is in one place, and any changes to it
needs to be made in one place. If it is not actually complex, then who
cares? So what if you have to write "line = next(iterator).strip()" twice?

If the body of the while loop is big enough that there genuinely is a
risk that you won't notice the two assignments to line, then you should
refactor the body of the while loop. Or change the loop:

while True:
line = function(x, y, z)
if not line:
break
big ugly block of code


There you go. DRY.

There is nothing in this proposal for while assignment that cannot be
*trivially* solved using existing syntax.

The 'canonical way'
while True:
line = complex_expression
if not line:
break
do_something_with(line)

avoids this problem, but I was never really convinced about the beauty /
readbility of this construct.

It's sure as hell more beautiful and readable than assignment as an
expression.

One misses the exit condition of the while loop on the first glance.

If we are going to judge code on the ability of people to take a quick
glance and immediately understand it, then pretty much nothing but
trivial one-liners will pass. Certainly assignment as an expression won't:

while (look_ahead(it) and next(it) or default).strip().lower() as line:
spam(x)

is easy to miss that it is an assignment to x. *Much* easier than missing
an indented "break".

In
my opinion I shouldn't be obliged to read any of the indented lines of
the while statement on a first 'visual' pass through somebody elses code
and still be able to see what the loop iterates through.

Fine. Then write your code as:

line = function(x, y, z)
while line:
do something with(line)
line = function(x, y, z)

Following looks in my opinion nicer.

while complex_expression as line:
do_something_with(line)

That's only because you don't actually have a complex expression. Just
writing "complex_expression" doesn't make it complex -- it's actually
trivially simple, a mere rebinding of a name to a name, no more "complex"
than

while x as y:
...

See how easy to read it is! Well duh. But write out an *actual* complex
expression, and the "as name" can easily disappear into the noise.

Complex expressions are a code smell at least as bad as violating DRY,
and frequently much worse. If the expression is too complex to write
twice, refactor it into a function, *and write tests for the function*.
Otherwise that complex expression is a bug waiting to bite, whether you
write it once or twice.

 

[rusi]

It's sure as hell more beautiful and readable than assignment as an
expression.

If we are going to judge code on the ability of people to take a quick
glance and immediately understand it, then pretty much nothing but
trivial one-liners will pass. Certainly assignment as an expression won't:

while (look_ahead(it) and next(it) or default).strip().lower() as line:
    spam(x)

is easy to miss that it is an assignment to x. *Much* easier than missing
an indented "break".


Fine. Then write your code as:

line = function(x, y, z)
while line:
     do something with(line)
     line = function(x, y, z)



That's only because you don't actually have a complex expression. Just
writing "complex_expression" doesn't make it complex -- it's actually
trivially simple, a mere rebinding of a name to a name, no more "complex"
than

while x as y:
    ...

See how easy to read it is! Well duh. But write out an *actual* complex
expression, and the "as name" can easily disappear into the noise.

Complex expressions are a code smell at least as bad as violating DRY,
and frequently much worse. If the expression is too complex to write
twice, refactor it into a function, *and write tests for the function*.
Otherwise that complex expression is a bug waiting to bite, whether you
write it once or twice.

The details of the various syntaxes proposed Ive not fully gone into
and I am not discussing.
The general notion of having assignments inside expressions would put
python into the C bracket
I had written about the 'all-hell-breaks-loose' when using C to teach
programming:
http://www.the-magus.in/Publications/chor.pdf

That paper is dated... if C is dated. Otherwise I would request
people having this idea to read it.

In particular, with respect to this suggestion, heres how things tend
to pan out when the language allows assignments inside expressions:
You cant really *teach* students to use
i=i+1;
instead of
i++;
because its unidiomatic C.

And then someone (with no malice just ignorance) writes:
i=i++;
What does the teacher do? You can simply say: "Dont do that!!" and
confuse the students with what exactly is the set of "Dont-dos" (or
worse leave them smug that they understand when they dont)

Or you can explain why it could increment i or leave it unchanged and
so is indeterminate.

So I would take the 'muscular' approach -- Show expressions like:
i=i++-1;
1. Explain why it could increment, decrement or leave i unchanged
2. Elaborate compilation strategies for each of the above
3. Work out sufficient conditions for the behavior of an expression to
be determinate, viz. Variables that are assigned inside an expression,
should have only a single occurence, multiply occurring variables
should not be assigned. [And assignment means all forms: = += ++ etc)

Naturally when students are taught like this, they tend to become
cowboys -- learning all kinds of tricks of a narrow and useless sort
and ignoring more important aspects of their studies. And if python
chooses such a direction its sad (for us teachers)

Having said this I need to add: the imperatives of teaching and
professional development are not the same. If professional developers
feel that adding confusing semantics for the sake of saving lines of
code is a benediction... Well...

 

[Devin Jeanpierre]

-- snip --
while True:
line = function(x, y, z)
if not line:
break
big ugly block of code


There you go. DRY.

Fine. Then write your code as:

line = function(x, y, z)
while line:
do something with(line)
line = function(x, y, z)

We have a problem, and two solutions. Solution 1 has downside A, and
solution 2 has downside B. If he complains about downside A, you say,
well, use solution 2. If he complains about downside B, you say, well,
use solution 1.

What if he wants to avoid both downsides A and B? What solution does
he use then?

-- Devin

 

[Chris Angelico]

What if he wants to avoid both downsides A and B? What solution does
he use then?

He switches to a language whose BDFL is not Steven D'Aprano.

No offense meant Steven...

ChrisA

 

[Steven D'Aprano]

We have a problem, and two solutions. Solution 1 has downside A, and
solution 2 has downside B. If he complains about downside A, you say,
well, use solution 2. If he complains about downside B, you say, well,
use solution 1.

What if he wants to avoid both downsides A and B? What solution does he
use then?

"I want to have my cake, and eat it too." Every solution has some
downside. Just because no other solution is perfect (whatever that
means!) doesn't mean we must keep adding more and more ways to solve the
same problem into a language.

The proposed solution, assignment as an expression, has multiple
downsides:

* Since it doesn't actually exist yet, you can't use it. It's
not a solution until *at least* Python 3.4. The first alpha
is scheduled in August 2013, the first stable release is not
due until end of Feb 2014. So you have to wait 16 months
before you can use this in production code.

* If you support multiple versions of Python, you can't use
this. Assuming, conservatively, that you need to support
the current version of Python plus two older versions, that
means you can't use this until Python 3.6, which is probably
due out in 2017. "Wait five years" is hardly a solution for
code you are writing today.

* You can't just wave a magic wand and have Python support this
new syntax. Somebody has to write a PEP and get it approved;
somebody has to modify the parser, write documentation for it,
write tests, ensure it doesn't break anything.

* And once it does exist, it adds to the complexity of Python
the language. People learning the language have one more
feature to learn. Every time you write a loop, you have one
more decision to make -- should I write this loop using A,
or B, or C?

* Like all features, it is open to abuse. Does the benefit
from the good uses outweigh the cost of the abuses? Given
that I think the benefit is at best minor, I doubt that it
will outweigh the harm to readability and maintainability
when it is abused.

* What unexpected problems is this going to cause? I don't
know -- but that's the point, if they exist they will be
*unexpected*. Python 3 introduced the literal ... as an
alias for Ellipsis. Nobody realised that this would have
consequences for doctests until somebody reported a
problem, by which time it was too late.

All these downsides make the barrier to entry for new syntax very high.
Python is a 20 year old mature language. Most, perhaps all, of the low-
hanging fruit syntax-wise has been picked. Don't be surprised when
there is opposition to adding new syntax. With very few exceptions, new
syntax has real costs and little or questionable benefit.

Adding syntax is not free, it costs somebody time and effort. Unless the
syntax is a big win, that time and effort is probably better put into
fixing bugs. There is a great shortage of time and manpower for the
Python developers, there is a list of open bugs half a mile long. Every
minute spent adding new syntax for some minor benefit is time that could
be fixing bugs that cause actual problems with real-world code, not just
to satisfy some minor concern about DRY purity.

 

[F.R.]

line = function(x, y, z)

How about:

line = True
while line:
line = function(x, y, z)
do something with(line)

?

Frederic

 

[Chris Angelico]

How about:

line = True
while line:

line = function(x, y, z)
do something with(line)

?

That's going to go through the body of the loop with a false line
before breaking out. In some situations that's not a problem, but it's
distinctly different, so it's not a drop-in replacement for the
others.

ChrisA

 

[Neil Cerutti]

We have a problem, and two solutions. Solution 1 has downside
A, and solution 2 has downside B. If he complains about
downside A, you say, well, use solution 2. If he complains
about downside B, you say, well, use solution 1.

What if he wants to avoid both downsides A and B? What solution
does he use then?

You abandon the while loop and compose a generator.