"as" keyword woes

[Warren DeLano]

A bottom line / pragmatic question... hopefully not a FAQ.

Why was it necessary to make "as" a reserved keyword?

And more to the point, why was it necessary to prevent developers from
being able to refer to attributes named "as"?

For example, this code breaks as of 2.6 / 3.0:

Class C:
Pass

obj = C()

obj.bs = 2 # valid

obj.as = 1 # syntax error

obj.cs = 3 # valid

Just to be clear: I understand why it breaks, since "as" is now a
keyword, I also know that one can use workarounds such as:

obj.__dict__['as'] = 1

to maintain compatibility.

What I want to understand is why this parser change was necessary in
order to enable new 2.6/3.0 features. Was this change potentially
avoidable?

Why can't the parser distinguish between a standalone " as " keyword and
".as" used as an object/attribute reference?

Couldn't we have continued along just fine using a smarter parser
without elevating "as" to reserved status (and thus potentially breaking
a 10+ years of existing code)?

Thank you for enlighting me!

(Unfortunately, our project may now have to maintain a branch at 2.5.x
in order to preserve compatibility with existing third-party scripts &
infrastructure which routinely rely upon "as" as an object method.
Sigh.)

Cheers,
Warren

 

[Martin P. Hellwig]

A bottom line / pragmatic question... hopefully not a FAQ.

Why was it necessary to make "as" a reserved keyword?

<cut>
Because it can be used at the import statement to let the imported thing
be known under another name?
Something like:

 

[Martin P. Hellwig]

<cut>
Because it can be used at the import statement to let the imported thing
be known under another name?
Something like:

Sorry forget about the post, you know full well why not

 

[Steven D'Aprano]

<cut>
Because it can be used at the import statement to let the imported thing
be known under another name?
Something like:

Martin, that doesn't answer the OP's question *at all*. Python 2.5 uses
"as" in that way, and it is not a keyword.
<stdin>:1: Warning: 'as' will become a reserved keyword in Python 2.6
45

I'd guess that the change was to simplify the CPython parser. I have no
idea if it was a tiny change or a significant change, if it made a huge
difference to Python-dev or a little difference. Perhaps someone on the
dev team could comment.

While I feel sympathy for the OP, I do have to ask: he's been using
Python 2.5 for, what, a couple of years now? How many times did he see
the depreciation warning, and almost certainly the pending depreciation
warning before that? Python-dev has been talking about making "as" a
keyword since at least Python 2.3. Why wait until after version 2.6 is
released before saying anything?

 

[Robert Kern]

While I feel sympathy for the OP, I do have to ask: he's been using
Python 2.5 for, what, a couple of years now? How many times did he see
the depreciation warning, and almost certainly the pending depreciation
warning before that? Python-dev has been talking about making "as" a
keyword since at least Python 2.3. Why wait until after version 2.6 is
released before saying anything?

It's entirely possible that he did not see the warning. Python 2.5 has a bug
where the warning gets silenced by an intervening import.

http://bugs.python.org/issue3936

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Mensanator]

Martin, that doesn't answer the OP's question *at all*. Python 2.5 uses
"as" in that way, and it is not a keyword.


<module 'math' from '/usr/lib/python2.5/lib-dynload/mathmodule.so'>>>> as = 45

<stdin>:1: Warning: 'as' will become a reserved keyword in Python 2.6>>> as

<stdin>:1: Warning: 'as' will become a reserved keyword in Python 2.6
45

I'd guess that the change was to simplify the CPython parser. I have no
idea if it was a tiny change or a significant change, if it made a huge
difference to Python-dev or a little difference. Perhaps someone on the
dev team could comment.

While I feel sympathy for the OP, I do have to ask: he's been using
Python 2.5 for, what, a couple of years now? How many times did he see
the depreciation warning, and almost certainly the pending depreciation
warning before that? Python-dev has been talking about making "as" a
keyword since at least Python 2.3. Why wait until after version 2.6 is
released before saying anything?

When I brought this up a short while ago (because sympy crashed in
Python 2.6) someone said that there was a bug in Python 2.5 that
prevented the display of the deprecation message (when "as" appeared
inside imported modules).

So apparently, the sympy developers never saw a deprecation warning
in all the years they were using 2.5. There was, however, no excuse
for not testing it in 2.6.

 

[Martin P. Hellwig]

Martin, that doesn't answer the OP's question *at all*. Python 2.5 uses
"as" in that way, and it is not a keyword.

Yes Steven, I realized after the apple fell on my head that I was
barking at the wrong tree, in a totally different forest, sorry for the
noise.

 

[Matimus]

What I want to understand is why this parser change was necessary in

order to enable new 2.6/3.0 features. Was this change potentially
avoidable?

Does it really matter? The change occurred and it isn't going to go
back. What you should be asking yourself is whether the affect it had
on your code could have been avoided. What could you have done to
prevent your current situation?

Couldn't we have continued along just fine using a smarter parser
without elevating "as" to reserved status (and thus potentially breaking
a 10+ years of existing code)?

Nothing broke your code. It works just fine under the version it was
developed for. Who forced you to upgrade to python2.6?

'as' was already coming close to being a keyword with its use in
import statements. It be came a full fledged keyword to support
context managers:

with open('filename.x', 'w') as fout:
# do stuff

(Unfortunately, our project may now have to maintain a branch at 2.5.x
in order to preserve compatibility with existing third-party scripts &
infrastructure which routinely rely upon "as" as an object method.
Sigh.)

Not necessarily. You can do something like this:

import sys
import warnings

class MyClass(object):
def new_as(self): # call this something appropriate, it is your
new 'as' method
pass # do what 'as' does

if sys.version[:3] <= '2.5':
def _old_as(self):
warnings.warn(
DeprecationWarning(
'This method is deprecated, use `new_as`
instead'))
return self.new_as()
exec 'as = _old_as'

You could even write a metaclass that does this (a little more
elegantly). Your users will get a nice warning telling them not to use
the 'as' method anymore. It will work in both code bases, and you can
schedule to remove it at some later version after customers have had
the opportunity to remove 'as' from their code.

Matt

 

[Steven D'Aprano]

Nothing broke your code. It works just fine under the version it was
developed for. Who forced you to upgrade to python2.6?

Be reasonable. Python 2.5 is not very far away from being put into
"security updates only" mode, and in a year or so it won't even get
security updates. I dare say there are already platforms that use Python
2.6 as standard. Tying your software to an obsolete version of a language
is a good way to force your software into obsolescence.

Not that 2.5 is obsolete *now*. But it will be soon (for some definition
of soon): in no more than a year or three, software that only runs on
Python 2.5 would be like software that only runs on 2.3 now.

 

[Matimus]

Be reasonable. Python 2.5 is not very far away from being put into
"security updates only" mode, and in a year or so it won't even get
security updates. I dare say there are already platforms that use Python
2.6 as standard. Tying your software to an obsolete version of a language
is a good way to force your software into obsolescence.

Not that 2.5 is obsolete *now*. But it will be soon (for some definition
of soon): in no more than a year or three, software that only runs on
Python 2.5 would be like software that only runs on 2.3 now.

Here is the list of downloads from python.org:

# Python 3.0 (December 3, 2008)
# Python 2.6 (October 1, 2008)
# Python 2.5.2 (February 22, 2008)
# Python 2.4.5 (March 11, 2008)
# Python 2.3.7 (March 11, 2008)

Notice that Python 2.3 was given the update treatment in March of this
year. I don't think I was being unreasonable. The point was that there
is that new releases don't _break_ anything. You should always expect
to have to test and update your code when going from Python2,x to
Python2.(x+1).


Matt

 

[John Nagle]

Embrace the pain.

John Nagle

 

[Steven D'Aprano]

The point was that there
is that new releases don't _break_ anything.

But that's clearly not true, because the OP is pointing out that the new
release from 2.5 to 2.6 *does* break his code.

 

[Wolfgang Strobl]

One now has to ask what "break" really meant... For this example,
the change did not break Python SYNTAX -- just a complaint about using
what is now a reserved word as an object name.

Of course it does:

C:\Python26>python
Python 2.6 (r26:66721, Oct 2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win 32
Type "help", "copyright", "credits" or "license" for more information. File "<stdin>", line 1
as=2
^
SyntaxError: invalid syntax
Making a former syntactically valid two letter name a reserved word in
2.6 was a mistake, IMHO. What's next? What about making i,j,k, x and y
reserved words in 2.7? =:-/

 

[Mensanator]

On 05 Dec 2008 05:21:25 GMT, Steven D'Aprano
<[email protected]> declaimed the following in
comp.lang.python:

� �One now has to ask what "break" really meant... For this example,
the change did not break Python SYNTAX -- just a complaint about using
what is now a reserved word as an object name.

Of course it does:

C:\Python26>python
Python 2.6 (r26:66721, Oct �2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win 32
Type "help", "copyright", "credits" or "license" for more information.>>> as=2

� File "<stdin>", line 1
� � as=2
� � �^
SyntaxError: invalid syntax

I disagree. "Broken" is something you can't work
around. In this case, simply saying as_=2 works fine.

A better example of broken was when the gmpy module
wouldn't solve a certain linear congruence problem
because the problem was not invertible. But
mathematically, solving a linear congruence does
NOT depend on the problem being invertible. It was
the ALGORITHM that depended on the problem being
invertible and there was nothing the user could do
to make the algorithm behave properly. The algorithm
had to be altered to fix the special case of a
solvable linear congruence not being invertible.

Making a former syntactically valid �two letter name a reserved word in
2.6 was a mistake, IMHO.

I think you're in the minority there.

What's next? What about making i,j,k, x and y
reserved words in 2.7? =:-/

Yeah, right. That'll happen. You ought to be more
concerned about real problems.

 

[MRAB]

On 05 Dec 2008 05:21:25 GMT, Steven D'Aprano
<[email protected]> declaimed the following in
comp.lang.python:
On Thu, 04 Dec 2008 08:44:19 -0800, Matimus wrote:
The point was that there
is that new releases don't _break_ anything.
But that's clearly not true, because the OP is pointing out that the new
release from 2.5 to 2.6 *does* break his code.
� �One now has to ask what "break" really meant... For this example,
the change did not break Python SYNTAX -- just a complaint about using
what is now a reserved word as an object name.

Of course it does:

C:\Python26>python
Python 2.6 (r26:66721, Oct �2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win 32
Type "help", "copyright", "credits" or "license" for more information.>>> as=2

� File "<stdin>", line 1
� � as=2
� � �^
SyntaxError: invalid syntax

I disagree. "Broken" is something you can't work
around. In this case, simply saying as_=2 works fine.

A better example of broken was when the gmpy module
wouldn't solve a certain linear congruence problem
because the problem was not invertible. But
mathematically, solving a linear congruence does
NOT depend on the problem being invertible. It was
the ALGORITHM that depended on the problem being
invertible and there was nothing the user could do
to make the algorithm behave properly. The algorithm
had to be altered to fix the special case of a
solvable linear congruence not being invertible.

Making a former syntactically valid �two letter name a reserved word in
2.6 was a mistake, IMHO.

I think you're in the minority there.

I think that its special use is clear from the syntax, so IMHO it
could've been left for Python 3, but I'm not going to lose any sleep
over it.

 

[Wolfgang Strobl]

On 05 Dec 2008 05:21:25 GMT, Steven D'Aprano
<[email protected]> declaimed the following in
comp.lang.python:

The point was that there
is that new releases don't _break_ anything.

But that's clearly not true, because the OP is pointing out that the new
release from 2.5 to 2.6 *does* break his code.

? ?One now has to ask what "break" really meant... For this example,
the change did not break Python SYNTAX -- just a complaint about using
what is now a reserved word as an object name.

Of course it does:

C:\Python26>python
Python 2.6 (r26:66721, Oct ?2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win 32
Type "help", "copyright", "credits" or "license" for more information.>>> as=2

? File "<stdin>", line 1
? ? as=2
? ? ?^
SyntaxError: invalid syntax

I disagree. "Broken" is something you can't work
around.

Well, that language change _did_ break the OPs code, and he already
explained why it isn't as simple to work around as you obviously happen
to believe. Calling it "not a SYNTAX error" is not only mistaken, it's
weaseling around the problem too.

In this case, simply saying as_=2 works fine.

Certainly not, for example when that name is already used outside your
control. In addition, you'll have to find all places, where the now
syntactically illegal name is or might be used. This can get arbitrarily
hard in generated or foreign code.

A better example of broken was when the gmpy module
wouldn't solve a certain linear congruence problem
because the problem was not invertible.

What does that have to to with anything? We're disussing a language
change on the syntax level which breaks existing legal code, not how a
numerical library behaves for borderline cases.

[...]

I think you're in the minority there.

Perhaps. Does that matter?

Yeah, right. That'll happen.

Certainly not. On the other hand, I was quite flabbergasted by the
change in question, likewise. Wasn't that something reserved for 3.0:
breaking code mercilessly? What is 3.0 good for, when we have to bear
such breakage in a minor release, already?

You ought to be more
concerned about real problems.

It is a real problem for the OP.

 

[Terry Reedy]

In my opinion, this thread is a crock of balony.

Python *occasionally* adds keywords after giving a warning or requiring
a future import in previous versions.

In 2.2, one had to 'from __future__ import generators' to make a
generator because doing so required the new 'yield' keyword.
In 2.3, yield became a keyword without the import.

In 2.5, one had to 'from __future__ import with_statement' to make a
'with' statement. In 2.6, with because a keyword without the import.
And no one seems to have noticed. No '"with" keyword woes.

In 2.6, 'as' also because a full-time keyword after several releases of
being an anomalous part-time contextual keyword.

tjr

 

[Mensanator]

Mensanator <[email protected]>:

Dennis Lee Bieber <[email protected]>:
On 05 Dec 2008 05:21:25 GMT, Steven D'Aprano
<[email protected]> declaimed the following in
comp.lang.python:
On Thu, 04 Dec 2008 08:44:19 -0800, Matimus wrote:
The point was that there
is that new releases don't _break_ anything.
But that's clearly not true, because the OP is pointing out that the new
release from 2.5 to 2.6 *does* break his code.
? ?One now has to ask what "break" really meant... For this example,
the change did not break Python SYNTAX -- just a complaint about using
what is now a reserved word as an object name.
Of course it does:
C:\Python26>python
Python 2.6 (r26:66721, Oct ?2 2008, 11:35:03) [MSC v.1500 32 bit
(Intel)] on win 32
Type "help", "copyright", "credits" or "license" for more information.>>> as=2
? File "<stdin>", line 1
? ? as=2
? ? ?^
SyntaxError: invalid syntax

I disagree. "Broken" is something you can't work
around.

Well, that language change _did_ break the OPs code, and he already
explained why it isn't as simple to work around as you obviously happen
to believe.

It was extremely simple for me to fix the sympy
module where I noticed it. I'm not saying it
wasn't a problem, I'm saying it wasn't BROKEN.
And no, I couldn't use sympy after the fix
because apparently there is some change the way
2.6 hashes which truly breaks the sympy code, so
no using sympy with 2.6 until a corrected version
is released. And the real problem is that the
latest version was released without ever testing it
under 2.6. Hopefully, that won't happen again.

� Calling it "not a SYNTAX error" is not only mistaken, it's
weaseling around the problem too.

It's not weaseling around the problem, it's rating
the seriousness of the problem. Right now if I need
sympy, I HAVE to use 2.5, if I need itertools.permutations
I HAVE to use 2.6. But I canot use both in the same
program which is a tad more serious than having to
rename a variable I had business using in the first place.

Certainly not, for example when that name is already used outside your
control. In addition, you'll have to find all places, where the now
syntactically illegal name is or might be used. This can get arbitrarily
hard in generated �or foreign code.

The difficulty of fixing it isn't the issue, it's
whether it can be fixed at all.

What does that have to to with anything?

It's an example of what it means to be BROKEN,
and yes, nothing at all to do with SYNTAX.

We're disussing a language
change on the syntax level which breaks existing legal code, not how a
numerical library behaves for borderline cases.

And yet, you don't consider this a case of "weasel words".

Strange.

[...]

I think you're in the minority there.

Perhaps. Does that matter?

Yeah, if you think it will be resolved in your favor.

Certainly not. �On the other hand, I was quite flabbergasted by the
change in question, likewise. Wasn't that something �reserved for 3.0:
breaking code mercilessly? �

Mercilessly? If there were no deprecation warnings,
you might have a case there. You act as if the failure
to display such warnings was deliberate, but that was due
to a BUG, not merciless behaviour on the part of the
developers.

What is 3.0 good for, when we have to bear
such breakage in a minor release, already?

You're making a mountain out of a molehill.

It is a real problem for the OP.

Who gave him permission to use 2.6? Is 2.5 suddenly
unavailable? Assholes like Microsoft pull stunts
like that, but Python.org doesn't.

If it's his code, he can easily fix it.

If it's someone else's code, he should be using
a version guaranteed to work in 2.6.

 

[Steven D'Aprano]

It was extremely simple for me to fix the sympy module where I noticed
it. I'm not saying it wasn't a problem, I'm saying it wasn't BROKEN.

If it wasn't broken, why did you need to fix it?

"Broken" means "not working", not "unfixable".

 

[Mensanator]

If it wasn't broken, why did you need to fix it?

If my tire is flat, I have to fix it. But it may just need air,
in which case it's not broken.

"Broken" means "not working", not "unfixable".

So, you're saying that Python is broken and will remain so forever,
since "as" will remain a keyword?

Are you advocating that we all switch to Ruby?