Confused about pep 318

[Edward K. Ream]

Hello all,

First of all, my present state of mind re pep 318 is one of sheepish
confusion. I suspect pep 318 will not affect Leo significantly, but I am
most surprised that an apparently controversial feature is being added
without a notice, say, in comp.lang.python.announce. I say sheepish,
because everyone knows that one should make sure new proposals don't bite
you. Still, I would have thought that there were other ways of keeping
abreast of potentially major changes to Python besides reading py-dev...

The following are some comments to recent posts on py-dev. I hope to convey
by these remarks my sense of bewilderment.

There is little point in going over all the same arguments again and

again. For this alpha release, no amount of arguing will change what has
been committed. You really have to organize a public outcry if you want the
syntax changed before 2.4. [Martin v. Löwis]

I'd like to defer the question of a "public outcry" until later, hopefully
much later.

But I have no idea what the proposed syntax is(!!) In particular, there is
no mention of '@' directly on the page
http://www.python.org/peps/pep-0318.html

Is there a summary of recent work? An announcement? Some indication of
consensus?
unadorned with decorators.

Right. Outside the test suite, there are very few uses of classmethod,

staticmethod and property in the standard library. GvR

Hmm... I see nowhere in pep 318 any mention that the pep will affect the
following section of the Reference manual:
http://docs.python.org/ref/delimiters.html

The following printing ASCII characters are not used in Python. Their
occurrence outside string literals and comments is an unconditional error:
@ $ ?
[end quote]

I would regard any change to this paragraph as a major change to Python, in
spite of its innocent appearance. Indeed, this paragraph is an excellent
"line in the sand" for other tools.

To repeat, I suspect adding @ would not affect Leo significantly in any
direct way, and a workaround would probably exist even if it did. However,
this proposal probably would affect the noweb markup language on which Leo
is (loosely) based. See http://www.eecs.harvard.edu/~nr/noweb/

BTW, a long while back I suggested that there was no reason for the parser
to need colons after "if" statements, etc. The response was: true, but
allowing colons to be elided would impact existing tools. So I gave up. Am
I the first to suggest that allowing '@' characters in valid Python programs
outside string literals and comments might negatively impact existing tools?

Sigh. This discussion is going around in pointless circles; we're

*months* past that point. You're wasting your time (and mine). [GvR]

I confess that I don't understand this remark at all. How has it happened
that so many people are confused about this proposal? And if everything is
so clear, why isn't the clarity reflected in pep 318 itself?

http://www.python.org/peps/pep-0318.html talks about

- Background
- Design goals
- Proposed syntax (no mention of @!)
- Alternative proposals
etc.

Excuse me, but if everything is now set in stone, then this is an
extraordinarily misleading document.

Sorry. I think you'd be wise to consider the months without consensus as

evidence of a problem. [Jim Fulton]

This is my present opinion as well.

Summary

I wish to disavow any knowledge, or any interest in, the wider issues of
whether pep 318 is a good idea or not. My concern is exclusively with the
following two issues:

1. IMO, allowing '@' in valid Python programs (outside string literals or
comments) is a major change to the language that has significant potential
to disrupt already existing tools. At the very least, some discussion of
http://docs.python.org/ref/delimiters.html should be in pep 318.

2. pep 318 apparently does not reflect the present state of affairs, and
afaik there has been no announcement of proposed changes on
comp.lang.python.announce. This is disturbing. If anyone could clarify the
present status of pep 318 I would be most grateful. I would prefer not to
be told to read all the py-dev archives. If everything is now clear, where
can I find out about it?

Edward

 

[Skip Montanaro]

Edward> But I have no idea what the proposed syntax is(!!) In
Edward> particular, there is no mention of '@' directly on the page
Edward> http://www.python.org/peps/pep-0318.html

Anthony Baxter said yesterday on python-dev he was going to attend to that
shortly. As in many other software efforts, the code has moved along a bit
faster than the documentation (after all, what's more fun to do?). I
believe the only significant differences from a pure functional
documentation standpoint to apply to the PEP are the syntax and the issue of
decorated classes (I don't think support for decorated classes will make
it). By way of parallel examples here's how the new syntax corresponds to
the syntax I used in the last revision of the PEP:

#1 from the PEP: Given this function:

def onexit(f):
import atexit
atexit.register(f)
return f

the new syntax will be:

@onexit
def func():
...

#4 from the PEP: Given these functions:

def accepts(*types):
def check_accepts(f):
assert len(types) == f.func_code.co_argcount
def new_f(*args, **kwds):
for (a, t) in zip(args, types):
assert isinstance(a, t), \
"arg %r does not match %s" % (a,t)
return f(*args, **kwds)
return new_f
return check_accepts

def returns(rtype):
def check_returns(f):
def new_f(*args, **kwds):
result = f(*args, **kwds)
assert isinstance(result, rtype), \
"return value %r does not match %s" % (result,rtype)
return result
return new_f
return check_returns

the new syntax will be:

@accepts(int, (int,float))
@returns((int,float))
def func(arg1, arg2):
return arg1 * arg2

I believe the order of application of the above decorators would be like so:

func = accepts(int, (int, float))(returns((int, float))(func)

Edward> I confess that I don't understand this remark at all. How has
Edward> it happened that so many people are confused about this
Edward> proposal? And if everything is so clear, why isn't the clarity
Edward> reflected in pep 318 itself?

There was a lot of discussion on python-dev, but none very recently (last
month or so). Guido indicated there that he brought up the topic at
EuroPython in his keynote talk and entertained discussion from the floor.
Based upon that discussion he decided to go with the @-decorator syntax.
Since EuroPython most/all the discussion went on in private email or on
irc. I think it would be nice if this conversation was summarized in the
PEP, but that will have to come from one of the participants.

Skip

 

[Edward K. Ream]

As in many other software efforts, the code has moved along a bit

faster than the documentation (after all, what's more fun to do?). ....
Guido indicated there that he brought up the topic at
EuroPython in his keynote talk and entertained discussion from the floor.
Based upon that discussion he decided to go with the @-decorator syntax.

Well, this isn't good. We are talking about a basic design document here.
I feel like I have been kept in the dark; my previous relative silence is
hardly evidence of my agreement. How is it possible to pretend that people
agree with a proposal that DOES NOT EXIST? I am -10 on pep 318 now: this
process amounts to a rigged election. I might become neutral in the future
once the hood is removed from over the candidate's head

I believe the only significant differences from a pure functional
documentation standpoint to apply to the PEP are the syntax

Why is something "spectacular" like the '@' sign is needed? Presumably
there is some problem with the compiler? Has anyone suggested something
like:

from __future__ import annotation as annotate

annotate.accepts(int, (int,float))

instead of:

@accepts(int, (int,float))

Clearly, with enough work the Python compiler could recognize this. Yes,
it's a special case, but so what? This would avoid most problems with
reserved words or keywords. And it would be in the spirit of letting
modules encapsulate most major features.

Edward

P.S. The more I think of this proposal, the more unhappy I become. In fact,
this proposal may have a most unfortunate effect on Leo's future, for two
reasons:

1. Although Leo handles constructs like @accepts, it does so by generating
lines like this:

#@verbatim
@accepts(int, (int,float))

So _Leo_ has no problem, but Leo's users will likely complain that more
cruft has been added to their files.

2. Leo presently supports the following directives:

@ (followed by one or more whitespace characters)
@all, @asis,
@c, @code, @color, @comment,
@delims, @doc,
@encoding, @end_raw,
@file, @first,
@header,
@ignore,
@killcolor,
@language, @last, @lineending,
@nocolor, @noheader, @noref, @nosent, @nowrap,
@others,
@pagewidth, @path,
@quiet,
@raw,
@root, @root-code, @root-doc,
@silent,
@tabwidth, @terse, @thin,
@unit,
@verbose,
@wrap

Worse, Leo allows plugins to define their own directives. There is already
a plugin that defines @wiki. Furthermore, @run, @test and @suite are also
used in special contexts.

What happens when another Python @x construct conflicts with one of these
directives or some other directive? I'm starting to wonder whether Python
is a snake or a gorilla

Yes, Python does have the right to use '@'. But I would hope that GvR would
choose not to do so. I was always under the impression that the clear
statement in the Reference Manual that at signs are invalid everywhere
(except in comments and strings) was a clear signal of an intention to keep
those symbols for other valid purposes. I am extremely unhappy that this
may not be so.

P.P.S. I did respond earlier to the requested survey, saying in brief that
I had no problems with 318 provided that '@' signs did not become a part of
the syntax. I have received no reply from anyone. There is no way somebody
could construe my previous statements as approval.

EKR

 

[Michele Simionato]

There was a lot of discussion on python-dev, but none very recently (last
month or so). Guido indicated there that he brought up the topic at
EuroPython in his keynote talk and entertained discussion from the floor.
Based upon that discussion he decided to go with the @-decorator syntax.
Since EuroPython most/all the discussion went on in private email or on
irc. I think it would be nice if this conversation was summarized in the
PEP, but that will have to come from one of the participants.

Skip

I think the way the decision came up was absolutely unfair to the PEP
proponents and to the Python community at large. This disturb me more
than the syntax itself (which I don't like, but this not the point).
I agree that Guido is the Dictator For Life, but he is supposed to
be Benevolent!

I-will-never-submit-a-PEP-ly,

Michele Simionato

 

[Steven Bethard]

the new syntax will be:

@onexit
def func():
...

Wow, this one really feels like it was slipped in while no one was
looking. While I don't go to Python conferences, I do occasionally
check python-dev, and I totally missed whatever thread introduced the
@ syntax. Scary...

Not a big fan of this syntax -- no intuitive reason why @ would mean
decorator -- but I presume this has already been complained about...
Could someone direct me to the discussions about this? I'd like to
see why it was the eventual favored syntax...

Steve

 

[Christopher T King]

Wow, this one really feels like it was slipped in while no one was
looking.

That's because it was.

While I don't go to Python conferences, I do occasionally
check python-dev, and I totally missed whatever thread introduced the
@ syntax. Scary...

This is the post that started it all:
http://mail.python.org/pipermail/python-dev/2004-August/046599.html

Quote:
The @decorator patch has been landed on the trunk, after
getting the go-ahead from the BDFL. I'll update PEP-0318
with the final syntax in the next day or so.

That's the first most people had heard of it. Guido had given Anthony
Baxter the go-ahead in a private e-mail. There had previously been no
public discussions about it.

Not a big fan of this syntax -- no intuitive reason why @ would mean
decorator -- but I presume this has already been complained about...

It has, but the more people complaining, the better, if you're against it.

 

[Anthony Baxter]

This is the post that started it all:
http://mail.python.org/pipermail/python-dev/2004-August/046599.html

Quote:
The @decorator patch has been landed on the trunk, after
getting the go-ahead from the BDFL. I'll update PEP-0318
with the final syntax in the next day or so.

That's the first most people had heard of it. Guido had given Anthony
Baxter the go-ahead in a private e-mail. There had previously been no
public discussions about it.

Nonsense.

http://mail.python.org/pipermail/python-dev/2004-June/thread.html#45516
was over a month earlier. In addition, the particular SF item for this
http://www.python.org/sf/979728 wasn't exactly hidden.

Anthony

 

[Christopher T King]

http://mail.python.org/pipermail/python-dev/2004-June/thread.html#45516
was over a month earlier.

That thread reaches no consensus, other than, in his sole post in the
thread, Guido stating "I would love to see an implementation of this
idea." Nothing about "this is going to be in 2.4a2, anybody vehemently
opposed to it?" Like I said, the go-ahead was given privately.

In addition, the particular SF item for this
http://www.python.org/sf/979728 wasn't exactly hidden.

Indeed; I'm surprised it isn't titled "Beware of the Leopard".

 

[Anthony Baxter]

http://mail.python.org/pipermail/python-dev/2004-June/thread.html#45516

That thread reaches no consensus, other than, in his sole post in the
thread, Guido stating "I would love to see an implementation of this
idea." Nothing about "this is going to be in 2.4a2, anybody vehemently
opposed to it?" Like I said, the go-ahead was given privately.

None of the decorator discussions _ever_ reached a conclusion. As mwh put it,
"bike-shed-the-ultimate". At the end of the day, Guido's the one who makes
these decisions. His decision was that the @ syntax should go into 2.4a2.

As far as "private go-ahead", I've lost count of the number of people who
seem to think that this is somehow an issue. Would the bitching have been
seriously that much better if, half an hour before I committed the patch,
Guido had posted a note saying pretty much what I forwarded on from him?
If so, why? Guido followed up to my note pretty much straight away confirming
it. Or would it have someone been better if Guido had done the CVS commit?

Yes, it would have been nice if PEP-0318 was updated in advance of this. I
made an effort, but have run out of resources to work on it. If no-one else
gets to it before me, I will work on it again next week. But, suprise, Python's
a volunteer effort. No-one's paying me to do this, or Guido, or anyone else.
If someone wants to offer to pay me, or someone else, to actually do all this,
*great*.

 

[Peter Hansen]

But, suprise, Python's
a volunteer effort. No-one's paying me to do this, or Guido, or anyone else.
If someone wants to offer to pay me, or someone else, to actually do all this,
*great*.

I'll make another donation to PSF if the final decorator syntax
does not Perlishly use arbitrary punctuation as the @ syntax does.

-Peter

 

[Edward K. Ream]

There had previously been no public discussions about it.

Nonsense.

I could not disagree more. Pretending that discussions on py-dev and
SourceForge count as truly public discussions of pep 318 is most unwise.
pep 318 does not discuss '@' at all. People like me, with a strong interest
in how Python uses '@', would not naturally have known about the proposed
new syntax until the stuff hit the fan.

Indeed, pep 318 is grossly misleading; reading it one gets the distinct
impression that the design is far from complete. It is my strong opinion
that _no_ public discussion of this new code has taken place, and none _can_
take place until we see what it is exactly that is being proposed. This is
an issue of basic fairness and openness. I have complained loudly to the
DFL. We shall see...

Edward

 

[Anthony Baxter]

I'll make another donation to PSF if the final decorator syntax
does not Perlishly use arbitrary punctuation as the @ syntax does.

FFS. What exactly is "Perlish" about @? It's an unused symbol. That's all.

 

[Anthony Baxter]

I'll make another donation to PSF if the final decorator syntax
does not Perlishly use arbitrary punctuation as the @ syntax does.

FFS. What exactly is "Perlish" about @? It's an unused symbol. That's all.

 

[Anthony Baxter]

I'll make another donation to PSF if the final decorator syntax
does not Perlishly use arbitrary punctuation as the @ syntax does.

FFS. What exactly is "Perlish" about @? It's an unused symbol. That's all.

 

[Anthony Baxter]

I'll make another donation to PSF if the final decorator syntax
does not Perlishly use arbitrary punctuation as the @ syntax does.

FFS. What exactly is "Perlish" about @? It's an unused symbol. That's all.
It's hardly arbitrary - Java, for instance, already uses @ for the same thing.

 

[Anthony Baxter]

I'll make another donation to PSF if the final decorator syntax
does not Perlishly use arbitrary punctuation as the @ syntax does.

FFS. What exactly is "Perlish" about @? It's an unused symbol. That's all.
It's hardly arbitrary - Java, for instance, already uses @ for the same thing.

 

[Christopher T King]

None of the decorator discussions _ever_ reached a conclusion. As mwh put it,
"bike-shed-the-ultimate". At the end of the day, Guido's the one who makes
these decisions. His decision was that the @ syntax should go into 2.4a2.

I can't speak for everybody, but I would've liked some closure on the
issue before it was injected into 2.4a2. From what I can tell, in the
public's mind, the syntax of decorators was still an open issue, and if
anything was going to make it in, it was going to be the "def foo()
[dec]:" syntax that was currently in favour. No-one was prepared for the
introduction of the @ syntax.

 

[Edward K. Ream]

None of the decorator discussions _ever_ reached a conclusion.

I find this oddly reassuring, in a sorta black-humorous way. At least we
won't have to undo the previous agreement

Would the bitching have been
seriously that much better if, half an hour before I committed the patch,
Guido had posted a note saying pretty much what I forwarded on from him?

The issue is not about rights to commit code, the issues are whether

a) the question was ever publicly discussed at all and
b) whether any consensus was reached.

Apparently, the answer to both questions is No.

Yes, it would have been nice if PEP-0318 was updated in advance of this.

Not nice. Essential. It is fundamentally unfair to pretend to have
discussed a proposal publicly that has always been grossly misrepresented in
the one place one would naturally look to find information about it.

Edward

 

[Anthony Baxter]

I could not disagree more. Pretending that discussions on py-dev and
SourceForge count as truly public discussions of pep 318 is most unwise.
pep 318 does not discuss '@' at all. People like me, with a strong interest
in how Python uses '@', would not naturally have known about the proposed
new syntax until the stuff hit the fan.

Sorry, but if discussions on python-dev are not considered "public", what is?
It's an open list, with publically available archives. Expecting the python-dev
team to read all of comp.lang.python is foolish - most of us have very little
spare time as it is. For instance, I usually only follow the list immediately
after a release. I have no time to wade through the hundreds of posts a day
on a regular basis. Indeed, I'm currently up way way too late at night to
participate in this discussion, because the chances are I'll have no time
tomorrow to do so.

Indeed, pep 318 is grossly misleading; reading it one gets the distinct
impression that the design is far from complete. It is my strong opinion
that _no_ public discussion of this new code has taken place, and none _can_
take place until we see what it is exactly that is being proposed.

See above. Exactly how is a public mailing list with an open subscription
policy, no limitations on who can post, and with web searchable archives,
not public. And if you think there hasn't been discussion on this matter,
you obviously haven't even bothered to look at the archives. There has
been a overwhelming amount of discussion on this.

This is
an issue of basic fairness and openness. I have complained loudly to the
DFL. We shall see...

Fariness and openness? This isn't a debating club! As far as fairness - well,
after all the discussions were had, Guido made a judgment call. That's what
he does. Python, thank the gods, is not designed by some system of voting
and the like.

To summarise:
As I've stated already, I'm aware that PEP 318 needs an update. If no-one
else gets it done before next week, I will have time to work on it again. If one
of the people complaining about it wants to sit down and trawl the (literally)
hundreds and hundreds of python-dev messages, spread over the last 2+
years, and extract the relevant posts, that would be excellent! If you're only
going to sit and bitch about it, well, I can spend my time better trawling the
archives.

PEP 318 _will_ be updated and complete before 2.4 final is done - it's on my
list as a blocker for the final. Ideally (and I plan for this) it will
be done before a3.

Anthony

 

[Christopher T King]

To summarise: As I've stated already, I'm aware that PEP 318 needs an
update. If no-one else gets it done before next week, I will have time
to work on it again. If one of the people complaining about it wants to
sit down and trawl the (literally) hundreds and hundreds of python-dev
messages, spread over the last 2+ years, and extract the relevant posts,
that would be excellent! If you're only going to sit and bitch about it,
well, I can spend my time better trawling the archives.

I'd be willing to do this this weekend (esp. being Python Bug Day),
assuming Skip doesn't get to it first.

On a related note, I suggest anyone interested read this thread on
python-dev, "Call for defense of @decorators":
http://mail.python.org/pipermail/python-dev/2004-August/046878.html

In it, Guido makes some very interesting remarks, most notably:

I also want to find out about superior syntax proposals (from __future__
import decorators might be acceptable).

and

... I suggest that the proponents of syntax alternatives will have
to agree amongst themselves on a single alternative that they can
present to me.