A decorator syntax not yet mentioned (I think!)

Discussion in 'Python' started by John Marshall, Aug 11, 2004.

  1. How about the following, which I am almost positive
    has not been suggested:
    -----
    class Klass:
    def __init__(self, name):
    self.name = name

    deco meth0:
    staticmethod
    def meth0(x):
    return x

    deco meth1:
    classmethod
    def meth1(cls):
    return cls

    deco sayhello:
    funcattrs(name='GvR', language='python')
    log(file='func.log')
    def sayhello(self):
    print 'hello python world'

    -----
    1) The decorators clearly apply to a specific method/function,
    therefore there is no need to do any stack pushing in memory
    until the method/function definition is done.
    2) The decorators are "outside" of the method/function they
    decorate:
    a) which will please those who want the outside location
    b) will not be folded within the function
    c) folding on the decorators can be done so that the
    def is not obfuscated
    d) can be located anywhere in the code--but most likely
    before the "def ...()"
    3) The sequence in which the decorators are applied is just
    like code would be--this is certainly intuitive given
    that we are writing code.

    This approach could also be applied to classes in case
    decorators should ever be extended to them:
    -----
    deco Klass:
    doc("This is a class to ...")
    class Klass:
    :
    :
    -----

    Any comments?

    John
     
    John Marshall, Aug 11, 2004
    #1
    1. Advertising

  2. John Marshall

    Peter Hansen Guest

    John Marshall wrote:
    > How about the following, which I am almost positive
    > has not been suggested:
    > -----
    > class Klass:
    > def __init__(self, name):
    > self.name = name
    >
    > deco meth0:
    > staticmethod
    > def meth0(x):
    > return x
    >
    > -----
    > 1) The decorators clearly apply to a specific method/function,
    > therefore there is no need to do any stack pushing in memory
    > until the method/function definition is done.


    Problems with that: duplication (since you now have the name
    of the function in two places) and readability (it's harder to
    see the "def" because the name you picked is too close to it
    in length and appearance).

    On the other hand, there's no need to have the function name
    there (was anyone complaining about "stack pushing"?), so
    maybe just using "decorate" would be okay, or a different
    keyword:

    class Klass:
    # init set

    decorate:
    staticmethod
    def meth0(x):
    return x

    > 2) The decorators are "outside" of the method/function they
    > decorate:
    > a) which will please those who want the outside location
    > b) will not be folded within the function
    > c) folding on the decorators can be done so that the
    > def is not obfuscated
    > d) can be located anywhere in the code--but most likely
    > before the "def ...()"
    > 3) The sequence in which the decorators are applied is just
    > like code would be--this is certainly intuitive given
    > that we are writing code.


    > Any comments?


    Has potential, I think, with some changes.

    I like the following about it:

    1) keyword instead of punctuation
    2) indentation consistent with rest of Python
    3) similarity to try/finally or while/else and friends where
    the two clauses are coupled

    I wouldn't mind seeing exactly the same idea allowed for
    decorators *inside* the function as well, so that we can
    experiment for a while and see which, if any, is preferred,
    but I doubt that would happen.

    -Peter
     
    Peter Hansen, Aug 11, 2004
    #2
    1. Advertising

  3. Peter Hansen wrote:
    > class Klass:
    > # init set
    >
    > decorate:
    > staticmethod
    > def meth0(x):
    > return x


    If it counts for anything, +1.

    This strikes me personally as a very nice alternative to the
    current suggestion.

    The idea of paired keywords is sufficiently common in python
    to strike me as another positive element.

    Using the examples on http://www.python.org/moin/PythonDecorators

    You gain something like:
    class C(object):

    decorate:
    staticmethod,
    funcattrs(grammar="'@' dotted_name [ '(' [arglist] ')' ]",
    status="experimental", author="BDFL")
    def longMethodNameForEffect(longArgumentOne=None,
    longArgumentTwo=42):
    """This method blah, blah.

    It supports the following arguments:
    - longArgumentOne -- a string giving ...
    - longArgumentTwo -- a number giving ...

    blah, blah.

    """
    raise NotYetImplemented

    And:

    decorate:
    accepts(int,int)
    returns(float)
    def bar(low,high):
    pass

    Which strikes me as pretty good - the fact the method is a static method
    is still in your face. The decorate keyword suggests some voodoo
    happening you might want to be wary of.

    The only downside I can see is that it might not be 100% obvious that it
    relates to the following function to newcomers to python.

    The other really nice thing about it is it directly suggests that the
    following functions will be applied to the function.

    Taking a subset of a simple PLY type grammar from, you gain something
    like:

    class Calc(Parser):
    decorate:
    tokenrule(r'\d+')
    def t_NUMBER(self, t):
    t.value = int(t.value)
    return t

    decorate:
    tokenrule(r'\n+')
    def t_newline(self, t):
    t.lineno += t.value.count("\n")

    decorate:
    grammarrule('statement : NAME EQUALS expression')
    versioninfo("Added in 2.2")
    typeinfo(int)
    def p_statement_assign(self, p):
    names[p[1]] = p[3]

    decorate:
    grammarrule('statement : expression')
    versioninfo("Added in 2.4")
    deprecated
    typeinfo(None)
    def p_statement_expr(self, p):
    print p[1]

    decorate:
    grammarrule("""
    expression : expression PLUS expression
    | expression MINUS expression
    | expression TIMES expression
    | expression DIVIDE expression
    | expression EXP expression
    """)
    versioninfo("Added in 2.4")
    typeinfo(int)
    def p_expression_binop(self, p):
    p[0] = func[p[2]](p[1],p[3])

    decorate:
    grammarrule('expression : MINUS expression %prec UMINUS')
    versioninfo("Added in 1.5")
    typeinfo(int)
    def p_expression_uminus(self, p):
    p[0] = -p[2]

    decorate:
    grammarrule('expression : LPAREN expression RPAREN')
    versioninfo("Added in 1.1")
    typeinfo(int)
    def p_expression_group(self, p):
    p[0] = p[2]

    In a syntax highlighted editor that also looks *alot* cleaner than
    beforehand. IMO, it also looks slightly cleaner than the @version - for
    one very, very simple reason - if you have a very big decorator block,
    for whatever reason, the "def" stands out clearly so you can skim where
    the function starts - even without syntax highlighting.

    My initial reaction to @decorators was "ugh!". However I was coming to
    accept it as inevitable after reading the various arguments for/against
    various syntaxes. *This* syntax strikes me personally as having
    distinct advantages - largely for clarity of code, and solves the
    "where *DOES* the function start?" question with large decorator
    blocks.

    My tuppenceworth... (I really like this version)


    Michael.
    --

    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.
     
    Michael Sparks, Aug 11, 2004
    #3
  4. John Marshall

    Paul McGuire Guest

    "Peter Hansen" <> wrote in message
    news:p...
    >
    > class Klass:
    > # init set
    >
    > decorate:
    > staticmethod
    > def meth0(x):
    > return x
    >

    BEAUTIFUL!!! Looks very Pythonic.

    This is the first prefix syntax proposal I've seen that looks to address all
    of the technical and esthetic objections.

    And NO magic punctuation - hooray!

    -- Paul
     
    Paul McGuire, Aug 11, 2004
    #4
  5. John Marshall

    Paul McGuire Guest

    Paul McGuire, Aug 11, 2004
    #5
  6. John Marshall

    Jeff Shannon Guest

    Michael Sparks wrote:

    >Peter Hansen wrote:
    >
    >
    >>class Klass:
    >> # init set
    >>
    >> decorate:
    >> staticmethod
    >> def meth0(x):
    >> return x
    >>
    >>

    >
    >[snip]
    >My initial reaction to @decorators was "ugh!". However I was coming to
    >accept it as inevitable after reading the various arguments for/against
    >various syntaxes. *This* syntax strikes me personally as having
    >distinct advantages - largely for clarity of code, and solves the
    >"where *DOES* the function start?" question with large decorator
    >blocks.
    >


    I agree. This seems to fit the absolute requirements I've heard passed
    down from GvR, and also avoids all of the things that really made my
    teeth grate about the @decorator syntax. It's the first prefix syntax
    I've seen that actually *looks* like Python to my eyes.

    There's still room for discussion about the specific keyword (I'd be
    happy enough with "decorate", and not so fond of "using" or "with", but
    a case could easily be made for something else...) and about ordering --
    presenting the decorators as a suite

    decorate:
    d1
    d2
    def method( ... )
    pass

    suggests to me an equivalence to

    method = d2(d1(method))

    which is the reverse of the ordering of @decorators. How significant
    the difference is, I can't say...

    Jeff Shannon
    Technician/Programmer
    Credit International
     
    Jeff Shannon, Aug 11, 2004
    #6
  7. Michael Sparks <> wrote in message news:<cfdh4q$kpm$>...
    > The only downside I can see is that it might not be 100% obvious that it
    > relates to the following function to newcomers to python.


    This is a downside, but as far as I can tell, it's a downside of all
    proposals that place the decorator declaration before the function.
    Since it looks like we're stuck with pre-def decorators, I wouldn't
    really worry about this issue too much.

    > In a syntax highlighted editor that also looks *alot* cleaner than
    > beforehand. IMO, it also looks slightly cleaner than the @version - for
    > one very, very simple reason - if you have a very big decorator block,
    > for whatever reason, the "def" stands out clearly so you can skim where
    > the function starts - even without syntax highlighting.


    This is a great point. I definitely agree -- it's much easier to find
    the def than even the @ solution. Compare:

    @grammarrule('statement : expression')
    @versioninfo("Added in 2.4")
    @deprecated
    @typeinfo(None)
    def p_statement_expr(self, p):
    print p[1]

    vs.

    decorate:
    grammarrule('statement : expression')
    versioninfo("Added in 2.4")
    deprecated
    typeinfo(None)
    def p_statement_expr(self, p):
    print p[1]

    Of course, you could allow the same thing with '@':

    @:
    grammarrule('statement : expression')
    versioninfo("Added in 2.4")
    deprecated
    typeinfo(None)
    def p_statement_expr(self, p):
    print p[1]

    But this looks really cryptic -- unlike 'decorate', '@' doesn't really
    read as anything. You probably know that the '@' "applies" to all the
    lines below it, but unless you already know what '@' means, you
    probably have no idea how it "applies". The keyword version seems
    much clearer on this point.


    I know there were complaints before about this sort of indentation,
    but I couldn't find them in python-dev and the comments in the wiki
    don't discuss this in any detail. Can anyone tell me why this
    indentation syntax was dispreferred? Specifically, I'm interested in
    why this is so drastically different from the other paired blocks:
    if/elif/else, try/except, try/finally, etc. Also, there's a comment
    in the wiki that says that there are technical problems with the
    grammar if a block *starts* with an optional part. Does this not
    apply to the @ case (which also starts a block with an optional part)?

    Thanks,

    Steve
     
    Steven Bethard, Aug 11, 2004
    #7
  8. John Marshall

    Carl Banks Guest

    (Steven Bethard) wrote in message news:<>...

    > I know there were complaints before about this sort of indentation,
    > but I couldn't find them in python-dev and the comments in the wiki
    > don't discuss this in any detail. Can anyone tell me why this
    > indentation syntax was dispreferred? Specifically, I'm interested in
    > why this is so drastically different from the other paired blocks:
    > if/elif/else, try/except, try/finally, etc.


    One thing that's different is that, in all those cases, the second
    block has a keyword that can't appear alone. You can't have an except
    without a try, or an else without an if, but you could have a def
    without a decorate.

    For that reason, I suggest it would be a bit more congruent (not
    necessarily enough to justify another new keyword) with the rest of
    Python to also use a keyword different from def, say ddef.

    decorate:
    ...
    ddef function(args):
    ...

    Either is definitely very passable if we must go the before-the-def
    route.


    > Also, there's a comment
    > in the wiki that says that there are technical problems with the
    > grammar if a block *starts* with an optional part. Does this not
    > apply to the @ case (which also starts a block with an optional part)?


    It doesn't seem like there would be a problem defining it in, for
    example, yacc. def and decorate...def could be two different
    statements.
     
    Carl Banks, Aug 12, 2004
    #8
  9. John Marshall

    Peter Hansen Guest

    Carl Banks wrote:

    > (Steven Bethard) wrote in message news:<>...
    >
    >>I know there were complaints before about this sort of indentation,
    >>but I couldn't find them in python-dev and the comments in the wiki
    >>don't discuss this in any detail. Can anyone tell me why this
    >>indentation syntax was dispreferred? Specifically, I'm interested in
    >>why this is so drastically different from the other paired blocks:
    >>if/elif/else, try/except, try/finally, etc.

    >
    > One thing that's different is that, in all those cases, the second
    > block has a keyword that can't appear alone. You can't have an except
    > without a try, or an else without an if, but you could have a def
    > without a decorate.


    Is this merely a pedantic argument (not sure I use "pedantic" correctly)
    or is this an argument based on presumed difficulties in implementing
    the idea?

    I ask because I'm not sure the issue matters to anyone writing
    or reading the code. It certainly wouldn't bother me that
    with if/else it's the first part that's required, while with
    decorate/def it's the second part.

    -Peter
     
    Peter Hansen, Aug 12, 2004
    #9
  10. John Marshall

    Carl Banks Guest

    Peter Hansen wrote:
    >
    >
    > Carl Banks wrote:
    >
    >> (Steven Bethard) wrote in message news:<>...
    >>
    >>>I know there were complaints before about this sort of indentation,
    >>>but I couldn't find them in python-dev and the comments in the wiki
    >>>don't discuss this in any detail. Can anyone tell me why this
    >>>indentation syntax was dispreferred? Specifically, I'm interested in
    >>>why this is so drastically different from the other paired blocks:
    >>>if/elif/else, try/except, try/finally, etc.

    >>
    >> One thing that's different is that, in all those cases, the second
    >> block has a keyword that can't appear alone. You can't have an except
    >> without a try, or an else without an if, but you could have a def
    >> without a decorate.

    >
    > Is this merely a pedantic argument (not sure I use "pedantic" correctly)
    > or is this an argument based on presumed difficulties in implementing
    > the idea?


    Pedantic.


    > I ask because I'm not sure the issue matters to anyone writing
    > or reading the code. It certainly wouldn't bother me that
    > with if/else it's the first part that's required, while with
    > decorate/def it's the second part.


    Well, the only thing is, whenever you see a def statement, you don't
    know if it's decorated right away, so you have to expend one or two
    extra ergs of energy to look above it and see if it is. (And let's
    face it, with the dozens and dozens of decorators that will be applied
    to each and every function, it might be a couple screens up. :)

    To me, it's totally irrelevant.

    I think it would bother some other people, though. I've seen the very
    same argument used against a do...while statement. (The do...while
    statement didn't bother me either, although ideally I'd prefer do to
    replace try in try...finally.)


    --
    CARL BANKS http://www.aerojockey.com/software
    "If you believe in yourself, drink your school, stay on drugs, and
    don't do milk, you can get work."
    -- Parody of Mr. T from a Robert Smigel Cartoon
     
    Carl Banks, Aug 12, 2004
    #10
  11. Peter Hansen wrote:
    > Carl Banks wrote:
    >> One thing that's different is that, in all those cases, the second
    >> block has a keyword that can't appear alone. You can't have an
    >> except without a try, or an else without an if, but you could have a
    >> def without a decorate.

    ....
    > is this an argument based on presumed difficulties in
    > implementing the idea?


    Looking at the wiki it states (as negative points):
    1 New keyword
    2 Overkill for the simple case like classmethod
    3 Many people felt it was wrong use of an identation suite.
    4 Has the same problem as 5.A, in that the decorate block implicitly
    affects the following def. This does not occur elsewhere in Python.
    5 Technical problems with the current grammar parser if a suite
    *starts* with an optional part. (Ending with an optional part, such
    as "else:" is OK, but starting with one is not.)
    6 No implementation currently exists.

    1 isn't a huge issue if the syntax has significant benefits - but I
    suspect that they would have to be significant to warrant the change.
    (Guido says he's more relaxed about the idea, but that might me a
    relaxation to just "No" rather than "no way, no how, buzz off" kind of
    feeling :)

    As per the comment on the wiki regarding 2), I'm not convinced that:

    decorate:
    staticmethod
    def somefunction(some, args):
    """some
    multi line
    comment"""
    x=function(call)
    if x = ...
    ...
    etc...

    is worse than:

    def somefunction(some, args):
    """some
    multi line
    comment"""
    x=function(call)
    if x = ...
    ...
    etc...
    somefunction = staticmethod(somefunction)

    It *is* marginally longer than @staticmethod, but I don't think it's any
    worse ? On the positive side the "decorate" approach appears to me
    MUCH clearer in complex cases, which strikes me as more pythonic. I
    love perl and python about as much as each other, but for different
    reasons - with one of the things I like about python is that when code
    gets complex it generally stays readable.

    Item 3 is opinion, but I don't know where the arguments are leading to
    that point, so I'll just take that at face value. I'm not convinced,
    but hey :)

    Regarding 4, it strikes me that this isn't the case (unless it strictly
    means "block"!). An import at the beginning of a block changes the
    meaning of a block. Similarly a class statement changes the meaning of
    the defs immediately following it. I'd agree that it's not ideal, but
    the argument that code preceding the def isn't allowed because it
    changes the meaning of the def doesn't make sense to me - that's what
    the @ syntax does.

    Item 6 is always the case for any new feature, so I doubt that's the
    real problem - the real problem here strikes me as item 5.

    I do wonder how difficult it would be to add though...


    Michael.
    --

    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.
     
    Michael Sparks, Aug 12, 2004
    #11
  12. John Marshall

    Paul McGuire Guest

    "Michael Sparks" <> wrote in message
    news:cffbbu$amv$...
    > Peter Hansen wrote:
    > > Carl Banks wrote:
    > >> One thing that's different is that, in all those cases, the second
    > >> block has a keyword that can't appear alone. You can't have an
    > >> except without a try, or an else without an if, but you could have a
    > >> def without a decorate.

    > ...
    > > is this an argument based on presumed difficulties in
    > > implementing the idea?

    >
    > Looking at the wiki it states (as negative points):
    > 1 New keyword
    > 2 Overkill for the simple case like classmethod
    > 3 Many people felt it was wrong use of an identation suite.
    > 4 Has the same problem as 5.A, in that the decorate block implicitly
    > affects the following def. This does not occur elsewhere in Python.
    > 5 Technical problems with the current grammar parser if a suite
    > *starts* with an optional part. (Ending with an optional part, such
    > as "else:" is OK, but starting with one is not.)
    > 6 No implementation currently exists.
    >

    <snip>
    > Item 6 is always the case for any new feature, so I doubt that's the
    > real problem - the real problem here strikes me as item 5.
    >
    > I do wonder how difficult it would be to add though...
    >


    Looking at the code, it appears that this is how the current '@' syntax is
    defined, that a funcdef is optionally preceded by a 'decorators' group,
    consisting of one or more 'decorator' (looking at both Grammar/Grammar and
    compile.c).

    So I think this 'technical problem' is just conjecture. (Should check with
    Anthony Baxter to confirm.)

    -- Paul
     
    Paul McGuire, Aug 12, 2004
    #12
  13. Regarding J2 on http://www.python.org/moin/PythonDecorators ...
    ....
    > 5 Technical problems with the current grammar parser if a suite
    > *starts* with an optional part. (Ending with an optional part,
    > such as "else:" is OK, but starting with one is not.)

    ....
    > Item 6 is always the case for any new feature, so I doubt that's the
    > real problem - the real problem here strikes me as item 5.
    >
    > I do wonder how difficult it would be to add though...


    It doesn't actually seem that difficult to modify the grammar to handle
    this if the decorator block handles *only* decorators. I've just tried
    modifying the Grammar/Grammar file to see how plausible this is, and I
    can get python to build and parse it. (It bombs out because I've not
    done any backend work, and this is the first time I've touched the python
    compiler source)

    The change I made was this:

    --- Python-2.4a2/Grammar/Grammar 2004-08-02 07:09:53.000000000 +0100
    +++ Python-2.4a2-MS/Grammar/Grammar 2004-08-12 11:05:04.085386128 +0100
    @@ -27,10 +27,13 @@
    single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
    file_input: (NEWLINE | stmt)* ENDMARKER
    eval_input: testlist NEWLINE* ENDMARKER
    -
    -decorator: '@' dotted_name [ '(' [arglist] ')' ]
    +# decorator: '@' dotted_name [ '(' [arglist] ')' ]
    +# decorators: decorator ([NEWLINE] decorator)* NEWLINE
    +# funcdef: [decorators] 'def' NAME parameters ':' suite
    +decorator: dotted_name [ '(' [arglist] ')' ]
    decorators: decorator ([NEWLINE] decorator)* NEWLINE
    -funcdef: [decorators] 'def' NAME parameters ':' suite
    +decoratedfuncdef: 'decorate' ':' NEWLINE INDENT decorators DEDENT funcdef
    +funcdef: 'def' NAME parameters ':' suite
    parameters: '(' [varargslist] ')'
    varargslist: (fpdef ['=' test] ',')* ('*' NAME [',' '**' NAME] | '**' NAME) | fpdef ['=' test] (',' fpdef ['=' test])* [',']
    fpdef: NAME | '(' fplist ')'
    @@ -59,7 +62,7 @@
    exec_stmt: 'exec' expr ['in' test [',' test]]
    assert_stmt: 'assert' test [',' test]

    -compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef
    +compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef| decoratedfuncdef | classdef
    if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
    while_stmt: 'while' test ':' suite ['else' ':' suite]
    for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]

    This change seems to allow the following code to be parsed happily:

    class Foo:
    decorate:
    staticmethod
    def hello(who):
    print "woo?", who

    def hello2(who):
    print "woo?", who

    I say *seems* because I'm getting seg faults when I try and *run* the
    above, but the following produces an expected syntax error - _implying_
    that the code above is parsed correctly...

    class ThisShouldBreak:
    decorate:
    staticmethod
    def hello(who):
    print "woo?", who

    def hello2(who):
    print "woo?", who

    def crashandburnhere(who)
    print "woo?", who

    (Backtrace from the seg faults seem to point at other stuff as well,
    but I don't know the python source well enough to say for certain!
    I suspect the reason is because I've yanked the leading "@" more than
    anything else)

    There is a restriction here - the contents of the decorate block must
    be limited to what you would get after an "@" - ie just declarations -
    it's not a "suite" (bunch of statements).


    Michael.
    --

    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.
     
    Michael Sparks, Aug 12, 2004
    #13
  14. Michael Sparks wrote:

    > Regarding J2 on http://www.python.org/moin/PythonDecorators ...
    > ...
    >> 5 Technical problems with the current grammar parser if a suite
    >> *starts* with an optional part. (Ending with an optional part,
    >> such as "else:" is OK, but starting with one is not.)

    > ...
    >> Item 6 is always the case for any new feature, so I doubt that's the
    >> real problem - the real problem here strikes me as item 5.
    >>
    >> I do wonder how difficult it would be to add though...

    >
    > It doesn't actually seem that difficult to modify the grammar to
    > handle this if the decorator block handles *only* decorators. I've
    > just tried modifying the Grammar/Grammar file to see how plausible
    > this is, and I can get python to build and parse it. (It bombs out
    > because I've not done any backend work, and this is the first time
    > I've touched the python compiler source)


    Done a bit more work and it certainly *is* caused by the backend
    logic not in step with the grammar change I made rather than it not
    building/parsing correctly. I've tried changing things to make this
    work, but at this exact instant I don't have the time to do this. (I
    might take another look this evening, it doesn't look *too* difficult
    to do)

    I've also changed the grammar rules again to make it a smaller change:

    --- Python-2.4a2/Grammar/Grammar 2004-08-02 07:09:53.000000000 +0100
    +++ Python-2.4a2-MS/Grammar/Grammar 2004-08-12 12:12:11.567115840 +0100
    @@ -28,9 +28,10 @@
    file_input: (NEWLINE | stmt)* ENDMARKER
    eval_input: testlist NEWLINE* ENDMARKER

    -decorator: '@' dotted_name [ '(' [arglist] ')' ]
    +decorator: dotted_name [ '(' [arglist] ')' ]
    decorators: decorator ([NEWLINE] decorator)* NEWLINE
    -funcdef: [decorators] 'def' NAME parameters ':' suite
    +funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT] 'def' NAME parameters ':' suite
    parameters: '(' [varargslist] ')'
    varargslist: (fpdef ['=' test] ',')* ('*' NAME [',' '**' NAME] | '**' NAME) | fpdef ['=' test] (',' fpdef ['=' test])* [',']
    fpdef: NAME | '(' fplist ')'

    This builds, and I've changed some of Python/compile.c to handle this:
    --- Python-2.4a2/Python/compile.c 2004-08-04 11:26:08.000000000 +0100
    +++ Python-2.4a2-MS/Python/compile.c 2004-08-12 12:19:47.570792744 +0100
    @@ -4043,13 +4043,13 @@
    {
    /* decorator: '@' dotted_name [ '(' [arglist] ')' ] */
    int nch = NCH(n);
    - assert(nch >= 2);
    - REQ(CHILD(n, 0), AT);
    - com_decorator_name(c, CHILD(n, 1));
    + assert(nch >= 1);
    + // REQ(CHILD(n, 0), AT);
    + com_decorator_name(c, CHILD(n, 0));

    if (nch > 2) {
    assert(nch == 4 || nch == 5);
    - REQ(CHILD(n, 2), LPAR);
    + REQ(CHILD(n, 1), LPAR);
    REQ(CHILD(n, nch - 1), RPAR);
    com_call_function(c, CHILD(n, 3));
    }
    @@ -4083,11 +4083,14 @@
    PyObject *co;
    int ndefs, ndecorators;
    REQ(n, funcdef);
    - /* -6 -5 -4 -3 -2 -1
    + /* -6 -5 -4 -3 -2 -1
    funcdef: [decorators] 'def' NAME parameters ':' suite */
    + /* -7 -6 -5 -4 -3 -2 -1
    + funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT ] 'def' NAME parameters ':' suite */
    +

    - if (NCH(n) == 6)
    - ndecorators = com_decorators(c, CHILD(n, 0));
    + if (NCH(n) == 11)
    + ndecorators = com_decorators(c, CHILD(n, 4));
    else
    ndecorators = 0;

    @@ -5823,9 +5826,9 @@
    */
    case decorator:
    if (TYPE(n) == decorator) {
    - /* decorator: '@' dotted_name [ '(' [arglist] ')' ] */
    + /* decorator: dotted_name [ '(' [arglist] ')' ] */
    node *name, *varname;
    - name = CHILD(n, 1);
    + name = CHILD(n, 0);
    REQ(name, dotted_name);
    varname = CHILD(name, 0);
    REQ(varname, NAME);

    However I'm now getting a new error instead (just before a controlled
    core dump):

    Fatal Python error: unknown scope for staticmethod in Foo(1) in ./foo.py
    symbols: {'hello2': 2, 'hello': 2}
    locals: {'hello2': 0, 'hello': 1}
    globals: {}

    Aborted (core dumped)

    Smoke test is this:
    ---------
    class Foo:
    decorate:
    staticmethod
    def hello(who):
    print "woo?", who
    def hello2(who):
    print "woo?", who

    Foo.hello("HOO")
    ---------

    I've got to put this aside for the moment, but I'll come back to it
    later. (I think this is actually pretty close to working though...)


    Michael.
    --

    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.
     
    Michael Sparks, Aug 12, 2004
    #14
  15. John Marshall

    Peter Hansen Guest

    Carl Banks wrote:

    > Well, the only thing is, whenever you see a def statement, you don't
    > know if it's decorated right away, so you have to expend one or two
    > extra ergs of energy to look above it and see if it is.


    Well, that argument could be used against any decorator syntax,
    (except those which completely obscure the def, and there would be
    obvious objections to such a syntax as well).

    Likewise, when you see a def, you don't know for sure right away if
    it has a body, or just a big doc string, or perhaps only a pass
    statement.

    Neither can you tell without looking in a module whether it has
    functions, classes, or just data. Or maybe just a bunch of comments,
    or many, many empty lines. You even have to turn on the computer
    before you can find the name of the file!

    Damn, you mean we actually have to look at the code to figure
    out what it does?! ;-)

    -Peter
     
    Peter Hansen, Aug 12, 2004
    #15
  16. [Implementation] (first attempt) Re: A decorator syntax not yet mentioned (I think!)

    Michael Sparks wrote:
    >> Regarding J2 on http://www.python.org/moin/PythonDecorators ...
    >> ...
    >>> 5 Technical problems with the current grammar parser if a suite
    >>> *starts* with an optional part. (Ending with an optional part,
    >>> such as "else:" is OK, but starting with one is not.)

    >> ...
    >>> Item 6 is always the case for any new feature, so I doubt that's the
    >>> real problem - the real problem here strikes me as item 5.
    >>>
    >>> I do wonder how difficult it would be to add though...


    I've implemented this now, and I have this (kinda*) working. I've
    patched the 2.4a2 release from python.org and made the following
    changes.
    * Kinda - there's something wrong with my implementation due to
    scoping, and since I only first looked at the python source for the
    first time 4-5 hours ago, I think this is progress :)

    I changed the grammar to allow syntax J2:

    --- Python-2.4a2/Grammar/Grammar 2004-08-02 07:09:53.000000000 +0100
    +++ Python-2.4a2-MS/Grammar/Grammar 2004-08-12 13:21:16.577978408 +0100
    @@ -30,5 +30,5 @@

    -decorator: '@' dotted_name [ '(' [arglist] ')' ]
    +decorator: dotted_name [ '(' [arglist] ')' ]
    decorators: decorator ([NEWLINE] decorator)* NEWLINE
    -funcdef: [decorators] 'def' NAME parameters ':' suite
    +funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT ] 'def' NAME parameters ':' suite
    parameters: '(' [varargslist] ')'

    I then modified comple.c to make this syntax usable:

    --- Python-2.4a2/Python/compile.c 2004-08-04 11:26:08.000000000 +0100
    +++ Python-2.4a2-MS/Python/compile.c 2004-08-12 14:24:50.537168480 +0100
    @@ -4045,12 +4045,12 @@
    int nch = NCH(n);
    - assert(nch >= 2);
    - REQ(CHILD(n, 0), AT);
    - com_decorator_name(c, CHILD(n, 1));
    + assert(nch >= 1);
    + com_decorator_name(c, CHILD(n, 0));

    if (nch > 2) {
    - assert(nch == 4 || nch == 5);
    - REQ(CHILD(n, 2), LPAR);
    + assert(nch == 3 || nch == 4);
    + REQ(CHILD(n, 1), LPAR);
    REQ(CHILD(n, nch - 1), RPAR);
    - com_call_function(c, CHILD(n, 3));
    + com_call_function(c, CHILD(n, 2));
    }
    +
    }
    @@ -4085,7 +4085,7 @@
    REQ(n, funcdef);
    - /* -6 -5 -4 -3 -2 -1
    - funcdef: [decorators] 'def' NAME parameters ':' suite */
    -
    - if (NCH(n) == 6)
    - ndecorators = com_decorators(c, CHILD(n, 0));
    + /* -7 -6 -5 -4 -3 -2 -1
    + funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT ] 'def' NAME parameters ':' suite */
    +
    + if (NCH(n) == 11)
    + ndecorators = com_decorators(c, CHILD(n, 4));
    else
    @@ -5825,5 +5825,5 @@
    if (TYPE(n) == decorator) {
    - /* decorator: '@' dotted_name [ '(' [arglist] ')' ] */
    + /* decorator: dotted_name [ '(' [arglist] ')' ] */
    node *name, *varname;
    - name = CHILD(n, 1);
    + name = CHILD(n, 0);
    REQ(name, dotted_name);

    ====================================
    The one sting in this, which I haven't figured out is that the decorate
    keyword and indentation appear to knock the scoping for six (which I
    suspect/hope would be simple to solve) so for *this* implementation you
    need to do something like:

    =========================
    class Foo:
    staticmethod # This is cruft to force staticmethod into scope

    decorate:
    staticmethod
    def hello(who):
    print "woo?", who

    Foo.hello("HOO")
    =========================

    If you miss out the cruft above, python can't (with the above small
    change) find the right scope for "staticmethod".

    I suspect that someone with more experience with the source tree would
    be able to remove the cruft line above and allow the scope to be searched
    correctly.

    Either way, the argument on http://www.python.org/moin/PythonDecorators
    regarding J2 not having an implementation just (80%) disappeared :)

    Should I forward this to the dev list?


    Michael.
    --

    British Broadcasting Corporation, Research and Development
    Kingswood Warren, Surrey KT20 6NP

    This message (and any attachments) may contain personal views
    which are not the views of the BBC unless specifically stated.
     
    Michael Sparks, Aug 12, 2004
    #16
  17. On Wed, 11 Aug 2004 11:40:32 -0400, Peter Hansen <> wrote:

    > John Marshall wrote:
    >> How about the following, which I am almost positive
    >> has not been suggested:
    >> -----
    >> class Klass:
    >> def __init__(self, name):
    >> self.name = name
    >>
    >> deco meth0:
    >> staticmethod
    >> def meth0(x):
    >> return x
    >>
    >> -----
    >> 1) The decorators clearly apply to a specific method/function,
    >> therefore there is no need to do any stack pushing in memory
    >> until the method/function definition is done.

    >
    > Problems with that: duplication (since you now have the name
    > of the function in two places) and readability (it's harder to
    > see the "def" because the name you picked is too close to it
    > in length and appearance).
    >
    > On the other hand, there's no need to have the function name
    > there (was anyone complaining about "stack pushing"?), so
    > maybe just using "decorate" would be okay, or a different
    > keyword:
    >
    > class Klass:
    > # init set
    >
    > decorate:
    > staticmethod
    > def meth0(x):
    > return x

    I think 'mutate' is better then 'decorate', and anyway I like to see what
    is mutated and the possibility to explicitate it make possible
    to move decorations from this position ,I put them where I like (at least
    if I have to call them decorations) even in another file...

    so

    class Klass:
    def meth0(x):
    return x
    mutate meth0:
    staticmethod

    mutate Klass.meth0:
    debugged


    reads good to me.


    Paolino



    --
    .....lotta dura per la verdura
     
    paolo veronelli, Aug 12, 2004
    #17
  18. John Marshall

    Mark Bottjer Guest

    Michael Sparks wrote:
    > Looking at the wiki it states (as negative points):
    > 1 New keyword
    > 2 Overkill for the simple case like classmethod
    > 3 Many people felt it was wrong use of an identation suite.
    > 4 Has the same problem as 5.A, in that the decorate block implicitly
    > affects the following def. This does not occur elsewhere in Python.
    > 5 Technical problems with the current grammar parser if a suite
    > *starts* with an optional part. (Ending with an optional part, such
    > as "else:" is OK, but starting with one is not.)
    > 6 No implementation currently exists.

    [snip]
    > Item 3 is opinion, but I don't know where the arguments are leading to
    > that point, so I'll just take that at face value.


    The contention was that indentation is used to indicate the effective
    scope in other contexts: e.g., def changes how the statements *indented
    under it* are handled (they are packed into a code object instead of
    executed immediately), but doesn't change how the next statement at the
    same indentation level is handled (the statement result may change, due
    to coupling via common data, but the statement itself is handled exactly
    as it would be were the previous statement not present). With this
    syntax, though, the decorate block changes how the def statement is
    handled, even though they are at the same indentation level.

    Put another way: applying what I know about how indentation is used
    elsewhere in Python to this syntax, I would expect the effect of the
    decorate statement to be limited to the statements indented under it. I
    would not expect it to affect the next statement at the same level
    except by the normal coupling of namespace (program state).

    Of course, this argument also applies to the prefix @ syntax, but at
    least with that we have a funny character cluing us in to the special
    behavior.

    > Regarding 4, it strikes me that this isn't the case (unless it strictly
    > means "block"!).


    It meant strictly "block". :)

    > An import at the beginning of a block changes the
    > meaning of [the rest of] a block.


    Yes, by means of changing the program state--specifically, the namespace
    used to resolve identifiers. The decorate block changes the meaning of
    only the *next* statement by means of changing the parser (not program)
    state. I see them as quite different. YMMV.

    > Similarly a class statement changes the meaning of
    > the defs immediately following it.


    It affects the meaning of the defs indented under it. The only way it
    affects anything after it is by the modifications it makes to the
    namespace. Again, the statements at the same indentation level are
    coupled only via namespace, not parser state. The statements in indented
    suites are coupled via parser state, but that coupling is made obvious
    via the indentation.

    > I'd agree that it's not ideal, but
    > the argument that code preceding the def isn't allowed because it
    > changes the meaning of the def doesn't make sense to me - that's what
    > the @ syntax does.


    Indeed, and that partially why I like the '@' symbol (or possibly a meta
    keyword). If we're going to add something which completely disregards
    the existing indentation/coupling idioms, then it should have an obvious
    visual trigger.

    -- Mark
     
    Mark Bottjer, Aug 12, 2004
    #18
  19. John Marshall

    Mark Bottjer Guest

    Paul McGuire wrote:
    > Looking at the code, it appears that this is how the current '@'
    > syntax is defined, that a funcdef is optionally preceded by a
    > 'decorators' group, consisting of one or more 'decorator' (looking at
    > both Grammar/Grammar and compile.c).
    >
    > So I think this 'technical problem' is just conjecture. (Should
    > check with Anthony Baxter to confirm.)


    It looks like they plan on getting around this "problem" by explicitly
    eating newlines whenever they match the special '@' rule. This solves
    the look-ahead problem with the parser, albeit in kind of an ugly way.

    I don't have my source handy, does anything else eat newlines like this?

    -- Mark
     
    Mark Bottjer, Aug 12, 2004
    #19
  20. John Marshall

    Peter Hansen Guest

    Mark Bottjer wrote:
    > The contention was that indentation is used to indicate the effective
    > scope in other contexts: e.g., def changes how the statements *indented
    > under it* are handled (they are packed into a code object instead of
    > executed immediately), but doesn't change how the next statement at the
    > same indentation level is handled (the statement result may change, due
    > to coupling via common data, but the statement itself is handled exactly
    > as it would be were the previous statement not present).


    > With this
    > syntax, though, the decorate block changes how the def statement is
    > handled, even though they are at the same indentation level.


    Changes it how? The definition of this whole decorator idea has been
    that it is equivalent to applying the decorator functions *after* the
    def has completed, as in "func = decorator(func)". This does not
    in any way changed how the def statement itself is handled.

    > Put another way: applying what I know about how indentation is used
    > elsewhere in Python to this syntax, I would expect the effect of the
    > decorate statement to be limited to the statements indented under it. I
    > would not expect it to affect the next statement at the same level
    > except by the normal coupling of namespace (program state).


    You don't think of "if" and "else" as being related? When the
    expression that is evaluated in the if is true, the "else"
    is skipped over... I know: that's just program state as you said.
    But so is the effect of the decorator, whether it's spelled "@"
    or "decorate:"...

    Anyway, whatever works for @pie syntax should work for decorate:
    syntax, I would think. @pie is even less like the rest of Python
    (especially now with this weird "swallow newlines" hack to work
    around the possibility that people might try to put multiple
    @decorators on the same line).

    > Of course, this argument also applies to the prefix @ syntax, but at
    > least with that we have a funny character cluing us in to the special
    > behavior.


    While here we have a nice explicit keyword "decorate:" which one
    can easily find with a search in the documentation, as opposed to
    trying to look up a symbol "@". I don't buy the argument that a
    funny symbol is somehow going to help people who don't already know
    what decorators are, any more than an explicit "decorate:" line
    would. Either one says "this is different, go look up what it
    means" to a newcomer.

    -Peter
     
    Peter Hansen, Aug 12, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steven Bethard

    Think you've got a new decorator syntax?

    Steven Bethard, Aug 16, 2004, in forum: Python
    Replies:
    1
    Views:
    281
    David Fraser
    Aug 19, 2004
  2. Ron_Adam
    Replies:
    3
    Views:
    301
    Ron_Adam
    Apr 5, 2005
  3. Replies:
    0
    Views:
    613
  4. Berehem
    Replies:
    4
    Views:
    562
    Lawrence Kirby
    Apr 28, 2005
  5. glomde
    Replies:
    5
    Views:
    525
    glomde
    Mar 29, 2007
Loading...

Share This Page