About the grammar

[franck]

Dear all,

I'm wondering why in Python's grammar, keyword arguments are specified
as:

argument: ... | test '=' test

I would have expected something like

argument: ... | NAME '=' test

Indeed, I cannot imagine a case where the keyword is something else
than an identifier. Moreover, in the Python language reference (see
http://docs.python.org/reference/expressions.html#grammar-token-keyword_item)
one can read:

keyword_item ::= identifier "=" expression

which is what I was expecting.

Does any one knows why the grammar is so coded? Any intuition?

Thanks in advance!
Franck

 

[Steven D'Aprano]

Dear all,

I'm wondering why in Python's grammar, keyword arguments are specified
as:

argument: ... | test '=' test

Where are you finding that?

I would have expected something like

argument: ... | NAME '=' test

Indeed, I cannot imagine a case where the keyword is something else than
an identifier. Moreover, in the Python language reference (see
http://docs.python.org/reference/expressions.html#grammar-token- keyword_item)
one can read:

keyword_item ::= identifier "=" expression

which is what I was expecting.

This tells you that keyword arguments cannot have keywords that aren't
identifiers:
File "<stdin>", line 1
SyntaxError: keyword can't be an expression


The only thing I can think of is that you're extrapolating from use cases
like this:

.... return x
....2

But that's just a special case of this:
2


Of course, I could be completely misunderstanding what you mean.

 

[Martin v. Loewis]

Does any one knows why the grammar is so coded? Any intuition?

The 2.7 Grammar clarifies that:

# The reason that keywords are test nodes instead of NAME is that using
# NAME results in an ambiguity. ast.c makes sure it's a NAME.
argument: test [comp_for] | test '=' test

The ambiguity is this: if I have

foo(a, b = 2)

and have parsed it up to

foo(a,

then, if I get "b", should I enter the "test" clause (for the left
alternative) or the right clause? With the grammar being as stated,
it can be parsed as

argument: test ([comp_for] | '=' test)

so that parsing always starts with a test. Afterwards, I either get a
'=', indicating a keyword argument, or not: the '=' is not in the FIRST
set of comp_for, so there is no ambiguity here.

HTH,
Martin

 

[franck]

Thank you for this very clear (and quick) explanation!

Cheers,
Franck

# The reason that keywords are test nodes instead of NAME is that using
# NAME results in an ambiguity. ast.c makes sure it's a NAME.
argument: test [comp_for] | test '=' test

[...]

 

[franck]

    argument: ... | test '=' test

Where are you finding that?

This comes from Python-2.6/Grammar/Grammar in the source distribution.

This tells you that keyword arguments cannot have keywords that aren't
identifiers:


  File "<stdin>", line 1
SyntaxError: keyword can't be an expression

Sure! So my surprise.

But Martin did provide a very good explanation that this form in the
grammar actually allows to avoid an ambiguity.

Cheers,
Franck