pyparsing and 'keywords'

B

Berteun Damman

Hello,

I'm having some problems with pyparsing, I could not find how to tell
it to view certain words as keywords, i.e. not as a possible variable
name (in an elegant way),
for example, I have this little grammar:

terminator = Literal(";")
expr = Word(alphas)
body = Forward();
ifstat = "if" + body + "fi"
stat = expr | ifstat
body << OneOrMore(stat + terminator)
program = body

I.e. some program which contains statements separated by semicolons. A
statement is either an if [....] fi statement or simply a word.

If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

How to fix this?

Thank you,

Berteun
 
P

Paul McGuire

Berteun Damman said:
Hello,

I'm having some problems with pyparsing, I could not find how to tell
it to view certain words as keywords, i.e. not as a possible variable
name (in an elegant way),
for example, I have this little grammar:

terminator = Literal(";")
expr = Word(alphas)
body = Forward();
ifstat = "if" + body + "fi"
stat = expr | ifstat
body << OneOrMore(stat + terminator)
program = body

I.e. some program which contains statements separated by semicolons. A
statement is either an if [....] fi statement or simply a word.

If I try however to parse the String "if test; testagain; fi;", it does
not work, because the fi is interpreted as an expr, not as the end of
the if statement, and of course, adding another fi doesn't solve this
either.

How to fix this?

Thank you,

Berteun
Berteun -

The simplest way I can think of for this grammar off the top of my head is
to use a parse action to reject keywords.

keywords = [ "if", "fi", "else", "return" ]
def rejectKeywords(string,loc,tokens):
if tokens[0] in keywords:
raise ParseException(string,loc,"found keyword %s" % tokens[0])
expr.setParseAction( rejectKeywords )

I took a different tack in the idl parser that is included in the pyparsing
examples directory, but it is more extensive.

-- Paul
 
B

Berteun Damman

The simplest way I can think of for this grammar off the top of my head is
to use a parse action to reject keywords.

Thank you for your quick and good solution, that did the trick indeed.

But I'm wondering, isn't it possible to have some sort of lexing phase
which already identifies keywords as such? Or to provide a table with
keywords, which pyparsing is able to automatically recognize?

So you would be able to do IF = Keyword("if"), just as a Literal now is
created, but now the parser knows this word shouldn't be interpreted any
other way than as a keyword. Or would that be a bad idea?

Berteun
 
P

Paul McGuire

Berteun Damman said:
Thank you for your quick and good solution, that did the trick indeed.

But I'm wondering, isn't it possible to have some sort of lexing phase
which already identifies keywords as such? Or to provide a table with
keywords, which pyparsing is able to automatically recognize?

So you would be able to do IF = Keyword("if"), just as a Literal now is
created, but now the parser knows this word shouldn't be interpreted any
other way than as a keyword. Or would that be a bad idea?

Berteun
Berteun -

This is not a bad idea in and of itself, as I mentioned earlier, I had
similar problems with the idl parser. For example, if in the body of your
if block you had a statement:

ifThisWorksItsAMiracle;

The leading "if" would match the Literal("if"), although the statement is
actually the complete 'ifThisWorksItsAMiracle' method call. That is,
Literals are not automatically assumed to be whitespace-terminated. This is
part of pyparsing's philosophy of being whitespace-blind.

It's been a while since v 1.2.2 was released, I think adding a Keyword class
much as you describe (on top of the other bug-fixes and additions since
then) would be sufficient material for coming up with a 1.3 release. I'll
work on it over the holiday "break," assuming I get one!

-- Paul
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

ANN: pyparsing 1.5.6 released! 1
pyparsing problem 3
[ANN] pyparsing 1.5.3 released 0
PyParsing and Headaches 4
pyparsing question 3
pyparsing and svg 2
help with pyparsing 1
Pyparsing... 2

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top