simpleparse parsing problem

Discussion in 'Python' started by David Hirschfield, Sep 2, 2006.

  1. Anyone out there use simpleparse? If so, I have a problem that I can't
    seem to solve...I need to be able to parse this line:

    """Cen2 = Cen(OUT, "Cep", "ies", wh, 544, (wh/ht));"""

    with this grammar:

    grammar = r'''
    declaration := ws, line, (ws, line)*, ws
    line := (statement / assignment), ';', ws
    assignment := identifier, ws, '=', ws, statement
    statement := identifier, '(', arglist?, ')', chars?
    identifier := ([a-zA-Z0-9_.:])+
    arglist := arg, (',', ws, arg)*
    arg := expr/ statement / identifier / num / str /
    curve / spline / union / conditional / definition
    definition := typedef?, ws, identifier, ws, '=', ws, arg
    typedef := ([a-zA-Z0-9_])+
    expr := termlist, ( operator, termlist )+
    termlist := ( '(', expr, ')' ) / term
    term := call / identifier / num
    call := identifier, '(', arglist?, ')'
    union := '{{', ws, (arg, ws, ';', ws)*, arg, ws, '}}'
    operator := ( '+' / '-' / '/' / '*' /
    '==' / '>=' / '<=' / '>' / '<' )
    conditional := termlist, ws, '?', ws, termlist, ws, ':', ws, termlist
    curve := (list / num), '@', num
    spline := (cv, ',')*, cv
    cv := identifier, '@', num
    list := '[', arg, (',', ws, arg)*, ']'
    str := '"', ([;] / chars)*, '"'
    num := ( scinot / float / int )
    <chars> := ('-' / '/' / '?' / [a-zA-Z0-9_.!@#$%^&\*\+=<> :])+
    <int> := ([-+]?, [0-9]+)
    <float> := ([-+]?, [0-9\.]+)
    <scinot> := (float, 'e', int)
    <ws> := [ \t\n]*
    '''

    But it fails. The problem is with how arglist/arg/expr are defined,
    which makes it unable to handle the parenthesized expression at the end
    of the line:

    (wh/ht)

    But everything I've tried to correct that problem fails. In the end, it
    needs to be able to parse that line with those parentheses around wh/ht,
    or without them.
    Recursive parsing of expressions just seems hard to do in simpleparse,
    and is beyond my parsing knowledge.

    Here's the code to get the parser going:

    from simpleparse.parser import Parser
    p = Parser(grammar, 'line')
    import pprint
    bad_line = """Cen2 = Cen(OUT, "Cep", "ies", wh, 544, (wh/ht));"""

    pprint.pprint(p.parse(bad_line))


    Any help greatly appreciated, thanks,
    -Dave


    --
    Presenting:
    mediocre nebula.
    David Hirschfield, Sep 2, 2006
    #1
    1. Advertising

  2. David Hirschfield

    Paul McGuire Guest

    "David Hirschfield" <> wrote in message
    news:...
    > Anyone out there use simpleparse? If so, I have a problem that I can't
    > seem to solve...I need to be able to parse this line:
    >
    > """Cen2 = Cen(OUT, "Cep", "ies", wh, 544, (wh/ht));"""
    >
    > with this grammar:
    >
    > grammar = r'''
    > declaration := ws, line, (ws, line)*, ws
    > line := (statement / assignment), ';', ws
    > assignment := identifier, ws, '=', ws, statement
    > statement := identifier, '(', arglist?, ')', chars?
    > identifier := ([a-zA-Z0-9_.:])+
    > arglist := arg, (',', ws, arg)*
    > arg := expr/ statement / identifier / num / str /
    > curve / spline / union / conditional / definition
    > definition := typedef?, ws, identifier, ws, '=', ws, arg
    > typedef := ([a-zA-Z0-9_])+
    > expr := termlist, ( operator, termlist )+
    > termlist := ( '(', expr, ')' ) / term
    > term := call / identifier / num
    > call := identifier, '(', arglist?, ')'
    > union := '{{', ws, (arg, ws, ';', ws)*, arg, ws, '}}'
    > operator := ( '+' / '-' / '/' / '*' /
    > '==' / '>=' / '<=' / '>' / '<' )
    > conditional := termlist, ws, '?', ws, termlist, ws, ':', ws, termlist
    > curve := (list / num), '@', num
    > spline := (cv, ',')*, cv
    > cv := identifier, '@', num
    > list := '[', arg, (',', ws, arg)*, ']'
    > str := '"', ([;] / chars)*, '"'
    > num := ( scinot / float / int )
    > <chars> := ('-' / '/' / '?' / [a-zA-Z0-9_.!@#$%^&\*\+=<> :])+
    > <int> := ([-+]?, [0-9]+)
    > <float> := ([-+]?, [0-9\.]+)
    > <scinot> := (float, 'e', int)
    > <ws> := [ \t\n]*
    > '''

    <snip>
    >


    David -

    I converted your simpleparse grammar to pyparsing, which I could then
    troubleshoot. Here is a working pyparsing grammar, perhaps you can convert
    it back to simpleparse form and see if you make any better progress.

    -- Paul


    test = """Cen2 = Cen(OUT, "Cep", "ies", wh, 544, (wh/ht));"""


    from pyparsing import *

    # recursive items need forward decl - assign contents later using '<<'
    operator
    arg = Forward()
    expr = Forward()
    statement = Forward()

    float_ = Regex (r"[-+]?[0-9]+\.[0-9]*")
    int_ = Regex (r"[-+]?[0-9]+")
    scinot = Combine(float_ + oneOf(list("eE")) + int_)
    num = scinot | float_ | int_
    str_ = dblQuotedString
    list_ = "[" + delimitedList(arg) + "]"
    identifier = Word(alphas, srange("[a-zA-Z0-9_.:]"))
    cv = identifier + "@" + num
    spline = delimitedList(cv)
    curve = (list_ | num) + "@" + num
    conditional = expr + "?" + expr + ":" + expr
    operator = oneOf( ('+', '-', '/', '*', '==', '>=', '<=', '>', '<') )
    union = "{{" + delimitedList( arg, delim=";" ) + "}}"
    call = identifier + "(" + delimitedList(arg) + ")"
    term = call | identifier | num | Group( "(" + expr + ")" )
    expr << (term + ZeroOrMore( operator+term ) )
    typedef = Word( alphas, alphanums )
    definition = ( (typedef + identifier) | identifier ) + "=" + arg
    arg << (expr | statement | identifier | num | str_ | "." |
    curve | spline | union | conditional | definition)
    assignment = identifier + "=" + statement
    statement << ( call | assignment )
    line_ = statement + ';'
    declaration = OneOrMore(line_)

    print declaration.parseString(test)

    Prints:
    ['Cen2', '=', 'Cen', '(', 'OUT', '"Cep"', '"ies"', 'wh', '544', ['(', 'wh',
    '/', 'ht', ')'], ')', ';']
    Paul McGuire, Sep 2, 2006
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. GIMME
    Replies:
    2
    Views:
    869
    GIMME
    Feb 11, 2004
  2. Naren
    Replies:
    0
    Views:
    572
    Naren
    May 11, 2004
  3. Fortepianissimo

    semantic operation in simpleparse?

    Fortepianissimo, Sep 18, 2003, in forum: Python
    Replies:
    0
    Views:
    309
    Fortepianissimo
    Sep 18, 2003
  4. David Isaac

    SimpleParse installer available for 2.5

    David Isaac, Oct 2, 2006, in forum: Python
    Replies:
    0
    Views:
    254
    David Isaac
    Oct 2, 2006
  5. Laszlo Nagy
    Replies:
    0
    Views:
    222
    Laszlo Nagy
    Feb 24, 2008
Loading...

Share This Page