flex: plex?

Discussion in 'Python' started by Peter Kleiweg, Aug 17, 2004.

  1. Hi,

    I am looking for a Python equivalent of Flex. After some
    browsing, it seems that Plex is my best bet, but I would like to
    hear suggestions before I dive in.

    I have been using Flex a lot in combination with C programming.
    I never use Yacc (or Bison), though Flex and Yacc are supposed
    to be used together. But I found Yacc too cumbersome, and
    unnecessary. Yacc is about context-free grammars, nice if you
    like to describe an input language as a set of context-free
    rewrite rules. Flex is about finite-state automata, but can be
    extended to push-down automata (equivalent to context-free
    grammars) and beyond, as long as it stays deterministic. I found
    this to be very productive. I prefer automata over rewrite rules.

    So, what I am looking for is a Python version of Flex that
    supports the following capabilities, which are also supported by Plex:

    - assigning any Python code to a matched input pattern

    - input pattern recognition depending on 'states', which can be
    changed through actions

    Things that seems to be missing from Plex:

    - patterns like:
    A/B : match A only if followed by B, give A as result, keep B
    in the input stream

    - actions like:
    REJECT: ask the flexer to come up with the next best match for the
    same input
    LESS(n): push n characters back into the input stream

    All these are not really necessary, but they make some things
    simpler.

    So, are there any other packages that do these things better
    than Plex?

    --
    Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
    info: http://www.let.rug.nl/~kleiweg/ls.html
    Peter Kleiweg, Aug 17, 2004
    #1
    1. Advertising

  2. Peter Kleiweg

    Jeff Epler Guest

    If plex uses python regular expressions, you can use the zero-length
    lookahead assertion instead of flex's A/B syntax: "A(?=B)"

    Jeff

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.4 (GNU/Linux)

    iD8DBQFBInqKJd01MZaTXX0RAh5dAKCivbanEnAb1N7QShDi5ULE/+32DwCfTG34
    cb0k64glgd54ryIoemvZBSo=
    =l+Mj
    -----END PGP SIGNATURE-----
    Jeff Epler, Aug 17, 2004
    #2
    1. Advertising

  3. At some point, Peter Kleiweg <> wrote:

    > Hi,
    >
    > I am looking for a Python equivalent of Flex. After some
    > browsing, it seems that Plex is my best bet, but I would like to
    > hear suggestions before I dive in.
    >
    > I have been using Flex a lot in combination with C programming.
    > I never use Yacc (or Bison), though Flex and Yacc are supposed
    > to be used together. But I found Yacc too cumbersome, and
    > unnecessary. Yacc is about context-free grammars, nice if you
    > like to describe an input language as a set of context-free
    > rewrite rules. Flex is about finite-state automata, but can be
    > extended to push-down automata (equivalent to context-free
    > grammars) and beyond, as long as it stays deterministic. I found
    > this to be very productive. I prefer automata over rewrite rules.


    If you've been using Flex, you might want to check out FlexModule at
    <http://www.cs.utexas.edu/users/mcguire/software/fbmodule/>
    which allows you to use Flex to generate a lexer callable from python.

    --
    |>|\/|<
    /--------------------------------------------------------------------------\
    |David M. Cooke
    |cookedm(at)physics(dot)mcmaster(dot)ca
    David M. Cooke, Aug 17, 2004
    #3
  4. Jeff Epler schreef:

    > If plex uses python regular expressions, you can use the zero-length
    > lookahead assertion instead of flex's A/B syntax: "A(?=B)"


    Alas, it doesn't.


    --
    Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
    info: http://www.let.rug.nl/~kleiweg/ls.html
    Peter Kleiweg, Aug 17, 2004
    #4
  5. David M. Cooke schreef:


    > If you've been using Flex, you might want to check out FlexModule at
    > <http://www.cs.utexas.edu/users/mcguire/software/fbmodule/>
    > which allows you to use Flex to generate a lexer callable from python.


    I have tried it. (Had to edit the Makefile by hand, because
    "make -f Makefile.pre.in boot" did not produce a valid Makefile,
    but I did manage to build and run the example.) The problem is,
    the Flex file (and Yacc file) is compiled into C, and then into
    a shared lib. This means I must use C code in the Flex file.

    What I want is something that lets me call arbitrary Python code
    from the lexer. Ideal would be to have a version of Flex that
    produces a Python file instead of a C file. I have kept looking,
    and I think I found what I was looking for: PyLly, which is part
    of PyGgy. http://www.lava.net/~newsham/pyggy/


    --
    Peter Kleiweg L:NL,af,da,de,en,ia,nds,no,sv,(fr,it) S:NL,de,en,(da,ia)
    info: http://www.let.rug.nl/~kleiweg/ls.html
    Peter Kleiweg, Aug 18, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. MM

    Flex model concept?

    MM, Oct 8, 2003, in forum: VHDL
    Replies:
    4
    Views:
    3,722
  2. news
    Replies:
    0
    Views:
    513
  3. Uwe Ziegenhagen
    Replies:
    5
    Views:
    7,602
    oceamus
    Jan 27, 2010
  4. Error in Plex 1.1.4.1

    , Jan 19, 2005, in forum: Python
    Replies:
    0
    Views:
    336
  5. greg

    ANN: Plex 1.1.5

    greg, Jan 27, 2007, in forum: Python
    Replies:
    1
    Views:
    324
Loading...

Share This Page