Writing lex-like scanners in Perl

[Clint Olsen]

I read the example in perlre(1) regarding writing lexical analyzers in Perl
using /gc, but I was wondering if there might be another approach. One
problem with doing the if{}elsif{}..else{} technique is that this does not
really simulate traditional scanner behavior. Scanners always try to make
the longest match, not the first. One way to mimic the behavior more
faithfully would be to put the entire lexical analyzer in one pattern. But
I wasn't sure if it was possible (even legibly) using extended regular
expressions (/x) to put executable code and embed it in the patterns:

/(alt1) {return "alt1"} |(alt2) { return "alt2"}|(alt3) { return
"alt3"}.../iogc

and so on.

If anyone has any experience with this kind of application, I'd like to
hear from you.

Thanks,

-Clint

 

[Ben Morrow]

I read the example in perlre(1) regarding writing lexical analyzers in Perl
using /gc, but I was wondering if there might be another approach. One
problem with doing the if{}elsif{}..else{} technique is that this does not
really simulate traditional scanner behavior. Scanners always try to make
the longest match, not the first. One way to mimic the behavior more
faithfully would be to put the entire lexical analyzer in one pattern. But
I wasn't sure if it was possible (even legibly) using extended regular
expressions (/x) to put executable code and embed it in the patterns:

/(alt1) {return "alt1"} |(alt2) { return "alt2"}|(alt3) { return
"alt3"}.../iogc

See (?{...}) and (??{...}) in perlre.

Ben

 

[Clint Olsen]

See (?{...}) and (??{...}) in perlre.

I did look at this, but I believe I found a problem with pos() for the
search string not being updated if I returned from the function from this
kind of code. So, I would repeat from the beginning of the string every
time. I'll give it a try again and defer the return until after the search
block.

Thanks,

-Clint