lexical analyzer generators

[Jay G. Scott]

i've asked this before, and i went off an read up on the answers,
ie, ParseLex.

well, this doesn't help. i don't see anything out there that's remotely
like lex/flex. the perl tools seem incapable of the sort of flexibility
i want. in flex, i can do things like this:

aab[cd]+ { C code }

acr[a-z][0-9][a-r]* { some other code }
[a-z0-9]+ {yet another chunk}

whereas i don't see anything akin to this in ParseLex.


question:

am i right in thinking that the perl lexical analyzer generators don't
actually generate lexical analyzers to order, they just use their idea of
a lexical analyzer and give you their routine results? i looked at
one of ParseLex's examples and it MAY be that i can get it to return a string
to me, but once i have the string i'd have to sort things to decide which
of the three rules above the string matches. that's exactly the thing i
want to avoid. so if that's what they do, then i don't want to use them.

j.

 

[Tad McClellan]

i've asked this before, and i went off an read up on the answers,
ie, ParseLex.

well, this doesn't help.

Yes it does.

i don't see anything out there that's remotely
like lex/flex.

The module's docs say:

If a sub ref (anonymous sub-
routine) is given as third argument, it is called when the token is
recognized. Its arguments are the "Parse::Token" instance and the
string recognized by the regular expression.

the perl tools seem incapable of the sort of flexibility
i want.

Generalizing to all Perl Tools based on your misreading of the
docs for one of the tools is being rather *too* flexible.

in flex, i can do things like this:

aab[cd]+ { C code }

acr[a-z][0-9][a-r]* { some other code }
[a-z0-9]+ {yet another chunk}

whereas i don't see anything akin to this in ParseLex.

This works for me:

-----------------------------
#!/usr/bin/perl
use warnings;
use strict;
use Parse::Lex;

my @token = (
C => 'aab[cd]+', \&C,
OTHER => 'acr[a-z][0-9][a-r]*', \&OTHER,
YAC => '[a-z0-9]+', \&YAC,
NL => '\n', \&NL,
);

my $lexer = Parse::Lex->new(@token);
$lexer->from(\*DATA);

$lexer->next while not $lexer->eoi;

sub C {warn "C() got called with <<$_[1]>>\n"}
sub OTHER {warn "OTHER() got called with <<$_[1]>>\n"}
sub YAC {warn "YAC() got called with <<$_[1]>>\n"}
sub NL {warn "NL() got called with <<$_[1]>>\n"}


__DATA__
foobar aabdddacrx9

 

[Jay G. Scott]

Yes it does.




The module's docs say:

If a sub ref (anonymous sub-
routine) is given as third argument, it is called when the token is
recognized. Its arguments are the "Parse::Token" instance and the
string recognized by the regular expression.




Generalizing to all Perl Tools based on your misreading of the
docs for one of the tools is being rather *too* flexible.

not what i meant, of course.

in flex, i can do things like this:

aab[cd]+ { C code }

acr[a-z][0-9][a-r]* { some other code }
[a-z0-9]+ {yet another chunk}

whereas i don't see anything akin to this in ParseLex.

This works for me:

-----------------------------
#!/usr/bin/perl
use warnings;
use strict;
use Parse::Lex;

my @token = (
C => 'aab[cd]+', \&C,
OTHER => 'acr[a-z][0-9][a-r]*', \&OTHER,
YAC => '[a-z0-9]+', \&YAC,
NL => '\n', \&NL,
);

my $lexer = Parse::Lex->new(@token);
$lexer->from(\*DATA);

$lexer->next while not $lexer->eoi;

sub C {warn "C() got called with <<$_[1]>>\n"}
sub OTHER {warn "OTHER() got called with <<$_[1]>>\n"}
sub YAC {warn "YAC() got called with <<$_[1]>>\n"}
sub NL {warn "NL() got called with <<$_[1]>>\n"}

not being a perl guru, i doubt i'd have ever figured
this out. certainly i didn't figure it out from the docs
i found.

yeah, i think you've got it for me. thanks.

(i've been on other things for a while, just got back.)

j.