Perl pattern 5.6+ bug causing Parse::Yapp to fail

Discussion in 'Perl Misc' started by Clint Olsen, Jan 16, 2004.

  1. Clint Olsen

    Clint Olsen Guest

    I'm using Perl 5.8.2, and I'm seeing some strange behavior from the pattern
    matching engine.

    Parse::Yapp is a module written to generate Perl in a yacc-like fashion.
    However, I can't get the debugging version to work because it's not
    correctly parsing between the markers in the file.

    In debug mode, the module calls and modifies itself, uncommenting debug
    blocks and then 'eval'-ing the string to produce a debug version of a
    particular subroutine.

    Here's the relevant section of code. The problem is the binary flip-flop
    operator and the search pattern. For some reason, the search patterns are
    not picking up the 'sub _Parse {' pattern seen in the next function.
    What's more strange is that if I create a separate program with the while
    loop shown in _DBLoad, I do in fact pick up the beginning and end of the
    function.

    Any suggestions?

    Thanks,

    -Clint

    sub _DBLoad {
    {
    no strict 'refs';

    exists(${__PACKAGE__.'::'}{_DBParse})#Already loaded ?
    and return;
    }
    my($fname)=__FILE__;
    my(@drv);
    open(DRV,"<$fname") or die "Report this as a BUG: Cannot open $fname";
    while(<DRV>) {
    /^\s*sub\s+_Parse\s*{\s*$/ .. /^\s*}\s*#\s*_Parse\s*$/
    and do {
    s/^#DBG>//;
    push(@drv,$_);
    }
    }
    close(DRV);

    $drv[0]=~s/_P/_DBP/;
    eval join('',@drv);
    }

    #Note that for loading debugging version of the driver,
    #this file will be parsed from 'sub _Parse' up to '}#_Parse' inclusive.
    #So, DO NOT remove comment at end of sub !!!
    sub _Parse {
    my($self)=shift;

    my($rules,$states,$lex,$error)
    = @$self{ 'RULES', 'STATES', 'LEX', 'ERROR' };
    my($errstatus,$nberror,$token,$value,$stack,$check,$dotpos)
    = @$self{ 'ERRST', 'NBERR', 'TOKEN', 'VALUE', 'STACK', 'CHECK', 'DOTPOS' };

    #DBG> my($debug)=$$self{DEBUG};
    #DBG> my($dbgerror)=0;

    #DBG> my($ShowCurToken) = sub {
    #DBG> my($tok)='>';
    #DBG> for (split('',$$token)) {
    #DBG> $tok.= (ord($_) < 32 or ord($_) > 126)
    #DBG> ? sprintf('<%02X>',ord($_))
    #DBG> : $_;
    #DBG> }
    #DBG> $tok.='<';
    #DBG> };

    $$errstatus=0;
    ....
     
    Clint Olsen, Jan 16, 2004
    #1
    1. Advertising

  2. Clint Olsen

    Anno Siegel Guest

    Clint Olsen <> wrote in comp.lang.perl.misc:
    > I'm using Perl 5.8.2, and I'm seeing some strange behavior from the pattern
    > matching engine.
    >
    > Parse::Yapp is a module written to generate Perl in a yacc-like fashion.
    > However, I can't get the debugging version to work because it's not
    > correctly parsing between the markers in the file.
    >
    > In debug mode, the module calls and modifies itself, uncommenting debug
    > blocks and then 'eval'-ing the string to produce a debug version of a
    > particular subroutine.
    >
    > Here's the relevant section of code. The problem is the binary flip-flop
    > operator and the search pattern. For some reason, the search patterns are
    > not picking up the 'sub _Parse {' pattern seen in the next function.


    This expression

    /^\s*sub\s+_Parse\s*{\s*$/ .. /^\s*}\s*#\s*_Parse\s*$/

    doesn't look for "sub _Parse {" in the next function, and it would never
    find it unless the source defined _Parse twice. It looks for the
    final "}#_Parse" that by convention ends that routine.

    Or have I entirely misunderstood your question?

    > What's more strange is that if I create a separate program with the while
    > loop shown in _DBLoad, I do in fact pick up the beginning and end of the
    > function.


    Is your code below an example of where it fails, or is it the self-
    created program that works as expected? In any case, it is incomplete
    because the crucial end of the _Parse routine isn't there. I've left
    it in for reference, but as an aid to answering your question it doesn't
    seem to help much.

    Anno


    > sub _DBLoad {
    > {
    > no strict 'refs';
    >
    > exists(${__PACKAGE__.'::'}{_DBParse})#Already loaded ?
    > and return;
    > }
    > my($fname)=__FILE__;
    > my(@drv);
    > open(DRV,"<$fname") or die "Report this as a BUG: Cannot open $fname";
    > while(<DRV>) {
    > /^\s*sub\s+_Parse\s*{\s*$/ .. /^\s*}\s*#\s*_Parse\s*$/
    > and do {
    > s/^#DBG>//;
    > push(@drv,$_);
    > }
    > }
    > close(DRV);
    >
    > $drv[0]=~s/_P/_DBP/;
    > eval join('',@drv);
    > }
    >
    > #Note that for loading debugging version of the driver,
    > #this file will be parsed from 'sub _Parse' up to '}#_Parse' inclusive.
    > #So, DO NOT remove comment at end of sub !!!
    > sub _Parse {
    > my($self)=shift;
    >
    > my($rules,$states,$lex,$error)
    > = @$self{ 'RULES', 'STATES', 'LEX', 'ERROR' };
    > my($errstatus,$nberror,$token,$value,$stack,$check,$dotpos)
    > = @$self{ 'ERRST', 'NBERR', 'TOKEN', 'VALUE', 'STACK', 'CHECK', 'DOTPOS' };
    >
    > #DBG> my($debug)=$$self{DEBUG};
    > #DBG> my($dbgerror)=0;
    >
    > #DBG> my($ShowCurToken) = sub {
    > #DBG> my($tok)='>';
    > #DBG> for (split('',$$token)) {
    > #DBG> $tok.= (ord($_) < 32 or ord($_) > 126)
    > #DBG> ? sprintf('<%02X>',ord($_))
    > #DBG> : $_;
    > #DBG> }
    > #DBG> $tok.='<';
    > #DBG> };
    >
    > $$errstatus=0;
    > ...
     
    Anno Siegel, Jan 19, 2004
    #2
    1. Advertising

  3. Clint Olsen

    Clint Olsen Guest

    On 2004-01-19, Anno Siegel <-berlin.de> wrote:
    >
    > This expression
    >
    > /^\s*sub\s+_Parse\s*{\s*$/ .. /^\s*}\s*#\s*_Parse\s*$/
    >
    > doesn't look for "sub _Parse {" in the next function, and it would never
    > find it unless the source defined _Parse twice. It looks for the
    > final "}#_Parse" that by convention ends that routine.


    My understanding of this operator is that it is true once the left hand
    side is true and remains true until it finds the left hand pattern. So, in
    essence it grabs the entire contents between these two markers. Yes, I did
    leave off the trailing marker for brevity reasons. Otherwise the sub was
    too long for posting.

    The problem is, the first pattern is never matched. But, when I create a
    separate program and run it on this file, it _does_ match. So, I think
    there's likely a bug lurking around somewhere in Perl.

    > Is your code below an example of where it fails, or is it the self-
    > created program that works as expected? In any case, it is incomplete
    > because the crucial end of the _Parse routine isn't there. I've left it
    > in for reference, but as an aid to answering your question it doesn't
    > seem to help much.


    As it stands, DBLoad is a sub which is contained within the package that
    will be parsed by that pattern shown above. It grabs the sub in between
    the two markers and strips out the #DBG> comments and then 'evals' the code
    on the fly to create a new routine.

    -Clint
     
    Clint Olsen, Jan 19, 2004
    #3
  4. Clint Olsen

    Anno Siegel Guest

    Clint Olsen <> wrote in comp.lang.perl.misc:
    > On 2004-01-19, Anno Siegel <-berlin.de> wrote:
    > >
    > > This expression
    > >
    > > /^\s*sub\s+_Parse\s*{\s*$/ .. /^\s*}\s*#\s*_Parse\s*$/
    > >
    > > doesn't look for "sub _Parse {" in the next function, and it would never
    > > find it unless the source defined _Parse twice. It looks for the
    > > final "}#_Parse" that by convention ends that routine.

    >
    > My understanding of this operator is that it is true once the left hand
    > side is true and remains true until it finds the left hand pattern. So, in

    ^^^^
    right

    > essence it grabs the entire contents between these two markers.


    Yes, after s/left/right/, that's what it does.

    > Yes, I did
    > leave off the trailing marker for brevity reasons. Otherwise the sub was
    > too long for posting.


    You should have left the final "}#_parse" in place, omitting other parts.

    > The problem is, the first pattern is never matched. But, when I create a
    > separate program and run it on this file, it _does_ match. So, I think
    > there's likely a bug lurking around somewhere in Perl.


    So how can we reproduce the bug, if there is one? Give us a recipe
    how to use Parse::Yapp so that the error occurs.

    Anno
     
    Anno Siegel, Jan 20, 2004
    #4
  5. Clint Olsen

    Clint Olsen Guest

    On 2004-01-20, Anno Siegel <-berlin.de> wrote:
    >
    > So how can we reproduce the bug, if there is one? Give us a recipe how
    > to use Parse::Yapp so that the error occurs.


    That's a good question.

    I just tried the example Calc.yp that came with it, and debug mode appears
    to be working. So, it's specific to my module which has sensitive data in
    it (can't be distributed). However, it compiles cleanly under 'strict' and
    'warning', so I'm not sure how I could isolate the problem. As usual with
    Perl, I guess I'll have to do the traditional 'binary search' approach
    where I gut halves of my code until I can see what's causing it.

    Having said that, can you envision why a simple parse like that would fail?

    Thanks,

    -Clint
     
    Clint Olsen, Jan 23, 2004
    #5
  6. Clint Olsen

    Clint Olsen Guest

    On 2004-01-24, pkent <> wrote:
    >
    > Try peppering your code/tokenizer/whatever with some debugging print()
    > statements, and printing out the location and any relevant variable
    > values. That might help, as does the debugging mode. Also are you using
    > the latest versions of perl, Parse::Yapp, etc?


    The problem has something to do with my code, but since the sub is just
    trying to parse its own file (the generated .pm file), it's not as simple
    as embedding print statements.

    I am using Perl 5.8.X and the latest Yapp.

    > Try putting all this so-sensitive data at the top of the module rather
    > than scattered throughout the code, if it is. That way you can easily
    > change the values to something inoffensive if you do need to show other
    > people the code if you still need help.


    Yeah, I need to tear out my stuff and put them in separate package files.

    Thanks,

    -Clint
     
    Clint Olsen, Feb 9, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Wenjie

    if (f() != FAIL) or if (FAIL != f())?

    Wenjie, Jul 28, 2003, in forum: C Programming
    Replies:
    3
    Views:
    457
    E. Robert Tisdale
    Jul 31, 2003
  2. Keithb
    Replies:
    1
    Views:
    427
    Mark Fitzpatrick
    Oct 30, 2006
  3. Clint Olsen

    Performance patch to Parse-Yapp 1.05

    Clint Olsen, Feb 9, 2004, in forum: Perl Misc
    Replies:
    2
    Views:
    87
    Clint Olsen
    Feb 11, 2004
  4. Prasad  Gadgil
    Replies:
    1
    Views:
    115
    Sherm Pendley
    Jul 25, 2005
  5. Casiano
    Replies:
    0
    Views:
    109
    Casiano
    Jan 30, 2007
Loading...

Share This Page