Odd regex behavior

[Mintcake]

I wouldd be grateful to anyone who can shed some light on the
unexpected
results from the regex in the following program.

#!/usr/local/bin/perl -l

use strict;

my $y = ' href="/foo/bar?d=1&c=2&f=1&cards=1" x="123"';

for ($y =~ /(\s+\w+=['"](.*?)["'])/gs)
{
print "1) $_";
print "2) [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "3) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "4) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "5) [$x] [$1][$2]";

print "";
}
__END__

The results I get are as follows

1) href="/foo/bar?d=1&c=2&f=1&cards=1"
2) [ x="123"][123]
3) [1] [href][/foo/bar?d=1&c=2&f=1&cards=1]
4) [1] [href][/foo/bar?d=1&c=2&f=1&cards=1]
5) [1] [href][/foo/bar?d=1&c=2&f=1&cards=1]

1) /foo/bar?d=1&c=2&f=1&cards=1
2) [href][/foo/bar?d=1&c=2&f=1&cards=1]
3) [] [href][/foo/bar?d=1&c=2&f=1&cards=1]
4) [] [href][/foo/bar?d=1&c=2&f=1&cards=1]
5) [] [=2&f=][]

1) x="123"
2) [=2&f=][]
3) [1] [x][123]
4) [1] [x][123]
5) [1] [x][123]

1) 123
2) [x][123]
3) [] [x][123]
4) [] [x][123]
5) [] [x][123]

Now I accept that this code is sloppy for several reasons but in my
defence I have
to say that it is not my code.

1. A while loop would probably be better than a foreach loop

2. The first regex is attempting to break the string in a list of
att="value" type
strings but is returning att="value" and "value" so the .*? should not
be parenthesized

3. No attempt is made to ensure that the same type of quote is used at
the start and
end of the value

The thing I cannot explain are the results from the second iteration
of the loop. The
same regex is executed three times and each time it fails (correctly),
however, the third
time the $1 and $2 values are overwritten. I have always believed
that the $digit variable
would be preserved if the regex failed to match. Reading the Camel
indicates that this
should indeed be the case.

No matter how many times the regex is executed within the loop it is
only on the final one
$1 and $2 are overwritten

 

[Ben Bullock]

I wouldd be grateful to anyone who can shed some light on the
unexpected
results from the regex in the following program.

#!/usr/local/bin/perl -l

use strict;

Adding the line

use warnings;

to your script gives the answer to your problem.

 

[Paul Lalli]

I wouldd be grateful to anyone who can shed some light on the
unexpected
results from the regex in the following program.

#!/usr/local/bin/perl -l

use strict;

Why are you asking people for help before asking Perl for help? Why
haven't you enabled warnings?

my $y = ' href="/foo/bar?d=1&c=2&f=1&cards=1" x="123"';

for ($y =~ /(\s+\w+=['"](.*?)["'])/gs)
{
print "1) $_";
print "2) [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "3) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "4) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "5) [$x] [$1][$2]";

print "";}

__END__

Now I accept that this code is sloppy for several reasons but in my
defence I have to say that it is not my code.

1. A while loop would probably be better than a foreach loop

No, not probably. Definitely. They do not do the same thing at all
in this case, because m//g has very different meanings when evaluated
in a list vs a scalar context.

The thing I cannot explain are the results from the second
iteration of the loop. The same regex is executed three times

No it's not. It's only executed once, because you evaluated it in a
list context and then iterated over the results of that one
evaluation, rather than iterating it repeatedly (and progressively) in
a scalar context.

Paul Lalli

 

[Paul Lalli]

I wouldd be grateful to anyone who can shed some light on the
unexpected results from the regex in the following program.
my $y = ' href="/foo/bar?d=1&c=2&f=1&cards=1" x="123"';

for ($y =~ /(\s+\w+=['"](.*?)["'])/gs)
{
print "1) $_";
print "2) [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "3) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "4) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "5) [$x] [$1][$2]";

print "";}

My profuse apologies. I completely misparsed what your post was
getting at, and came back with a completely wrong answer. Having run
your code, I am also confused as to what's happening. How is $1 being
set to '=2&f=' and how is $2 being undefined, especially seeing as how
as you said, the pattern match is failing. I'm going to keep staring
at it, but I look forward to other responses to this thread. . .

Paul Lalli

 

[Greg Bacon]

Looks like you've found a bug. Please file a report!

Greg

 

[demerphq]

Adding the line

use warnings;

to your script gives the answer to your problem.

No, warnings have nothing to do with this.

Yves

 

[demerphq]

I wouldd be grateful to anyone who can shed some light on the
unexpected
results from the regex in the following program.

#!/usr/local/bin/perl -l

use strict;

my $y = ' href="/foo/bar?d=1&c=2&f=1&cards=1" x="123"';

for ($y =~ /(\s+\w+=['"](.*?)["'])/gs)
{
print "1) $_";
print "2) [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "3) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "4) [$x] [$1][$2]";

my $x = /(\w+)=['"](.*)["']/;
print "5) [$x] [$1][$2]";

print "";}

__END__

The results I get are as follows

1) href="/foo/bar?d=1&c=2&f=1&cards=1"
2) [ x="123"][123]
3) [1] [href][/foo/bar?d=1&c=2&f=1&cards=1]
4) [1] [href][/foo/bar?d=1&c=2&f=1&cards=1]
5) [1] [href][/foo/bar?d=1&c=2&f=1&cards=1]

1) /foo/bar?d=1&c=2&f=1&cards=1
2) [href][/foo/bar?d=1&c=2&f=1&cards=1]
3) [] [href][/foo/bar?d=1&c=2&f=1&cards=1]
4) [] [href][/foo/bar?d=1&c=2&f=1&cards=1]
5) [] [=2&f=][]

This is a bug for sure. Notice that '=2&f=' is the same length as
'cards'. How it ends up at that offset im not sure and I havent
debugged it to see whats up.

The good news is that I already fixed this for 5.10, although its hard
to say which fix was responsible, there were a number related to
capturing and rollbacks and the like done in the 5.9.x line.

The bad news is that the patch is highly unlikely to be back ported to
5.8.x :-(

Interesting bug tho. Cheers.

Yves

 

[Brian McCauley]

I wouldd be grateful to anyone who can shed some light on the
unexpected
results from the regex in the following program.

I suspect that this is pretty much the same issue as was discussed
here recently

http://groups.google.com/group/comp.lang.perl.misc/msg/d128a5c4d28a917b

Here's a much simpler way to reproduce it

use strict;
use warnings;

'From outside loop' =~ /(.*)/;

for my $pass ( 1, 2 ) {
print "$1\n";
'From later inside loop' =~ /(.*)/;
}
__END__

The above could reasonably be expected to print 'From outside loop'
twice but actually prints 'From later inside loop' the second time.

The work-round is simply to double the {}

use strict;
use warnings;

'From outside loop' =~ /(.*)/;

for my $pass ( 1, 2 ) {{
print "$1\n";
'From later inside loop' =~ /(.*)/;
}}
__END__

I am able to reproduce this in 5.9.5.

 

[Brian McCauley]

I suspect that this is pretty much the same issue as was discussed
here recently

Correction - if it wasn't for that issue you probably would not have
been able to observe the bug.

There is, of course, as Yves points out a much more serious bug here
too.

 

[Brian McCauley]

I'm a little unsure of the logic. In your loop, you do a regex behind
the print $1.

Yes, that's the whole point.

Wouldn't you expect the result from the last regex?

No I'd expect the result from the last regex excluding those from
dynamic scopes that have now ended. On the second iteration of the
loop the dynamic scope from the first iteration has ended so I should
not see the result of the regex.

If regex finally has "scope", you should expect garbage or unreliable results
in the first pass.

No, it is defined that if there has been no successful regex match in
the current dynamic scope then the parent dynamic scope is examined.
This is usual for dynamic scopes.

The for { } is scope, the second pass prints the inside.

Yes, this is the bug I'm reporting.

Probably, the $_ should clear the $n variables though, can't remember if it
does.

$_ is not involved anywhere in my example.

I didn't try your code.

I did.