regular expression variables under debugger

[wlcna]

I have a program I was running through the debugger that had a usage
like this (this is not the code but similar):

1: my $str = "hello there yes i am here";
2: $str =~ /([a-z]*)s/;
3: $yes = $1;

So, I'm running this in the debugger to check things out, make sure it's
working as I expect, and the debugger when I get to line 3 tells me that
$1 is "undef", and past line 3 it tells me that $yes did not receive any
value, it's "undef" as well.

I run this same program *NOT IN THE DEBUGGER* and everything is normal,
and I check by doing old-fashioned print statements. $yes is "yes" as
expected.

I've noticed problems before with viewing regular expression variables
in teh debugger.

I now assume that regular expressions and the debugger just don't mix
but prior to tonight I NEVER IMAGINED that something like my $yes
variable above would also not show the correct value in the debugger. I
had assumed the special regex variables scope meant if I tried to print
them directly from the debugger prompt, (i.e. p $1 or p $&) that
wouldn't work, but of course I thought running the program from inside
the debugger and watching its variables get set should still work
properly.

But now I don't even think that works. So what am I missing?

Thanks for any info!

 

[Dr.Ruud]

wlcna schreef:

I have a program I was running through the debugger that had a usage
like this (this is not the code but similar):

1: my $str = "hello there yes i am here";
2: $str =~ /([a-z]*)s/;
3: $yes = $1;

So, I'm running this in the debugger to check things out, make sure
it's working as I expect, and the debugger when I get to line 3 tells
me that $1 is "undef", and past line 3 it tells me that $yes did not
receive any value, it's "undef" as well.

I run this same program *NOT IN THE DEBUGGER* and everything is
normal, and I check by doing old-fashioned print statements. $yes is
"yes" as expected.

That is strange, it should be "ye".

perl -le '
$_ = "hello there yes i am here" ;
/([a-z]*)s/ ;
print $1
'
prints: ye


For me it works (with -d) as you expect.
Did you also try to add watch expressions with -w for $str and $1?

 

[Paul Lalli]

I have a program I was running through the debugger that had a usage
like this (this is not the code but similar):

Why don't you just post the *real* code? Create a short-but-complete
script that demonstrates the problem you're seeing. That should be the
first step to debugging any problem.

1: my $str = "hello there yes i am here";
2: $str =~ /([a-z]*)s/;
3: $yes = $1;

So, I'm running this in the debugger to check things out, make sure it's
working as I expect, and the debugger when I get to line 3 tells me that
$1 is "undef", and past line 3 it tells me that $yes did not receive any
value, it's "undef" as well.

I cannot duplicate your problem:

$ cat clpm.pl
my $str = "hello there yes i am here";
$str =~ /([a-z]*)s/;
$yes = $1;

$ perl -d clpm.pl

Loading DB routines from perl5db.pl version 1.27
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main:clpm.pl:1): my $str = "hello there yes i am here";
DB<1> n
main:clpm.pl:2): $str =~ /([a-z]*)s/;
DB<1> n
main:clpm.pl:3): $yes = $1;
DB<1> p $1
ye
DB<2> p $yes

DB<3> n
Debugged program terminated. Use q to quit or R to restart,
use O inhibit_exit to avoid stopping after program termination,
h q, h R or h O to get additional info.
DB<3> p $yes
ye
DB<4>



Please post a SHORT but COMPLETE script that actually demonstrates your
problem, rather than code that you claim is similar to your code, which
demonstrates that your problem does not exist.

Paul Lalli

 

[Xicheng Jia]

I have a program I was running through the debugger that had a usage
like this (this is not the code but similar):

1: my $str = "hello there yes i am here";
2: $str =~ /([a-z]*)s/;
3: $yes = $1;

So, I'm running this in the debugger to check things out, make sure it's
working as I expect, and the debugger when I get to line 3 tells me that
$1 is "undef", and past line 3 it tells me that $yes did not receive any
value, it's "undef" as well.

I run this same program *NOT IN THE DEBUGGER* and everything is normal,
and I check by doing old-fashioned print statements.
$yes is "yes" as expected.

how come??

I've noticed problems before with viewing regular expression variables
in teh debugger.

I now assume that regular expressions and the debugger just don't mix
but prior to tonight I NEVER IMAGINED that something like my $yes
variable above would also not show the correct value in the debugger. I
had assumed the special regex variables scope meant if I tried to print
them directly from the debugger prompt, (i.e. p $1 or p $&) that
wouldn't work, but of course I thought running the program from inside
the debugger and watching its variables get set should still work
properly.

But now I don't even think that works. So what am I missing?

the result of 'p' is not well-formatted, using 'x' instead of 'p'
command is better:

bash: ~$ perl -d test.pl
........
main:t1.pl:5): my $str = "hello there yes i am here";
DB<1> n
main:t1.pl:6): $str =~ /([a-z]*)s/;
DB<1> n
main:t1.pl:7): $yes = $1;
DB<2> x $1
'ye'
DB<3> x $yes
undef
DB<4> n
Debugged program terminated. Use q to quit or R to restart,
use O inhibit_exit to avoid stopping after program termination,
h q, h R or h O to get additional info.
DB<5> x $1
undef
DB<6> x $yes
'ye'
DB<7> p $yes ye
DB<8> q
____

After copy/paste the line "<DB> p $yes" to a text editor, you'll
probably see the result of 'p $yes' at the rightmost of the same line.

Xicheng

 

[wlcna]

main:t1.pl:5): my $str = "hello there yes i am here";
DB<1> n
main:t1.pl:6): $str =~ /([a-z]*)s/;
DB<1> n
main:t1.pl:7): $yes = $1;
DB<2> x $1
'ye'
DB<3> x $yes
undef
DB<4> n
Debugged program terminated. Use q to quit or R to restart,

Well, you (and the other who said this shouldn't be happening) are right
about the example I provided - it does not show the behavior I'm
describing though it is essentially the same code where I do see the
issue!! As I stated I just wrote the code to illustrate what I'm doing
and what I'm seeing in the debugger - also yes there was a little "bug"
in there that the $yes variable was getting set to "ye" rather than
"yes" as intended, no biggie for the issue though.

Frankly, I have no idea what's going on and I've tried a little to track
it down so I could repost something you guys could see too but it's true
that in the example I posted I can clearly see $1 and $& even in the
debugger, everything looks perfect whereas in the code I'm running
that's not the case. In the code I'm running too I've tested the regex
I'm using via copy/paste inside the bugger, evaluating something like
$str =~ /([a-z]*)s/; print $1; and the $1 comes through in that
scenario.

I don't foreclose the possibility that there might be something here I'm
just clueless about.

But I can tell you that this issue of not being able to see regular
expression status in the debugger is not new to me, I had perceived this
as an issue before this post, though I made this post because I felt not
only was I not seeing the regular expression stuff, it seemed like
running my program under the debugger was also changing how my program
ran! I'm not sure of that right now though.

Anyway, I still don't have a simple example that shows the problem but I
still see it happening in my full version, I'll try to repost an example
that is definitely reproducible for me.

FYI, these are the libraries I'm using at the top of my file:

use LWP::UserAgent;
use XML::TreeBuilder;
use Mail::Sender;

I'll hopefully post back with some better info, sorry I don't have some
code that clearly shows the problem, but I wanted to at least reply
now...

 

[Tad McClellan]

main:t1.pl:5): my $str = "hello there yes i am here";
DB<1> n
main:t1.pl:6): $str =~ /([a-z]*)s/;
DB<1> n
main:t1.pl:7): $yes = $1;
DB<2> x $1

Well, you (and the other who said this shouldn't be happening) are right
about the example I provided - it does not show the behavior I'm
describing though it is essentially the same code where I do see the ^^^^^^^^^^^^^^^^^^^^
issue!!

Then the problem is probably in the part that made you use
the "essentially" qualifier...

I'm using via copy/paste inside the bugger, evaluating something like
$str =~ /([a-z]*)s/; print $1; and the $1 comes through in that
scenario.

I don't foreclose the possibility that there might be something here I'm
just clueless about.

I have a guess that might explain what you see.

You have now _twice_ posted code that uses a dollar-digit variable
without first ensuring that the intended pattern match _succeeded_.

If the match you intended to set $1 fails, then the value you see
for $1 is some old stale value from a previous match that _did_
succeed (or undef if no successful matches).

 

[wlcna]

Well, you (and the other who said this shouldn't be happening) are
right about the example I provided - it does not show the behavior I'm
describing though it is essentially the same code where I do see the

OK, I have some new and improved information here.

I've confirmed that the behavior IS DIFFERENT under the debugger versus
not under the debugger as I originally stated. I.e. a Heisenberg
problem! The debugger is causing the problem that I am viewing!

I confirmed this via print statements again. I have a "sub" that gets
an argument that comes from the parsing of this regular expression (and
from the $1 result of such parsing as shown in the sample code) and
under the debugger, this sub bombs the program via a "die" statement b/c
I added a check of the argument to make sure it's a value and not undef.
But if I run the program not under the debugger, the print statement
shows the proper number and runs along without any weirdness.

Also, I took the suggestion of one of the guys out here and put a watch
on $1, and I noticed a bunch of utf8 stuff happening when the line of
regex code in question was evaluated. This utf8 stuff was shown b/c
apparently $1 was getting modified, modified to "#" I have no idea why
and then modified to undef, again no idea there either. There was no
"#" anywhere in the string being regexed.

And when I run this in the tester program I posted, there is no such
utf8 weirdness. Additionally, I also tried putting identical values and
identical regex (via copy and paste) into that tester program, no utf8
weirdness occurs. I think I'm onto some weirdness there eh?

As I say the code in question is virtually identical to what I posted,
the only difference is the libraries etc. being loaded and used it seems
to me, obviously there's more going on in the other code.

But here's the actual line of code that triggers this utf thing:

$str =~ /([0-9]*)\.html/;

And here's what this utf weirdness looks like in the debugger:

main:MYPROGRAMNAMEHERE:102): $str =~
/([0-9]*)\.html/;
DB<4>
Watchpoint 0: $1 changed:
old value: ''
new value: '#'
utf8::SWASHNEW(/usr/local/perl/lib/5.8.2/utf8_heavy.pl:203):
203: my $char = $1;
DB<4>

The "$str" is of the form: http://asite.aplace.org/adir/22221.html

Anyone have any opinions? I'm now thinking it has something to do with
these HTML/XML parsing libraries for some reason. As I say, I use these
alot and I have always had issues with reading these regular expression
variables while in debug mode.

I think I've got a theory: The characters coming back from this web
page via the XML:arser library are in UTF8 and this is some weirdness
connected to that.

What do people think? Why does this weirdness only happen under the
debugger?

 

[wlcna]

For me it works (with -d) as you expect.
Did you also try to add watch expressions with -w for $str and $1?

So, I thought putting a watch on $1 was a useful suggestion - I took
that suggestion. Please see my response in the other part of the
thread!

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
wlcna

I've confirmed that the behavior IS DIFFERENT under the debugger versus
not under the debugger as I originally stated. I.e. a Heisenberg
problem! The debugger is causing the problem that I am viewing!

No wonder. What you see is a bug; running under debugger causes
different code paths, so different bugs show...

the most crucial info is your perl -V. Why did not you post it yet?

If you are not running the latest released version of Perl, you must
recheck with the newer version. If it is the latest, nothing/nobody
would help you until you can provide a minimal possible script which
demonstrates the same problem

Hope this helps,
Ilya

 

[wlcna]

No wonder. What you see is a bug; running under debugger causes
different code paths, so different bugs show...

the most crucial info is your perl -V. Why did not you post it yet?

5.8.2 is the version.

Re: getting the latest, is there a way to update perl without losing all
the libraries that are installed? Can you give me a tip on dealing with
this issue? I compile my own perl (under Linux)...

 

[Paul Lalli]

I just ran this under the debugger and suspected that the second time
might vary from the first time and I was correct, check out this
weirdness:

DB<2> print "hello" if not $str =~ /([0-9]*)\.html/; print $1;
hello
DB<3> print "hello" if not $str =~ /([0-9]*)\.html/; print $1;
2222

Those are two identical lines of code evaluated in the debugger with the
same input and the second time gives a different result from the first
time. Note the "hello" indicates the problem, the 2222 is the string
that is expecting that SHOULD BE coming back in the $1 and that does
when not in the debugger.

Once again, I cannot duplicate your problem:
$ perl -d

Loading DB routines from perl5db.pl version 1.25
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

$str = "foo2222.html";
main:-:1): $str = "foo2222.html";
DB<1> print "hello" if not $str =~ /([0-9]*)\.html/; print $1

hello
DB<2> print "hello" if not $str =~ /([0-9]*)\.html/; print $1

hello


This is about the third time you've posted results without complete
code? WHY? Why are you making it so difficult to verify what you're
seeing? Why can't you post a *complete* program, including data, that
illustrates the supposed bugs?

Paul Lalli

 

[Henry Law]

Of course, but not really in this case. The offending code is
essentially IDENTICAL to what I posted. But the "inessential" that is
the key is *I think* WHERE THE INPUT IS COMING FROM, see my other post.

Oh for heaven's sake, man: post some runnable code that, when _that_
_exact_ _code_ is run, shows your problem in your environment. If
someone else runs it and gets the problem then they can help debug it;
if they don't then you have an environment or a procedural problem and
we'll help debug it.

 

[Klaus]

The offending code is essentially IDENTICAL to what I posted.

"essentially *identical*" is not the same as "*identical*"

so I'm going to change the code right now to see what it thinks [...]

Good.

To change the code, I suggest you proceed by the following 11 points:

1.) Erase your memory about this problem (in your head, not on your
hard drive)

2.) Make a copy of the original, unchanged program on your hard drive
and load that copy into your favourite text editor

3.) Look at the following two lines of code in your program

2: $str =~ /([a-z]*)s/;
3: $yes = $1;

you will notice that you are using $1 without checking whether or not
the match succeeded.

4.) fix that problem:

2: $str =~ /([a-z]*)s/ or die qq{No match in '$str'};
3: $yes = $1;

5.) add some prints to your code

: print '$str = ', unpack('H*', $str), "\n";
: $str =~ /([a-z]*)s/ or die qq{No match in '$str'};
: print '$1 = ', unpack('H*', $1), "\n";
: $yes = $1;

6.) Run the program twice: once with debugger, then without debugger.

7.) If the value in $1 is identical in both runs, with and without
debugger, then we're done and the problem is solved.

8.) If not, then strip-down (strategically remove) 10 - 20 lines of
code from your program, but do not touch the lines in question. (if
your program is very big, then you can delete more lines). Remember:
you are working with a copy of the original program.

9.) Again, run the stripped-down program twice: first with debugger,
then without debugger.

10.) If the value in $1 is identical in both runs, with and without
debugger, then *remember* what you have changed last in step 8.) --
That's what's caused the problem and we're done !

11.) If not, then strip down your program even further and go to step
8.) -- and don't forget: your program is getting smaller and smaller
with every step !

 

[Paul Lalli]

Yeah, you wouldn't be able to that way as I've pointed out. If my hunch
is correct about the problem, I should be able to have some reproducible
code shortly (as soon as I get a chance), but if you're curious now, you
can simply read the other part of the thread where I've detailed what I
think the problem is.

No, sorry, I have no interest in your conjectures. I do not mean to be
rude, but this is now bordering on the absurd - just post the program
that shows the bug!

Paul Lalli

 

[wlcna]

Here's code and debugger executions, this is under perl 5.8.2.

------------------------------------
#!/usr/bin/perl
use strict;

use LWP::UserAgent;
use HTML::TreeBuilder;
use XML::TreeBuilder;

my $strUrl = 'http://rss.news.yahoo.com/rss/us';

# retrieve
my $ua = new LWP::UserAgent;
$ua->timeout( 45 );
my $req = new HTTP::Request GET => $strUrl;
my $res = $ua->request( $req );
my $strHtml = $res->content;

# parse the data retrieved.
my $t = new XML::TreeBuilder;
$t->parse( $strHtml );
$t->eof;

my $str = $t->content->[1]->content->[5]->as_text;
$str =~ /([0-9]*)$/;
my $testPart = $1;
my $testWhole = $&;
my $breakpoint = 3;
print "testPart: $testPart, testWhole: $testWhole\n";


------------------------------------
Here's debugger execution showing the problem:

main:tester:8): my $strUrl = 'http://rss.news.yahoo.com/rss/us';
DB<1> c 22
main:tester:22): my $str =
$t->content->[1]->content->[5]->as_text;
DB<2> n
main:tester:23): $str =~ /([0-9]*)$/;
DB<2> n
main:tester:24): my $testPart = $1;
DB<2> n
main:tester:25): my $testWhole = $&;
DB<2> n
main:tester:26): my $breakpoint = 3;
DB<2> n
main:tester:27): print "testPart: $testPart, testWhole:
$testWhole\n";
DB<2> n
testPart: , testWhole:

------------------------------------
Here's another run showing the problem a different way:

main:tester:8): my $strUrl = 'http://rss.news.yahoo.com/rss/us';
DB<1> c 22
main:tester:22): my $str =
$t->content->[1]->content->[5]->as_text;
DB<2> n
main:tester:23): $str =~ /([0-9]*)$/;
DB<2>
main:tester:24): my $testPart = $1;
DB<2>
main:tester:25): my $testWhole = $&;
DB<2> n
main:tester:26): my $breakpoint = 3;
DB<3> $str =~ /([0-9]*)$/; print $1;
718

 

[wlcna]

Oh for heaven's sake, man: post some runnable code that, when _that_
_exact_ _code_ is run, shows your problem in your environment. If
someone else runs it and gets the problem then they can help debug it;
if they don't then you have an environment or a procedural problem and
we'll help debug it.

It's there, see my other post to your neighbor in this thread.

That's complete runnable code that reproduces the problem, for me of
course, every time, and two sets of debugger executions showing the
problem of the regex failing for no apparent reason.

That run includes all the anomalies I've previously mentioned, including
that the first time through the identical regex does not work, second
time through it does work even though all inputs are the same. That's
what happens for me and the code shows the precisely necessary steps to
reproduce it on my system. I didn't post this sooner BECAUSE I DID NOT
HAVE IT yet.

 

[Uri Guttman]

<snip of useless code for this problem>

w> my $str = $t->content->[1]->content->[5]->as_text;
w> $str =~ /([0-9]*)$/;
w> my $testPart = $1;
w> my $testWhole = $&;

why do you think $& will be different than $1?
why do you think you have a match??

YOU NEVER CHECK THE RESULT OF THE REGEX!!!!

you claim you know the data in $str. WHY DON'T YOU PRINT IT!!! i bet it
has no number string and so you have undefined values in $1 and $&.

w> my $breakpoint = 3;
w> print "testPart: $testPart, testWhole: $testWhole\n";

i see NO OUTPUT from that. it proves nothing since we don't know what
was in $str or what this actually printed.

w> ------------------------------------
w> Here's debugger execution showing the problem:

w> main:tester:8): my $strUrl = 'http://rss.news.yahoo.com/rss/us';
w> DB<1> c 22
w> main:tester:22): my $str =
w> $t->content->[1]->content->[5]->as_text;

where is the rest of the code in the debugger?

w> DB<2> n
w> main:tester:23): $str =~ /([0-9]*)$/;
w> DB<2> n

WHAT IS IN $str???? if you had a clue you would have just printed it
first. then you could have written 5 lines of code that would show (or
not show) the problem. but you have to claim stuff without any data to
prove it.

w> main:tester:24): my $testPart = $1;
w> DB<2> n
w> main:tester:25): my $testWhole = $&;
w> DB<2> n
w> main:tester:26): my $breakpoint = 3;
w> DB<2> n
w> main:tester:27): print "testPart: $testPart, testWhole:
w> $testWhole\n";
w> DB<2> n
w> testPart: , testWhole:

so?? that means YOU DIDN'T MATCH ANYTHING. you NEVER CHECKED FOR MATCH
SUCCESS IN EITHER CASE. YOU DIDN'T PRINT $str BEFORE THE MATCH.

your code is useless to show anything. i never thought you had found a
real problem and now i am positive you haven't. this was a simple issue
that you dragged out for way too long and then you finally show crap
code that proves nothing since you don't print the critical info and
don't do basic logic tests that any perl hacker would do.

uri

 

[wlcna]

<snip of useless code for this problem>

I must say a good number of you perl programmers seem like wusses. I'm
not primarily a perl programmer but just gotta say that. Not all of
you, but a whole bunch.

Dude, why don't you try to run the code before you talk out of your rear
end. This is some kind of bug in perl, I have other things to do than
hand-hold you wusses when I've pressed your faces into the wreaking
problem like 5x. Try to run the code, if you don't see the behavior I
see and you have a more current version, then hopefully it's fixed in
the latest.

First you wusses ask for code in addition to my initial code plus
detailed explanations, I give it to you plus debugger stuff showing the
problem, then you, wuss, tell me that the code doesn't show anything
which you've figured out before even trying it.

Anyway, you haven't read any of the important parts of this thread so I
don't even know why I'm responding to you and I'm certainly not
addressing any of your moronic points that I've addressed like 2 or 3
times already.

Read, wuss.

Run code, wuss.

 

[Uri Guttman]

w> Dude, why don't you try to run the code before you talk out of your rear
w> end. This is some kind of bug in perl, I have other things to do than
w> hand-hold you wusses when I've pressed your faces into the wreaking
w> problem like 5x. Try to run the code, if you don't see the behavior I
w> see and you have a more current version, then hopefully it's fixed in
w> the latest.

w> First you wusses ask for code in addition to my initial code plus
w> detailed explanations, I give it to you plus debugger stuff showing the
w> problem, then you, wuss, tell me that the code doesn't show anything
w> which you've figured out before even trying it.

you didn't do the correct thing in two places. i can't fix that. i have
no idea what was IN THE VARIABLE $str. i am not going to install some
html parser to run this. if you just printed the value you could see the
problem.

w> Anyway, you haven't read any of the important parts of this thread
w> so I don't even know why I'm responding to you and I'm certainly
w> not addressing any of your moronic points that I've addressed like
w> 2 or 3 times already.

you haven't done anything you were asked by me. when you finally posted
real code you didn't reduce it to a minimal script. the lwp and parse
stuff have nothing to do with your supposed bug. so if you just print
that value, set a variable to it, match it AND TEST THE MATCH RESULTS,
then you could show something we call evidence. but you don't seem to
get the train of logic here. you have been left at the station.

i don't run useless code. the onus is on you to prove a bug and you
haven't done diddly to do it nor do you get why you haven't.

enuff said,

uri

 

[Dr.Ruud]

wlcna schreef:

Dr.Ruud:

So, I thought putting a watch on $1 was a useful suggestion

How about the watch on $str?