Well, that's the most obscure Perl bug I've ever seen

[pete]

I was fixing a problem that "just appeared" in some otherwise working
code. There were no syntax errors and the program ran fine - it's just
that the result went from being correct to being uttterly and completely
wrong. A bit of lateral thinking and investigation solved the problem.
However, for your entertainment and education, here's a cut down version
that exhibts the same effect:

$

# set up a counter
$i = 0;

for($j = 0; $j < 10; $j++) {
print "j = $j\n";
$i++;
}
print "j = $j, i=$i\n";

which, when run produces:

root: perl bug.pl
j = 0
j = 1
j = 2
j = 3
j = 4
j = 5
j = 6
j = 7
j = 8
j = 9
j = 10, i=6308506

Yup, it's that solitary "$" all alone in the middle of nowhere.
it only took me about an hour and a half to spot - probably because
I was looking in entirely the wrong place as the real-life code had
a large block of comment lines between the "$" and "$i = 0;"

All I can say is aaaaaaaaaaaaaaaaaah!

 

[Teo]

Dear Pete,

I was fixing a problem that "just appeared" in some otherwise working
code. There were no syntax errors and the program ran fine - it's just
that the result went from being correct to being uttterly and completely
wrong. A bit of lateral thinking and investigation solved the problem.
However, for your entertainment and education, here's a cut down version
that exhibts the same effect:

$

# set up a counter
$i = 0;

for($j = 0; $j < 10; $j++) {
  print "j = $j\n";
  $i++;}

print "j = $j, i=$i\n";

which, when run produces:

root: perl bug.pl
j = 0
j = 1
j = 2
j = 3
j = 4
j = 5
j = 6
j = 7
j = 8
j = 9
j = 10, i=6308506

Yup, it's that solitary "$" all alone in the middle of nowhere.
it only took me about an hour and a half to spot - probably because
I was looking in entirely the wrong place as the real-life code had
a large block of comment lines between the "$" and "$i = 0;"

All I can say is aaaaaaaaaaaaaaaaaah!

Following simple good style guidelines would have spared you a lot of
time:

use warnings;
use strict;

and then it produces:

Global symbol "$i" requires explicit package name at test.pl line 8.
Global symbol "$j" requires explicit package name at test.pl line 9.
Global symbol "$j" requires explicit package name at test.pl line 9.
Global symbol "$j" requires explicit package name at test.pl line 9.
Global symbol "$j" requires explicit package name at test.pl line
10.
Global symbol "$i" requires explicit package name at test.pl line
11.
Global symbol "$j" requires explicit package name at test.pl line
14.
Global symbol "$i" requires explicit package name at test.pl line
14.
Execution of test.pl aborted due to compilation errors.

Fixing the warnings with

my $i;

then causes the error to be discovered:

Scalar found where operator expected at test.pl line 8, near "$
# set up a counter
my $i"
(Missing operator before $i?)
Global symbol "$my" requires explicit package name at test.pl line
8.
syntax error at test.pl line 8, near "$

in any case, if I'm not wrong the program was interpreted as

$$i = 0

And BTW running the script as root is not the best idea

Cheers,

Matteo

 

[ace]

j = 7
j = 8
j = 9
j = 10, i=6308506

Yup, it's that solitary "$" all alone in the middle of nowhere.
it only took me about an hour and a half to spot - probably because
I was looking in entirely the wrong place as the real-life code had
a large block of comment lines between the "$" and "$i = 0;"

It's your own fault actually. Put use strict; on top of your program and
try to reproduce misbehavior.

 

[Frank Seitz]

$

# set up a counter
$i = 0;

$$i = 0;

This statement creates a scalar reference and assigns it to $i.
This mechanism is called autovivification.
A reference is basically an integer which you can increment:

$$i = 0;
print "$i\n";
$i++;
print "$i\n";
__END__
SCALAR(0x94a3168)
155857257

Frank
--
Dipl.-Inform. Frank Seitz
Anwendungen für Ihr Internet und Intranet
Tel: 04103/180301; Fax: -02; Industriestr. 31, 22880 Wedel

Blog: http://www.fseitz.de/blog
XING-Profil: http://www.xing.com/profile/Frank_Seitz2

 

[pete]

It's your own fault actually. Put use strict; on top of your program and
try to reproduce misbehavior.

Not practical - it's not my original code, just my original bug :-(

 

[pete]

$$i = 0;

This statement creates a scalar reference and assigns it to $i.
This mechanism is called autovivification.
A reference is basically an integer which you can increment:

$$i = 0;
print "$i\n";
$i++;
print "$i\n";
__END__
SCALAR(0x94a3168)
155857257

Frank

What surprised me was that perl didn't see the $<extraneous stuff+whitespace># ...
and try to calculate the size of an array named by the comment, thus:

$# set ...
$i = 0;

which would at least have spat out a syntax error

 

[Frank Seitz]

No. A reference can be numified, and will be if you apply a numeric
operator like ++ to it.

Yes, this is what I meant. The conversion makes sense because
a reference is basically a memory address (plus type),
and a memory address is an integer.

The result is not a reference, so you can't use
this to go grubbing around in arbitrary bits of memory.

I did not say that I can do pointer arithmetic on the result.
We talk about Perl, not C

Frank
--
Dipl.-Inform. Frank Seitz
Anwendungen für Ihr Internet und Intranet
Tel: 04103/180301; Fax: -02; Industriestr. 31, 22880 Wedel

Blog: http://www.fseitz.de/blog
XING-Profil: http://www.xing.com/profile/Frank_Seitz2

 

[Frank Seitz]

What surprised me was that perl didn't see the $<extraneous stuff+whitespace># ...
and try to calculate the size of an array named by the comment, thus:

$# set ...
$i = 0;

which would at least have spat out a syntax error

"# set ..." is a comment and is ignored by Perl.
The rest is syntactically ok, because Perl allows whitespace
between the dereference operator $ and the variable $i.

Frank
--
Dipl.-Inform. Frank Seitz
Anwendungen für Ihr Internet und Intranet
Tel: 04103/180301; Fax: -02; Industriestr. 31, 22880 Wedel

Blog: http://www.fseitz.de/blog
XING-Profil: http://www.xing.com/profile/Frank_Seitz2

 

[sln]

$$i = 0;

This statement creates a scalar reference and assigns it to $i.
This mechanism is called autovivification.

But, a scalar dereference can't be declared as in:
my $$i = 0;

So, from a declaration standpoint, this
$$i = 0
is actually this
my $i = \0;
a reference to a read-only constant.

A reference is basically an integer which you can increment:

A reference is a pseudo pointer to other data (of any type).
The reference count increments as below.

$$i = \6;
print "$i\n";
print "$$i\n";
print "$$$i\n";

print "\n";

my $B = \\\\9;
print "$B\n";
print "$$B\n";
print "$$$B\n";
print "$$$$B\n";
print "$$$$$B\n";
__END__

REF(0x22ab94)
SCALAR(0x22ac44)
6

REF(0x182a9bc)
REF(0x182a99c)
REF(0x182a98c)
SCALAR(0x182a92c)
9

-sln

 

[Frank Seitz]

You miss pete's point, which was that the "#" was part of "$#...".

"$#set" is the last index of the @set array.

"$# set" is a syntax error.

I talked about the original code. In that code is whitespace
between the $ and the #, i.e. # starts a comment.

Frank
--
Dipl.-Inform. Frank Seitz
Anwendungen für Ihr Internet und Intranet
Tel: 04103/180301; Fax: -02; Industriestr. 31, 22880 Wedel

Blog: http://www.fseitz.de/blog
XING-Profil: http://www.xing.com/profile/Frank_Seitz2

 

[jl_post]

Not practical - it's not my original code, just my original bug :-(

I once inherited several Perl scripts with about one thousand lines
of code in each one. Unfortunately, none of those scripts had "use
strict" and "use warnings" in them.

Originally, I decided it was not practical to add "use strict" and
"use warnings" to the top of the scripts (although I did include them
in the blocks I added). Eventually, maintaining the scripts became
such a huge hassle that I "bit the bullet" and added "use strict" and
"use warnings" to the top of the scripts (and modified the code enough
to comply).

It wasn't easy adding "use strict" and "use warnings" to the files,
but now maintaining the scripts is about ten times easier.

So I recommend adding "use strict" and "use warnings" to the script
you're maintaining, even if it takes you several hours to do so. But
if you can't do that, I still recommend adding them in new blocks you
write (like inside loops, subroutines, and if-blocks). That's not
normally done, but I find it's much better than just never using
"strict" and "warnings" altogether.

-- Jean-Luc

 

[Uri Guttman]

FS> Yes, this is what I meant. The conversion makes sense because
FS> a reference is basically a memory address (plus type),
FS> and a memory address is an integer.

to be clear, a ref will return a number (the address) in a numeric
context or the TYPE(0xfdddd) style in a string context. it is not a
number but a multivalued thing that returns different things in
different contexts. it is a ref, number AND string all in the same
scalar if you want to see it that way. but once you modify it with ++ it
becomes only a number thereafter.

uri

 

[Frank Seitz]

FS> Yes, this is what I meant. The conversion makes sense because
FS> a reference is basically a memory address (plus type),
FS> and a memory address is an integer.

to be clear, a ref will return a number (the address) in a numeric
context or the TYPE(0xfdddd) style in a string context. it is not a
number but a multivalued thing that returns different things in
different contexts. it is a ref, number AND string all in the same
scalar if you want to see it that way. but once you modify it with ++ it
becomes only a number thereafter.

Very good explanation, except that I would not say that
a reference "returns" something, because it is a (passive) data structure.

Frank
--
Dipl.-Inform. Frank Seitz
Anwendungen für Ihr Internet und Intranet
Tel: 04103/180301; Fax: -02; Industriestr. 31, 22880 Wedel

Blog: http://www.fseitz.de/blog
XING-Profil: http://www.xing.com/profile/Frank_Seitz2

 

[Uri Guttman]

FS> Yes, this is what I meant. The conversion makes sense because
FS> a reference is basically a memory address (plus type),
FS> and a memory address is an integer.
FS> Very good explanation, except that I would not say that
FS> a reference "returns" something, because it is a (passive) data structure.

hard to find a better word for what a value is in different
contexts. returns seems to work even though it isn't a sub. you can say
the same for a sub that checks wantarray or other things that 'return'
different values in different contexts (e.g. arrays). how would you say
this?

arrays return their size in scalar context and a list of their elements
in list context.

uri

 

[Willem]

Ben Morrow wrote:
) There's no such thing in Perl. Evaluating a ref in numeric context
) numifies it. It's the same as
)
) float a;
) int b = 3;
) a = b;
)
) in C, which is actually an implicit function call (though Perl caches
) the result, which C doesn't).

How do you know it doesn't ? The optimizer is allowed to do
anything it pleases as long as the end effect is the same.
Caching results of function calls that are known not to have
side effects is one very common optimization.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

 

[Steve C]

Not practical - it's not my original code, just my original bug :-(

Not a very good excuse. You don't even have to change the code:

perl pete
j = 0
j = 1
j = 2
j = 3
j = 4
j = 5
j = 6
j = 7
j = 8
j = 9
j = 10, i=147926066

perl -w -Mstrict pete
Global symbol "$i" requires explicit package name at pete line 4.
Global symbol "$j" requires explicit package name at pete line 6.
Global symbol "$j" requires explicit package name at pete line 6.
Global symbol "$j" requires explicit package name at pete line 6.
Global symbol "$j" requires explicit package name at pete line 7.
Global symbol "$i" requires explicit package name at pete line 8.
Global symbol "$j" requires explicit package name at pete line 10.
Global symbol "$i" requires explicit package name at pete line 10.
Execution of pete aborted due to compilation errors.

 

[Ilya Zakharevich]

I agree with the surprise...
^^^^^^^^^^

No. I do not see whitespace there. What I see is "whitespace
interspersed with comments"; this is a different syntaxical element.
^^^^^^^^^^^^^^^^^^

How does one know? One cannot...

We can ask perl what it sees:

perl -MO=Deparse -e '
$
# a comment
$i = 0;
print $i;
'

which gives

$$i = 0;
print $i;
-e syntax OK

You can ask a particular COMPILE of a particular PORT of a particular
VERSION of perl. Given that the observed behaviour is obviously a
bug, you cannot expect that any other guy would reproduce anything
similar... Experimentation has very little value as a path to a
definitive answer.

The only recourse is looking for documentation, but as we all know,
Perl is practically undocumented...

===

Let me sum up: IMO, the bug is in Perl, and not in the script. The
particular construct should not allow interspersed comments.

Hope this helps,
Ilya

 

[sreservoir]

I agree with the surprise...

^^^^^^^^^^

No. I do not see whitespace there. What I see is "whitespace
interspersed with comments"; this is a different syntaxical element.

^^^^^^^^^^^^^^^^^^

How does one know? One cannot...


You can ask a particular COMPILE of a particular PORT of a particular
VERSION of perl. Given that the observed behaviour is obviously a
bug, you cannot expect that any other guy would reproduce anything
similar... Experimentation has very little value as a path to a
definitive answer.

The only recourse is looking for documentation, but as we all know,
Perl is practically undocumented...

===

Let me sum up: IMO, the bug is in Perl, and not in the script. The
particular construct should not allow interspersed comments.

The particular construct really shouldn't allow all sorts of things it
does.

 

[Kyle T. Jones]

It wasn't easy adding "use strict" and "use warnings" to the files,
but now maintaining the scripts is about ten times easier.

Why wasn't it easy? Surely you didn't go through and do it manually?
Uhhh, is this that "irony" thing my friends are always going on about?

Cheers.

 

[pete]

So, spending an hour and a half chasing down a bug that Perl could have
found for you in less than a second *is* practical?

I think we define that word differently...

Nooo, we define the word the asme. The difference is when you add something
to a few hundred lines of old Perl and then have to spend the rest of the
day nailing each particular non-strict complaint in turn. Then hoping I
don't break something else along the way. Then retesting all the stuff I've
changed instead of just testing the stuff I've added. It could easily turn
a half-day job into a week of work - if done properly, on hourly rates