Problem in nested sorts

D

David McNerney

I get some interesting results when I run the following code on Perl
5.8.5 or 5.8.6:

---- begin code ----
#!/usr/bin/perl -w
use strict;

package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort function 1] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
my @values2 = sort
{
print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
}

package my_package;
my @seqnums = (486098561006,486098561006);
@seqnums = sort
{
&another::testFunction;
return $a <=> $b;
} @seqnums;
-------- end code -----

It seems that the second time I try to do a sort in a function/method
invoked from within another sort, $a and $b are not defined. If I
define a separate sort function with prototypes, the problem goes away.
The problem also goes away if I have everything in the same package
(ie, get rid of the 2 "package" statements and the "another::" in the
call to &testFunction).

Can anyone see something I'm doing wrong here? Needless to say, the
above code is a minimal test case, and I'm aware that sorting
(1,2,3,4,5) and not doing anything with the results would not normally
be worthwhile...
 
B

Brian McCauley

David said:
I get some interesting results when I run the following code on Perl
5.8.5 or 5.8.6:
sub testFunction
{
}

package my_package;
@seqnums = sort
{
&another::testFunction;
return $a <=> $b;
} @seqnums;
It seems that the second time I try to do a sort in a function/method
invoked from within another sort, $a and $b are not defined.

Yes, $another::a and $my_package::a are not the same variable.
If I define a separate sort function with prototypes, the problem goes away.

Yes, that's largely why that option was introduced.
The problem also goes away if I have everything in the same package
(ie, get rid of the 2 "package" statements and the "another::" in the
call to &testFunction).

Yes, $main::a and $main::a are the same variable
Can anyone see something I'm doing wrong here?

Not reading "perldoc -f sort"? (In particular the bits containing the
word "package").
 
D

David McNerney

Yes, $another::a and $my_package::a are not the same variable.

Of course they are not the same variable, but if you re-read my code
you'll note that that isn't relevant here. :)

It sounds like you know Perl well enough. If you read over my code
carefully, I think you'll realize why what you said doesn't relate.

Hint: &testFunction is located entirely in the package "another". It is
doing a simple sort of a locally defined list. During that sort, $a and
$b should be defined, and they are -- but only the first time such a
sort is done. If the call to &testFunction is moved outside the sort
(but still in package "my_package") then both sorts work fine.

Another hint: it's great to try to help folks out on usenet, but try
actually >reading< before getting sarcastic. You thought you saw a
fairly common misunderstanding about the $a and $b vars in perl sorts,
and started typing before reading closely enough to realize that
something else was going on in this case.

Cheers,

Dave
 
A

Anno Siegel

David McNerney said:
I get some interesting results when I run the following code on Perl
5.8.5 or 5.8.6:

---- begin code ----
#!/usr/bin/perl -w
use strict;

package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort function 1] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
my @values2 = sort
{
print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
}

package my_package;
my @seqnums = (486098561006,486098561006);
@seqnums = sort
{
&another::testFunction;
return $a <=> $b;
} @seqnums;
-------- end code -----

It seems that the second time I try to do a sort in a function/method
invoked from within another sort, $a and $b are not defined. If I
define a separate sort function with prototypes, the problem goes away.
The problem also goes away if I have everything in the same package
(ie, get rid of the 2 "package" statements and the "another::" in the
call to &testFunction).

Can anyone see something I'm doing wrong here? Needless to say, the
above code is a minimal test case, and I'm aware that sorting
(1,2,3,4,5) and not doing anything with the results would not normally
be worthwhile...

The result is strange, but there's much optimization going on around
the comparison function and I'm no too amazed the mechanism can break
under stress. Note that the comparison function cannot be recursive.
I suppose the reason is related to what you are seeing.

In any case, if your comparison function contains an expensive call
(as to another sort()), that begs to be factored out of the sort, if
at all possible. See if a Schwartz transform, or the techniques offered
by Sort::Maker apply.

Anno
 
R

robic0

I get some interesting results when I run the following code on Perl
5.8.5 or 5.8.6:

---- begin code ----
#!/usr/bin/perl -w
use strict;

package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort function 1] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
my @values2 = sort
{
print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
}

package my_package;
my @seqnums = (486098561006,486098561006);
@seqnums = sort
{
&another::testFunction;
return $a <=> $b;
} @seqnums;
-------- end code -----

It seems that the second time I try to do a sort in a function/method
invoked from within another sort, $a and $b are not defined. If I
define a separate sort function with prototypes, the problem goes away.
The problem also goes away if I have everything in the same package
(ie, get rid of the 2 "package" statements and the "another::" in the
call to &testFunction).

Can anyone see something I'm doing wrong here? Needless to say, the
above code is a minimal test case, and I'm aware that sorting
(1,2,3,4,5) and not doing anything with the results would not normally
be worthwhile...

Apparently theres no problem with nested sorts within a package as
you've said. Nesting between pkgs seems to fail after the first
sucessfull intra-package nested sort. Even subsequent calls fail.

The $a and $b are confined withing the package as you said as well.

Also, you mentioned
"If I define a separate sort function with prototypes, the problem goes away."

Be aware there is nested sorts are not the same as sort comparison function.
You can define your own "comparison" function that returns true of false, you
can't define a sort function that will have any local affect as in a nested
sort call. Sorts are autonomous.
The form you might be thinking of here is the "sort SUBNAME list".

From the hard to understand docs:
"# using a prototype allows you to use any comparison subroutine
# as a sort subroutine (including other package's subroutines)
package other;
sub backwards ($$) { $_[1] cmp $_[0]; } # $a and $b are not set here
package main;
@new = sort other::backwards @old;
"
Note that $a and $b are not set here as it says. And it only returns t/f
based on the comparison.

This is very different from the behavior you have uncovered.
In general, intermediate $a, $b results are not usefull.

I don't know if "nesting" sorts is good for anything, it might be but
intermediate $a,$b values depend on the method and comparisons.
Nested or complex comparisons are usefull however. For instance
if $a, or $b contain a reference to multifield records where a
multi field sort is needed.

So the bottom like is you probably are just showing a behavior that
deadends in a place that is of no use and designed to fail gracefully
and without documentation. Of course $a and $b generate warnings when
they are not defined (I commented it out below).

Good find!


use strict;
#use warnings;

package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort function 1] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
my @values2 = sort
{
print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
}

package my_package;
my @seqnums = (3,2,1);
@seqnums = sort
{
&another::testFunction;
print "====== a=$a b=$b ================\n";
return $a <=> $b;
} @seqnums;

__END__

[testFunction, sort function 1] a=1 b=2
[testFunction, sort function 1] a=3 b=4
[testFunction, sort function 1] a=1 b=3
[testFunction, sort function 1] a=3 b=2
[testFunction, sort function 1] a=1 b=5
[testFunction, sort function 1] a=5 b=2
[testFunction, sort function 1] a=5 b=3
[testFunction, sort function 1] a=5 b=4
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
====== a=3 b=2 ================
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 1] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
[testFunction, sort function 2] a= b=
====== a=2 b=1 ================


***************************
***************************

use strict;
#use warnings;

#package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort function 1] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
my @values2 = sort
{
print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
}

#package my_package;
my @seqnums = (3,2,1);
@seqnums = sort
{
# &another::testFunction;
&testFunction;
print "====== a=$a b=$b ================\n";
my @values = sort
{
print "[testFunction, sort function 3] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
my @values2 = sort
{
print "[testFunction, sort function 4] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);

return $a <=> $b;
} @seqnums;

__END__

[testFunction, sort function 1] a=1 b=2
[testFunction, sort function 1] a=3 b=4
[testFunction, sort function 1] a=1 b=3
[testFunction, sort function 1] a=3 b=2
[testFunction, sort function 1] a=1 b=5
[testFunction, sort function 1] a=5 b=2
[testFunction, sort function 1] a=5 b=3
[testFunction, sort function 1] a=5 b=4
[testFunction, sort function 2] a=1 b=2
[testFunction, sort function 2] a=3 b=4
[testFunction, sort function 2] a=1 b=3
[testFunction, sort function 2] a=3 b=2
[testFunction, sort function 2] a=1 b=5
[testFunction, sort function 2] a=5 b=2
[testFunction, sort function 2] a=5 b=3
[testFunction, sort function 2] a=5 b=4
====== a=3 b=2 ================
[testFunction, sort function 3] a=1 b=2
[testFunction, sort function 3] a=3 b=4
[testFunction, sort function 3] a=1 b=3
[testFunction, sort function 3] a=3 b=2
[testFunction, sort function 3] a=1 b=5
[testFunction, sort function 3] a=5 b=2
[testFunction, sort function 3] a=5 b=3
[testFunction, sort function 3] a=5 b=4
[testFunction, sort function 4] a=1 b=2
[testFunction, sort function 4] a=3 b=4
[testFunction, sort function 4] a=1 b=3
[testFunction, sort function 4] a=3 b=2
[testFunction, sort function 4] a=1 b=5
[testFunction, sort function 4] a=5 b=2
[testFunction, sort function 4] a=5 b=3
[testFunction, sort function 4] a=5 b=4
[testFunction, sort function 1] a=1 b=2
[testFunction, sort function 1] a=3 b=4
[testFunction, sort function 1] a=1 b=3
[testFunction, sort function 1] a=3 b=2
[testFunction, sort function 1] a=1 b=5
[testFunction, sort function 1] a=5 b=2
[testFunction, sort function 1] a=5 b=3
[testFunction, sort function 1] a=5 b=4
[testFunction, sort function 2] a=1 b=2
[testFunction, sort function 2] a=3 b=4
[testFunction, sort function 2] a=1 b=3
[testFunction, sort function 2] a=3 b=2
[testFunction, sort function 2] a=1 b=5
[testFunction, sort function 2] a=5 b=2
[testFunction, sort function 2] a=5 b=3
[testFunction, sort function 2] a=5 b=4
====== a=2 b=1 ================
[testFunction, sort function 3] a=1 b=2
[testFunction, sort function 3] a=3 b=4
[testFunction, sort function 3] a=1 b=3
[testFunction, sort function 3] a=3 b=2
[testFunction, sort function 3] a=1 b=5
[testFunction, sort function 3] a=5 b=2
[testFunction, sort function 3] a=5 b=3
[testFunction, sort function 3] a=5 b=4
[testFunction, sort function 4] a=1 b=2
[testFunction, sort function 4] a=3 b=4
[testFunction, sort function 4] a=1 b=3
[testFunction, sort function 4] a=3 b=2
[testFunction, sort function 4] a=1 b=5
[testFunction, sort function 4] a=5 b=2
[testFunction, sort function 4] a=5 b=3
[testFunction, sort function 4] a=5 b=4
 
B

Brian McCauley

David said:
You thought you saw a
fairly common misunderstanding about the $a and $b vars in perl sorts,
and started typing before reading closely enough to realize that
something else was going on in this case.

Yes, sorry. May I be clamped in virtual stocks and pelted with virtual
rotten fruit.

I've reproduced your problem and it does look like a particularly nasty
bug in perl.

Actually better pennace for my crime may be to track down the bug and
submit a patch so I'll try that (it may be beyond me).
 
D

David McNerney

Thanks very much to everyone who replied here. Sorry I was a little
terse Brian! I just sent the following to (e-mail address removed):

When an anonymous sort function invokes a method or function defined in
another package, and that
method or function does multiple sorts using its own anonymous sort
functions, the second and later
of these latter sorts fail; $a and $b are not defined. This does not
occur if the function is moved
to the same package as the code performing the first sort, nor does it
occur if the latter "nested"
sorts use a seperately defined sort function with prototypes (in which
case, of course, the $a and $b
variables are not used). And, as implied above, the first "nested" sort
executes correctly.

One person on usenet suggested that nobody ought to be doing such a
thing as described above. Actually,
there are many times when working with a large object-oriented Perl
library when one might need
to sort a (often short) list of objects, and compare values returned by
object methods in the comparison function.
A programmer writing such code should not have to think about whether
the object methods they choose
to invoke happen to use multiple non-prototyped sorts in their
implementation.

Below is a test case, verified to demonstrate the problem on Perl 5.8.5
(Linux) and 5.8.6 (Mac OS X):


#!/usr/bin/perl -w
use strict;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";
}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;
print "\@list, sorted: ".join(", ", @list)."\n";

#&another::testFunction;
 
M

MSG

David said:
Thanks very much to everyone who replied here. Sorry I was a little
terse Brian! I just sent the following to (e-mail address removed):

When an anonymous sort function invokes a method or function defined in
another package, and that
method or function does multiple sorts using its own anonymous sort
functions, the second and later
of these latter sorts fail; $a and $b are not defined. This does not
occur if the function is moved
to the same package as the code performing the first sort, nor does it
occur if the latter "nested"
sorts use a seperately defined sort function with prototypes (in which
case, of course, the $a and $b
variables are not used). And, as implied above, the first "nested" sort
executes correctly.

One person on usenet suggested that nobody ought to be doing such a
thing as described above. Actually,
there are many times when working with a large object-oriented Perl
library when one might need
to sort a (often short) list of objects, and compare values returned by
object methods in the comparison function.
A programmer writing such code should not have to think about whether
the object methods they choose
to invoke happen to use multiple non-prototyped sorts in their
implementation.

Below is a test case, verified to demonstrate the problem on Perl 5.8.5
(Linux) and 5.8.6 (Mac OS X):


#!/usr/bin/perl -w
use strict;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";
}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;
print "\@list, sorted: ".join(", ", @list)."\n";

#&another::testFunction;

Although I agree that you have an rather interesting discovery,
(which may take a real Perl expert to explain), there is one even
more "interesting" that you haven't discovered yet, and this one is
going to reveal the fault on your part:

Every Perl textbook states that sort() is "special", its sub is
"special" and its $a and $b are "special". Some even warns "weird
and unexpected result" about messing with those $a and $b. Your
code demenstrated exactly that -- The subs for sort should contain
only comparision tests. Nothing more, nothing less! So if you go
back to your code and comment out those two print statements
inside the sort sub block, your code works perfectly!!

In retrospect, I don't remember ever seeing anyone putting "extra"
statements inside a sort sub.
 
A

Anno Siegel

MSG said:
David McNerney wrote:
[...]
Below is a test case, verified to demonstrate the problem on Perl 5.8.5
(Linux) and 5.8.6 (Mac OS X):


#!/usr/bin/perl -w
use strict;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";
}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;
print "\@list, sorted: ".join(", ", @list)."\n";

#&another::testFunction;
[...]

... So if you go
back to your code and comment out those two print statements
inside the sort sub block, your code works perfectly!!

I don't see the effect you claim to see.

Have you run your modified test with warnings switched on? Do so.
If you still believe the code works perfectly, please post exactly
what you are running.

Anno
 
P

Peter J. Holzer

Anno said:
MSG said:
David McNerney wrote:
[...]
Below is a test case, verified to demonstrate the problem on Perl
5.8.5 (Linux) and 5.8.6 (Mac OS X):


#!/usr/bin/perl -w
use strict;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";
}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;
print "\@list, sorted: ".join(", ", @list)."\n";

#&another::testFunction;
[...]

... So if you go
back to your code and comment out those two print statements
inside the sort sub block, your code works perfectly!!

I don't see the effect you claim to see.

I see it.
Have you run your modified test with warnings switched on?

-w is in the shebang line, isn't it?
Do so. If you still believe the code works perfectly, please post
exactly what you are running.

Exactly the code posted by David, except that the two print statements
in another::testFunction are commented out. Works with perl 5.8.0, 5.8.4
and 5.8.7 on Linux. I also modified the second half of testFunction to:

my @values2 = sort
{
# print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,3,4,2,5);
die unless (@values2 == 5 && $values2[0] == 1 && $values2[2] == 3);

to verify that the sort works correctly. It does.

I only get warnings if there is a print statement in both sorts.

Replacing the print statement with an assignment to a lexically scoped
variable doesn't change anything, so I think, it isn't the print
statement but rather the interpolation of $a and $b which breaks their
magic.

Oh, and BTW, wrapping { local ($a, $b); ... } around the sort doesn't
help, either.

hp
 
A

Anno Siegel

Peter J. Holzer said:
Anno said:
MSG said:
David McNerney wrote:
[...]

Below is a test case, verified to demonstrate the problem on Perl
5.8.5 (Linux) and 5.8.6 (Mac OS X):


#!/usr/bin/perl -w
use strict;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";
}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;
print "\@list, sorted: ".join(", ", @list)."\n";

#&another::testFunction;
[...]

... So if you go
back to your code and comment out those two print statements
inside the sort sub block, your code works perfectly!!

I don't see the effect you claim to see.

I see it.
Have you run your modified test with warnings switched on?

-w is in the shebang line, isn't it?

Ah, right -w. How unfashionable :) I had to change the shebang line
and -w got lost in the process. That's perhaps another reason to
avoid flags in the shebang line if possible. It makes a routine
replacement easier.
Do so. If you still believe the code works perfectly, please post
exactly what you are running.

Exactly the code posted by David, except that the two print statements
in another::testFunction are commented out. Works with perl 5.8.0, 5.8.4
and 5.8.7 on Linux. I also modified the second half of testFunction to:

my @values2 = sort
{
# print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,3,4,2,5);
die unless (@values2 == 5 && $values2[0] == 1 && $values2[2] == 3);

to verify that the sort works correctly. It does.

I only get warnings if there is a print statement in both sorts.

So commenting out only one of the print statements is enough?

I get warnings with and without the print statements. Perl 5.8.7,
on Darwin 8.5.0 and Linux 2.0.36.

[discussion snipped]

Anno
 
P

Peter J. Holzer

Anno said:
Peter J. Holzer said:
Anno said:
David McNerney wrote:
Below is a test case, verified to demonstrate the problem on
Perl 5.8.5 (Linux) and 5.8.6 (Mac OS X): [...]
... So if you go back to your code and comment out those two print
statements inside the sort sub block, your code works perfectly!!

I don't see the effect you claim to see.

I see it.
Have you run your modified test with warnings switched on?

-w is in the shebang line, isn't it?

Ah, right -w. How unfashionable :) I had to change the shebang line
and -w got lost in the process. That's perhaps another reason to
avoid flags in the shebang line if possible. It makes a routine
replacement easier.

Hmm. I don't think I ever accidentally removed it when I replaced the
interpreter in the file. But I agree that it is easy to forget if you
invoke the perl-interpreter explicitely on the command line.

Your comment made me think that maybe there is a difference between -w
and "use warnings;" in this case, but there isn't.

So commenting out only one of the print statements is enough?
Yes.

I get warnings with and without the print statements. Perl 5.8.7,
on Darwin 8.5.0 and Linux 2.0.36.

Very strange. Perl 5.8.7 on Linux was also one of the versions I tested:

Summary of my perl5 (revision 5 version 8 subversion 7) configuration:
Platform:
osname=linux, osvers=2.4.21-32.elsmp, archname=i686-linux
uname='linux habanero.wsr.ac.at 2.4.21-32.elsmp #1 smp fri apr 15
21:17:59 edt 2005 i686 i686 i386 gnulinux '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
ccversion='', gccversion='3.2.3 20030502 (Red Hat Linux 3.2.3-53)',
gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.3.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
Compile-time options: USE_LARGE_FILES
Built under linux
Compiled at Jan 19 2006 14:53:11
@INC:
/usr/local/lib/perl5/5.8.7/i686-linux
/usr/local/lib/perl5/5.8.7
/usr/local/lib/perl5/site_perl/5.8.7/i686-linux
/usr/local/lib/perl5/site_perl/5.8.7
/usr/local/lib/perl5/site_perl
 
M

MSG

Anno said:
I get warnings with and without the print statements. Perl 5.8.7,
on Darwin 8.5.0 and Linux 2.0.36.

What kind of warnings? I don''t get any thing except the correct
result when taking out those two pritnt statements.
Here is the exact code I ran. It is David's code but with print
commented out. Also I switched -w to 'use warnings' to avoide
any possible confusion.

##############
#!/usr/bin/perl
use strict;
use warnings;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
# print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
# print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";

}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;

print "\@list, sorted: ".join(", ", @list)."\n";

###############
Here is the result:
@otherList, sorted: 1, 2, 3, 4, 5
@otherList2, sorted: 1, 2, 3, 4, 5
@otherList, sorted: 1, 2, 3, 4, 5
@otherList2, sorted: 1, 2, 3, 4, 5
@list, sorted: 11, 12, 13

It gives clean and correct result as long as you keep the sort sub
clean.
 
A

Anno Siegel

MSG said:
What kind of warnings? I don''t get any thing except the correct
result when taking out those two pritnt statements.
Here is the exact code I ran. It is David's code but with print
commented out. Also I switched -w to 'use warnings' to avoide
any possible confusion.

The behavior is erratic. As reported earlier, I have seen the warnings
(and invalid sort results) without any of the two critical print statements.
Currently I have two scripts, one derived from the OP, the other from
the code you posted (now snipped). Both don't show the warnings when
all prints are commented out, but only with one of them is it sufficient
to comment out one of the warnings. Go figure. I don't even bother
to identify which script behaves in which way. It makes no sense on
Perl level.

Anno
 
M

MSG

Anno said:
The behavior is erratic. As reported earlier, I have seen the warnings
(and invalid sort results) without any of the two critical print statements.
I am afraid that you were not testing it right. For the code I posted
above, how many times did it give you a warning or a wrong result? Can
you re-produce it?
Currently I have two scripts, one derived from the OP, the other from
the code you posted (now snipped). Both don't show the warnings when
all prints are commented out, but only with one of them is it sufficient
to comment out one of the warnings.
I don't understand what you are saying here. Can you elaborate what
you are trying to test?
 
D

David McNerney

I duplicate your finding ... if those two print statements are
commented out, both the nested sorts work correctly. So, it certainly
does seem that the sort operator cannot always be relied upon when used
with anonymous functions that contain anything but simple comparison.
That's really fine, as far as I'm concerned, now that I know ... I can
always declare a prototyped sort function in the future if I'm writing
code that I expect to get called from within a sort. I don't, however,
find any warnings about such code in the "sort" function documentation
in the Camel book, my primary reference.

There are any number of reasons why one might want to have what you
refer to as "extra" code in a sort. This comes up when sorting objects.
We have a large class library at our site, and one of the chief reasons
that we use Perl, and not some other language like Java, C, or Visual
Basic, is because it allows us to program at a high conceptual level,
throwing around lists of objects with sort, grep and map. If you are
sorting a list of objects, you might have code like this:

my @sortedList = sort { $a->method() <=> $b->method() } @list;

Now, the "method" method might be a simple attribute accessor, or it
might have more complex code inside. It could even call other methods,
that in turn call others, one of which happens to do a couple of brief
sorts in its implementation. Not an efficiency issue, if the list of
objects is not so long, and the code that does all this is not to be
executed many times. Again, one of the main reasons we use Perl at our
site is to leverage >programmer< efficiency.

Thanks again to everyone who has taken an interest in this little
quirk. I posted it to get some perspective on what was going on; in
around 8 years of heavy Perl use, this is only the second or third time
that we've been bitten by a bug in the language. That's a great record,
and this little issue can easily be worked around now that I know about
it.

David said:
Thanks very much to everyone who replied here. Sorry I was a little
terse Brian! I just sent the following to (e-mail address removed):

When an anonymous sort function invokes a method or function defined in
another package, and that
method or function does multiple sorts using its own anonymous sort
functions, the second and later
of these latter sorts fail; $a and $b are not defined. This does not
occur if the function is moved
to the same package as the code performing the first sort, nor does it
occur if the latter "nested"
sorts use a seperately defined sort function with prototypes (in which
case, of course, the $a and $b
variables are not used). And, as implied above, the first "nested" sort
executes correctly.

One person on usenet suggested that nobody ought to be doing such a
thing as described above. Actually,
there are many times when working with a large object-oriented Perl
library when one might need
to sort a (often short) list of objects, and compare values returned by
object methods in the comparison function.
A programmer writing such code should not have to think about whether
the object methods they choose
to invoke happen to use multiple non-prototyped sorts in their
implementation.

Below is a test case, verified to demonstrate the problem on Perl 5.8.5
(Linux) and 5.8.6 (Mac OS X):


#!/usr/bin/perl -w
use strict;

package another;
#package my_package;
sub testFunction
{
my @otherList = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList, sorted: ".join(", ", @otherList)."\n";

my @otherList2 = sort
{
print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);
print "\@otherList2, sorted: ".join(", ", @otherList2)."\n";
}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
#&testFunction;
return $a <=> $b;
} @list;
print "\@list, sorted: ".join(", ", @list)."\n";

#&another::testFunction;

Although I agree that you have an rather interesting discovery,
(which may take a real Perl expert to explain), there is one even
more "interesting" that you haven't discovered yet, and this one is
going to reveal the fault on your part:

Every Perl textbook states that sort() is "special", its sub is
"special" and its $a and $b are "special". Some even warns "weird
and unexpected result" about messing with those $a and $b. Your
code demenstrated exactly that -- The subs for sort should contain
only comparision tests. Nothing more, nothing less! So if you go
back to your code and comment out those two print statements
inside the sort sub block, your code works perfectly!!

In retrospect, I don't remember ever seeing anyone putting "extra"
statements inside a sort sub.
 
B

Brian McCauley

Brian said:
I've reproduced your problem and it does look like a particularly nasty
bug in perl.

Well I've not tracked it down but here's something interesting.

If I evalute \$a,\$a,*a{SCALAR} (where $a is not declared lexically)
then I'd expect to get the same value three times. And in the absense
of sort thats what I do see. But inside sort something odd happens.
And it looks like under some circumstances this something odd fails to
happen properly. And now I think about it this may explain problems
I've seen in my programs in the past.

I haven't yet fully understood the behaviour of the following test but
here it is anyhow.

use strict;
no warnings;

package P;

{
my (%xref,@keep_alive);
my $i=1;
sub p {
printf "%-7s", shift;
for (@_) {
# Give each a new number the first time we see it
printf "%10s ", $xref{$_} ||= do {
# Avoid same address ever being alocated to another SV
push @keep_alive => $_;
$i++;
};
}
print "\n";
}
}

my $A='a'; # A symref

print " \\\$a \\\$a *a{SCALAR}\n";

sub testFunction {
no strict 'refs';
p 'before',\$a,\$a,*a{SCALAR};
my @values = sort
{
p 'sort1',\$a,\$a,*a{SCALAR};
$a <=> $b;
} (1,2);
p 'between',\$a,\$a,*a{SCALAR};
my @values2 = sort
{
p 'sort2',\$a,\$a,*a{SCALAR};
$a <=> $b;
} (1,2);
p 'after',\$a,\$a,*a{SCALAR};
}

print "------------------------------------------------------ in P\n";
testFunction;
print "------------------------------------------------------ in sort
in P\n";
my @seqnums1 = sort
{
testFunction;
return $a <=> $b;
} 48561006,488561006;
print "------------------------------------------------------ in Q\n";
package Q;
P::testFunction;
print "------------------------------------------------------ in sort
in Q\n";
my @seqnums2 = sort
{
&P::testFunction;
return $a <=> $b;
} 486098561006,486098561006;
__END__
\$a \$a *a{SCALAR}
------------------------------------------------------ in P
before 1 1 1
sort1 2 3 4
between 1 1 1
sort2 5 6 7
after 1 1 1
------------------------------------------------------ in sort in P
before 8 9 10
sort1 11 12 4
between 13 14 10
sort2 15 16 7
after 17 18 10
------------------------------------------------------ in Q
before 1 1 1
sort1 19 20 4
between 1 1 1
sort2 21 22 7
after 1 1 1
------------------------------------------------------ in sort in Q
before 1 1 1
sort1 23 24 4
between 1 1 1
sort2 1 1 1
after 1 1 1
 
A

Anno Siegel

MSG said:
I am afraid that you were not testing it right. For the code I posted
above, how many times did it give you a warning or a wrong result? Can
you re-produce it?

Hm? What has the number of warnings to do with anything? It is determined
by how many comparisons the sort algorithm does for the given lists.
I get 16 warnings, reproducibly.
I don't understand what you are saying here. Can you elaborate what
you are trying to test?

Sigh. This is pretty useless because the behavior seems to be governed
by factors we don't control. Even if you run my code and see different
results we'd be none the wiser. If you really want to follow this up:

This code gives me warnings:

#!/usr/local/bin/perl
use strict; use warnings; $| = 1;

package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort 1] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);

my @values2 = sort
{
# print "[testFunction, sort 2] a=$a b=$b\n";
$a <=> $b
} (5,4,3,2,1);

}

package my_package;
my @list = (13,12,11);
@list = sort
{
&another::testFunction;
return $a <=> $b;
} @list;

print "\@list, sorted: ".join(", ", @list)."\n";
__END__

This code runs without warnings:

#!/usr/local/bin/perl
use strict; use warnings; $| = 1;

package another;
sub testFunction
{
my @values = sort
{
print "[testFunction, sort function 1] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);

my @values2 = sort
{
# print "[testFunction, sort function 2] a=$a b=$b\n";
$a <=> $b
} (1,2,3,4,5);
}

package my_package;
my @seqnums = (486098561006,486098561006);
@seqnums = sort
{
&another::testFunction;
return $a <=> $b;
} @seqnums;

print "\@seqnums, sorted: ".join(", ", @seqnums)."\n";
__END__

Anno
 
R

robic0

Anno said:
Peter J. Holzer said:
Anno Siegel wrote:
David McNerney wrote:
Below is a test case, verified to demonstrate the problem on
Perl 5.8.5 (Linux) and 5.8.6 (Mac OS X): [...]
... So if you go back to your code and comment out those two print
statements inside the sort sub block, your code works perfectly!!

I don't see the effect you claim to see.

I see it.

Have you run your modified test with warnings switched on?

-w is in the shebang line, isn't it?

Ah, right -w. How unfashionable :) I had to change the shebang line
and -w got lost in the process. That's perhaps another reason to
avoid flags in the shebang line if possible. It makes a routine
replacement easier.

Hmm. I don't think I ever accidentally removed it when I replaced the
interpreter in the file. But I agree that it is easy to forget if you
invoke the perl-interpreter explicitely on the command line.

Your comment made me think that maybe there is a difference between -w
and "use warnings;" in this case, but there isn't.
[snipped]
Just a comment on -w and use warnings under windows.
Using the "shebanged the football team" line with -w
warnings showed up in the command console the first time it was
run. Subsequent "uparrow return"'s didn't show the results
depending on wheather the notepad code file was re-saved between
runs. It was intermittent.

Probably just my Perl installation but have seen this console behaviour
before. Just got rid of it and used the use warnings.
I can probably tell you that Perl's phase 1 (?) parser has problems.
Who knows how far back thats existed. Its pretty niaeve to think of
Perl as perfect.

robic0
 
R

robic0

I am afraid that you were not testing it right. For the code I posted
above, how many times did it give you a warning or a wrong result? Can
you re-produce it?

I don't understand what you are saying here. Can you elaborate what
you are trying to test?

I think its pretty fair to say to solve the argument, the results of the
the "other package" sort, are printed outside of the "other package" sort,
as a final step from the main thread execution chain.
Just make it a package global.

ONE other thing im not comprehending from this post, given the docs on sort.
sort block list
sort subname list

Am I missing something here?

sort {...} (...);
^^^^^

Is this NOT a block? Seems to be implied in this thread its
an "anonymous subroutine".

If it is contextually an anonymous sub, how is a block represented?

Im my eyes, the problem is as simple as I percieved in a different post in this
thread.

I can understand not "feeding the troll", but don't take my idea's and make it your
own without attribution.

It is as stands an anomoly. The OP did the right thing by submitting this!

robic0
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,479
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top