Win32 Perl slowdown

[Andy Culbertson]

I'm running ActivePerl on Windows XP SP2. My job involves processing
text files that are 8-12 M, which I do using regular expressions. Up
until last September I was able to do this fairly quickly, but at that
point my script suddenly took a lot longer to run, which really
hinders my work.

I thought maybe the problem was some change in the way ActivePerl
handled regular expressions, but someone has suggested that it might
have to do with memory allocation.

I wrote some test scripts that I profiled under various conditions,
and in a nutshell I found that the following had little to no effect:

- uninstalling the Windows updates I had installed between the time
the script ran slowly and the last time I remembered it running
normally (a period of about a month)

- testing the script under ActivePerl builds 817, 813, and 811.

- testing the script on three different machines running WinXP SP2

Running the same script on Linux, however, had a dramatic effect. The
regular expression portion of the script ran 200 times faster than it
did on Windows (though this is not the case with the script below).
On Linux, the speed of the script I normally work with runs at a
comparable speed to what I remember previously on Windows.

Given my test results, I can't imagine what could have caused such a
drastic change in speed or how to fix it. Does anyone have any
ideas? It's inconvenient to switch from Windows to Linux to do this
part of my work, but I can continue doing that if necessary.

Here is some of my data to give you an idea of what I found.
Unfortunately, my test script evolved during my investigation, so I
don't really have a consistent set of data to present, but here are
some basic tests with the latest version of the script. I can rerun
some of the earlier tests with this script if anyone needs me to.

First, the script (profile.pl):

------------------------

#!usr/bin/perl -w

use strict;

my $megabytes = 10;
print "Building string - $megabytes M\n";
my $string = buildString();
print "Tokenizing\n";
my @list = tokenize($string);
print "Stripping tags\n";
$string = stripTags($string);
# This splicing sub is *really* inefficient, so feel free to omit it.
# I was just testing different functions to see if they all slowed
down
# equally.
print "Splicing list\n";
@list = spliceList(@list);
print "Building another list\n";
&buildList;

sub buildString {
my $string;
for my $m (1..$megabytes) {
for my $i (0..65536) {
$string .= "abcdefgh";
$string .= "<abcdef>";
}
}
$string;
}

sub tokenize {
my $string = shift;
my @list = $string =~ /<[^<>]*>|[^<>]+/sg;
@list;
}

sub stripTags {
my $string = shift;
$string =~ s/<[^<>]*>//sg;
$string;
}

sub spliceList {
my @list = @_;
my $i = 0;
while ($i <= $#list) {
if ($list[$i] =~ /^</) {
splice @list, $i, 1;
$i--;
}
$i++;
}
@list;
}

sub buildList {
my @list;
for my $i (1..1000) {
push @list, "a";
}
}

------------------------

Seconds per call for each sub and system:

WinXP Linux Mac NetBSD
&spliceList 272.57 236.94 1689.1 329.69
&buildString 21.968 0.5600 1.1600 22.750
&tokenize 15.954 1.6300 10.750 6.7000
&stripTags 0.3440 0.3500 1.0900 0.4600
&buildList - 0.0600 0.0100 -


System details:

WinXP SP2
Intel(R) Pentium(R) 4 CPU 3.40GHz
This is perl, v5.8.7 built for MSWin32-x86-multi-thread
ActivePerl build 813

Linux version 2.6.15-27-386 (buildd@terranova) (gcc version 4.0.3
(Ubuntu 4.0.3-1ubuntu5))
Intel(R) Pentium(R) 4 CPU 3.40GHz
This is perl, v5.8.7 built for i486-linux-gnu-thread-multi

Mac OS X 10.4.8
1.33 GHz PowerPC G4
This is perl, v5.8.6 built for darwin-thread-multi-2level

NetBSD 3.0.1_STABLE (GENERIC.MPACPI)
Intel (686-class), 2992.59 MHz, id 0xf34
This is perl, v5.8.8 built for i386-netbsd-thread-multi