(Perl) FILE/IO and grep causes 100% CPU

[mark]

Hi There..

I've a perl script which amongst other things searches for text strings
in a large number of files. The problem I've got is, while the script
is running - idle cpu% drops to 0. (A mix of user and kernel usage).
The code is:

#!/usr/bin/perl
use strict;
my @files = `ls -lart /app/EV4/data/db/u* | awk '{print \$NF}'`;
foreach my $file (@files) {
chomp ($file);
(my $date) = $file =~ m/.*\.(.*)/;
my @result = `grep anicol $file`;
}


@files contains around 7800 entries. I've tried using alternatives to
grep (by opening each file and using pattern matching) but to no avail.


Sticking a 'sleep 1' in the foreach solves the problem.. but I dont
really want to wait 2 hours for the script to run.

Can anyone suggest a better way of doing the above or am I asking the
impossible?

Many thanks in advance - much appreciated,

Cheers,

Mark

 

[Anno Siegel]

Hi There..

I've a perl script which amongst other things searches for text strings
in a large number of files. The problem I've got is, while the script
is running - idle cpu% drops to 0. (A mix of user and kernel usage).

What's wrong with that? It's a busy program, you want to give it
all the CPU it can get. If that's a problem, read "man nice". But
that's Unix, not Perl.

The code is:

#!/usr/bin/perl
use strict;
my @files = `ls -lart /app/EV4/data/db/u* | awk '{print \$NF}'`;

Why do you "ls -l" and then use awk to get the file name?

foreach my $file (@files) {
chomp ($file);
(my $date) = $file =~ m/.*\.(.*)/;
my @result = `grep anicol $file`;
}

That's a shell script, shoehorned into a Perl wrapper.

Here is one way to get the files from Perl directly (untested):

my $d;
opendir $d, $_ or die "Can't open $_: $!" for '/app/EV4/data/db/';
while ( $_ = readdir $d ) {
next unless /^u/;
my ( $date) = m/.*\.(.*)/;
# now open the file in $_ and use Perl's grep() to extract
# the lines
}

The above is just a partial re-write of your script in Perl. It
won't be lighter on the CPU, and there is no general reason why it
should.

Anno

 

[Peter Sundstrom]

Hi There..

I've a perl script which amongst other things searches for text strings
in a large number of files. The problem I've got is, while the script
is running - idle cpu% drops to 0. (A mix of user and kernel usage).
The code is:

#!/usr/bin/perl
use strict;
my @files = `ls -lart /app/EV4/data/db/u* | awk '{print \$NF}'`;
foreach my $file (@files) {
chomp ($file);
(my $date) = $file =~ m/.*\.(.*)/;
my @result = `grep anicol $file`;
}

I've decided to create a SWIP (Shell Wrapped in Perl) page.

Congratulations, you earn the honour of holding the first entry in the list.