(Perl) FILE/IO and grep causes 100% CPU

Discussion in 'Perl Misc' started by mark@gowans.org, Nov 25, 2004.

  1. Guest

    Hi There..

    I've a perl script which amongst other things searches for text strings
    in a large number of files. The problem I've got is, while the script
    is running - idle cpu% drops to 0. (A mix of user and kernel usage).
    The code is:

    #!/usr/bin/perl
    use strict;
    my @files = `ls -lart /app/EV4/data/db/u* | awk '{print \$NF}'`;
    foreach my $file (@files) {
    chomp ($file);
    (my $date) = $file =~ m/.*\.(.*)/;
    my @result = `grep anicol $file`;
    }


    @files contains around 7800 entries. I've tried using alternatives to
    grep (by opening each file and using pattern matching) but to no avail.


    Sticking a 'sleep 1' in the foreach solves the problem.. but I dont
    really want to wait 2 hours for the script to run.

    Can anyone suggest a better way of doing the above or am I asking the
    impossible?

    Many thanks in advance - much appreciated,

    Cheers,

    Mark
     
    , Nov 25, 2004
    #1
    1. Advertising

  2. Anno Siegel Guest

    <> wrote in comp.lang.perl.misc:
    > Hi There..
    >
    > I've a perl script which amongst other things searches for text strings
    > in a large number of files. The problem I've got is, while the script
    > is running - idle cpu% drops to 0. (A mix of user and kernel usage).


    What's wrong with that? It's a busy program, you want to give it
    all the CPU it can get. If that's a problem, read "man nice". But
    that's Unix, not Perl.

    > The code is:
    >
    > #!/usr/bin/perl
    > use strict;
    > my @files = `ls -lart /app/EV4/data/db/u* | awk '{print \$NF}'`;


    Why do you "ls -l" and then use awk to get the file name?

    > foreach my $file (@files) {
    > chomp ($file);
    > (my $date) = $file =~ m/.*\.(.*)/;
    > my @result = `grep anicol $file`;
    > }


    That's a shell script, shoehorned into a Perl wrapper.

    Here is one way to get the files from Perl directly (untested):

    my $d;
    opendir $d, $_ or die "Can't open $_: $!" for '/app/EV4/data/db/';
    while ( $_ = readdir $d ) {
    next unless /^u/;
    my ( $date) = m/.*\.(.*)/;
    # now open the file in $_ and use Perl's grep() to extract
    # the lines
    }

    The above is just a partial re-write of your script in Perl. It
    won't be lighter on the CPU, and there is no general reason why it
    should.

    Anno
     
    Anno Siegel, Nov 25, 2004
    #2
    1. Advertising

  3. <> wrote in message
    news:...
    > Hi There..
    >
    > I've a perl script which amongst other things searches for text strings
    > in a large number of files. The problem I've got is, while the script
    > is running - idle cpu% drops to 0. (A mix of user and kernel usage).
    > The code is:
    >
    > #!/usr/bin/perl
    > use strict;
    > my @files = `ls -lart /app/EV4/data/db/u* | awk '{print \$NF}'`;
    > foreach my $file (@files) {
    > chomp ($file);
    > (my $date) = $file =~ m/.*\.(.*)/;
    > my @result = `grep anicol $file`;
    > }


    I've decided to create a SWIP (Shell Wrapped in Perl) page.

    Congratulations, you earn the honour of holding the first entry in the list.
     
    Peter Sundstrom, Nov 27, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Xavier Osa
    Replies:
    0
    Views:
    640
    Xavier Osa
    Jan 9, 2004
  2. Alex via .NET 247

    using COM from aspx causes CPU to rise to 100%

    Alex via .NET 247, Aug 4, 2004, in forum: ASP .Net Security
    Replies:
    0
    Views:
    119
    Alex via .NET 247
    Aug 4, 2004
  3. Michael Dichtl

    WIN32OLE_EVENT causes cpu running 100%

    Michael Dichtl, Jul 4, 2007, in forum: Ruby
    Replies:
    0
    Views:
    83
    Michael Dichtl
    Jul 4, 2007
  4. fred
    Replies:
    3
    Views:
    284
    Zifud
    Mar 17, 2005
  5. Replies:
    5
    Views:
    891
Loading...

Share This Page