Improving performance of a simple Perl script

Discussion in 'Perl Misc' started by vfoley@gmail.com, Jul 28, 2006.

  1. Guest

    Hello everyone,

    I wrote a little script in Perl to make stats on a log file. You can
    view the source at: http://pastebin.ca/104391 . When running on an
    OpenBSD 3.9 machine (Pentium III 450 MHz, 128 MB RAM), with Perl 5.8.6,
    the script takes 35 seconds to crunch through a file with 175,000
    lines. Not bad. However, a Ruby script (http://pastebin.ca/104393)
    that I wrote a while ago does the same file in 23 seconds. As far as I
    know, the Perl interpreter is faster than Ruby, so I would be
    interested to know how I could improve the execution speed of the Perl
    script. I'm still very new, so I don't know all the idioms and such,
    so any help would be greatly appreciated.

    Also, when I profiled the Perl script, dprofpp's time percentages only
    added up to like 5 or 10%. Why is that? Where is the remaining 90%?

    Regards,

    Vincent
     
    , Jul 28, 2006
    #1
    1. Advertising

  2. Guest

    wrote:
    > I wrote a little script in Perl to make stats on a log file. You can
    > view the source at: http://pastebin.ca/104391 . When running on an


    Why were you unable to post it here? Why should we go and visit your
    website to look at your script... expecting more hits for averts
    perhaps?

    Axel
     
    , Jul 28, 2006
    #2
    1. Advertising

  3. wrote:
    >
    > I wrote a little script in Perl to make stats on a log file. You can
    > view the source at: http://pastebin.ca/104391 . When running on an
    > OpenBSD 3.9 machine (Pentium III 450 MHz, 128 MB RAM), with Perl 5.8.6,
    > the script takes 35 seconds to crunch through a file with 175,000
    > lines. Not bad. However, a Ruby script (http://pastebin.ca/104393)
    > that I wrote a while ago does the same file in 23 seconds. As far as I
    > know, the Perl interpreter is faster than Ruby, so I would be
    > interested to know how I could improve the execution speed of the Perl
    > script. I'm still very new, so I don't know all the idioms and such,
    > so any help would be greatly appreciated.


    7 sub mktime {
    8 my ($day, $month_name, $year) = @_;
    9 my %months = (
    10 Jan => 0,
    11 Feb => 1,
    12 Mar => 2,
    13 Apr => 3,
    14 May => 4,
    15 Jun => 5,
    16 Jul => 6,
    17 Aug => 7,
    18 Sep => 8,
    19 Oct => 9,
    20 Nov => 10,
    21 Dec => 11
    22 );
    23 return timelocal(0, 0, 0, $day, $months{$month_name}, $year);
    24 }

    You should declare and define the %months hash outside the subroutine so it
    isn't assigned everytime the subroutine is called.

    Date::Calc may be faster than Time::Local.


    49 if (exists($counts{$date})) {
    50 unless (grep { $ip eq $_ } @{$counts{$date}}) {
    51 push(@{$counts{$date}}, $ip);
    52 }
    53 } else {
    54 $counts{$date} = [];
    55 }

    Using a Hash of Hashes would be faster than grepping through an array for
    every $ip and perl autovivifies so the test for exists() is superfluous.


    60 printf "%s\t%d\n", strftime("%d-%b-%Y", 0, 0, 0, $day, $month, $year),
    61 scalar(@{$counts{$key}});

    print() is safer and faster than printf() and you don't need printf() anyway.



    John
    --
    use Perl;
    program
    fulfillment
     
    John W. Krahn, Jul 29, 2006
    #3
  4. Guest

    Thank you for your suggestions, John. Using a hash of hashes really
    improved the performance, I went from 35-36 seconds to 6 seconds, a 6x
    improvement. Thank you again!
     
    , Jul 29, 2006
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Robin

    Improving Performance in ASP.Net

    Robin, Mar 4, 2005, in forum: ASP .Net
    Replies:
    1
    Views:
    415
    Scott Allen
    Mar 4, 2005
  2. ruds
    Replies:
    13
    Views:
    589
    squirrel
    Apr 8, 2007
  3. Dale
    Replies:
    2
    Views:
    781
    Mike Meyer
    Jan 11, 2008
  4. yawnmoth
    Replies:
    6
    Views:
    1,542
    yawnmoth
    Aug 5, 2008
  5. Ryan Rosario
    Replies:
    1
    Views:
    243
    odeits
    Mar 19, 2009
Loading...

Share This Page