perl hash speed and memory efficiency

Discussion in 'Perl Misc' started by odigity, May 27, 2004.

  1. odigity

    odigity Guest

    I'm running some tests to try to gauge the speed of perl's hashing
    abilities, as well as the memory footprint. I wrote this function:

    sub buildhash
    {
    my %hash;
    foreach my $foo (1..100_000) {
    foreach my $bar ('a'..'z') {
    $hash{"$foo.$bar"} = 1;
    }
    }
    undef %hash;
    }

    Then I threw in the Benchmark module, like this:

    timethis( 1, "buildhash()" );

    It seems to use about 200MB of memory for 2.6 million small key/value
    pairs, which is pretty efficient (~80bytes/pair). However, it doesn't
    release the memory after the undef (I checked by stopping execution at
    that point with a sleep statement and studying memory usage with
    `ps`).

    It either takes 11 seconds or 75 seconds depending on how I execute
    it. Let me explain.

    I first tried running it once, and it took 11 seconds. I tried twice,
    and it took 86. This didn't make any sense to me. Here's the command
    I used:

    timethis( 2, "buildhash()" );

    Then I tried unrolling it like this:

    timethis( 1, "buildhash()" );
    timethis( 1, "buildhash()" );

    And that took 22 seconds (11/each), as I expected the first time.

    So the question that is most driving me crazy is: For the sake of
    Pete, why the difference!?

    -ofer
    odigity, May 27, 2004
    #1
    1. Advertising

  2. odigity

    Ben Morrow Guest

    Quoth (odigity):
    > I'm running some tests to try to gauge the speed of perl's hashing
    > abilities, as well as the memory footprint. I wrote this function:
    >
    > sub buildhash
    > {
    > my %hash;
    > foreach my $foo (1..100_000) {
    > foreach my $bar ('a'..'z') {
    > $hash{"$foo.$bar"} = 1;
    > }
    > }
    > undef %hash;


    There is no need for this. %hash will be deallocated at end-of-scope.

    > }
    >
    > Then I threw in the Benchmark module, like this:
    >
    > timethis( 1, "buildhash()" );
    >
    > It seems to use about 200MB of memory for 2.6 million small key/value
    > pairs, which is pretty efficient (~80bytes/pair). However, it doesn't
    > release the memory after the undef (I checked by stopping execution at
    > that point with a sleep statement and studying memory usage with
    > `ps`).


    <sigh>

    Under most circumstances, memory once allocated to a process can never
    be released. The memory goes into a process-internal free pool, so if
    you were to run the sub again memory usage would not go up any further.

    Ben

    --
    'Deserve [death]? I daresay he did. Many live that deserve death. And some die
    that deserve life. Can you give it to them? Then do not be too eager to deal
    out death in judgement. For even the very wise cannot see all ends.'
    Ben Morrow, May 27, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    9
    Views:
    338
    Jerry Coffin
    Jul 28, 2006
  2. Replies:
    9
    Views:
    565
    SM Ryan
    Oct 20, 2007
  3. rp
    Replies:
    1
    Views:
    494
    red floyd
    Nov 10, 2011
  4. Anthony Martinez
    Replies:
    4
    Views:
    259
    Robert Klemme
    Jun 11, 2007
  5. tak

    Perl hash of hash efficiency.

    tak, Aug 2, 2006, in forum: Perl Misc
    Replies:
    28
    Views:
    429
    Ian Wilson
    Aug 7, 2006
Loading...

Share This Page