threads / freeing memory

[Stuart Kendrick]

i'm using threads to spawn a lot of tasks. the memory footprint of my
process keeps growing ... until the OS kills it. i was hoping that
detach or join would free memory ... but it looks like they don't. am
i missing something or am i looking at a bug?

perl -v

This is perl, v5.8.3 built for i686-linux-thread-multi
[...]

i'm running perl 5.8.3 under SuSE 8.2

test1.plx
#!/usr/bin/perl
use strict;
use threads;
use warnings;

our $big = 500;
our $i;

dome();

sub dome {
my $thr;
for ($i = 0; $i < $big; $i++) {
$thr = threads->new("test_me");
$thr->detach;
}
}

sub test_me {
print "Running thread $i\n";
}

when i run this program, "top" shows the memory footprint growing to
about 420MB (the box has 512MB total) ... and then the script
terminates with the message "Killed".


OK, so then I figured I'd try "joining" the things. I don't care
about the return value of "test_me" ... but hey, maybe "joining" frees
memory whereas "detaching" doesn't.

test2.plx
#!/usr/bin/perl

use strict;
use threads;
use warnings;

our $big = 5000;
our $i;
our $max = 30;

dome();

sub dome {
my $result;
my $thr;
for ($i = 0; $i < $big; $i++) {
$thr = threads->new("test_me");

$result = int($i / $max);
if ($i == $result * $max) {
print "Joining threads\n";
foreach $thr (threads->list) {
if ($thr->tid && !threads::equal($thr, threads->self)) {
$thr->join;
}
}
sleep 2;
}
}
}

sub test_me {
print "Running thread $i\n";
}


No difference -- the process grows to ~420MB and then terminates with
the message "Killed".

Insights?

--sk

Stuart Kendrick
FHCRC

 

[thundergnat]

i'm using threads to spawn a lot of tasks. the memory footprint of my
process keeps growing ... until the OS kills it. i was hoping that
detach or join would free memory ... but it looks like they don't. am
i missing something or am i looking at a bug?

perl -v

This is perl, v5.8.3 built for i686-linux-thread-multi
[...]

i'm running perl 5.8.3 under SuSE 8.2

test1.plx
#!/usr/bin/perl
use strict;
use threads;
use warnings;

our $big = 500;
our $i;

dome();

sub dome {
my $thr;
for ($i = 0; $i < $big; $i++) {
$thr = threads->new("test_me");
$thr->detach;
}
}

sub test_me {
print "Running thread $i\n";
}

when i run this program, "top" shows the memory footprint growing to
about 420MB (the box has 512MB total) ... and then the script
terminates with the message "Killed".

You may want to try keeping your variables in a tighter scope. I made a
few changes to your first script and reduced the momory footprint by 90%
or better. (On my system at least.) Don't know whether this will be
helpful or not, just an observation.


#!/usr/bin/perl
use strict;
use threads;
use warnings;

my $big = 500;

dome();

sub dome {
for (0..$big) {
my $thr = threads->new(\&test_me, $_);
$thr->detach;
}
}

sub test_me {
my $i = shift;
print "Running thread $i\n";
}