Help! How to find memory leak?

A

Amanda

I am using a Perl program that must process a large number (10**7)
of items. As the program runs, I've noticed that it becomes less
and less efficient in its processing, and at the same time, it
consumes more and more memory. Reading over the program's voluminous
code (not written by me), I can't find what causes this growth in
memory consumption. Are there any tools that could help figure
out where/how this is coming about?

Thanks!

Amanda

--
 
G

Gregory Toomey

Amanda said:
I am using a Perl program that must process a large number (10**7)
of items. As the program runs, I've noticed that it becomes less
and less efficient in its processing, and at the same time, it
consumes more and more memory. Reading over the program's voluminous
code (not written by me), I can't find what causes this growth in
memory consumption. Are there any tools that could help figure
out where/how this is coming about?

Thanks!

Amanda

--

Why do you assume anything is wrong? "Memory leak" is a MS windows bug;
nothing to do withe Perl.

gtoomey
 
S

Sam Holden

Why do you assume anything is wrong? "Memory leak" is a MS windows bug;
nothing to do withe Perl.

Perl programs can "leak" memory just fine thanks. Circular references are
one way, keeping around data that is no longer required is another.

while (<>) {
push @lines, $_;
print $lines[-1];
}

Will consume more and more memory, under unix even (so it's certainly
nothing to do with MS Windows).

my $something_that_will_one_day_be_false = 1;
while ($something_that_will_one_day_be_false) {
my $f = {};
$f->{f} = $f;
# code that one day makes $something_that_will_one_day_be_false false
}

Leaks memory, under unix even.
 
G

Gregory Toomey

Sam said:
Perl programs can "leak" memory just fine thanks. Circular references are
one way, keeping around data that is no longer required is another.

A MS WIndows "memory leak" occurs when memory is allocated my malloc or
similar, does not relase it, and loses track of it.

Perl knows about what memory it has allocated just fine, but chooses not to
do garbage collection on occasion for performance reasons.

gtoomey
 
S

Sam Holden

A MS WIndows "memory leak" occurs when memory is allocated my malloc or
similar, does not relase it, and loses track of it.

That's one (restrictive) definition of memory leak. However, the OP didn't
use the word "leak" anyway, so I can't see how it matters.

Both of my examples show programs whose memory usage grows over time,
which is what the OP talked about. If you want to introduce restrictive
terms which are irrelevant to the post that's fine, but don't expect
everyone to keep their discussion to your terms.
 
C

ctcgag

Amanda said:
I am using a Perl program that must process a large number (10**7)
of items.

Are all the items processed independently?
As the program runs, I've noticed that it becomes less
and less efficient in its processing, and at the same time, it
consumes more and more memory. Reading over the program's voluminous
code (not written by me), I can't find what causes this growth in
memory consumption. Are there any tools that could help figure
out where/how this is coming about?

I find "next", "return", and "#" to be the most useful tools. Use them
to do a binary-like search of the code. Also, Data::Dumper can be useful:
if you have a few suspect variables in mind, run a "print length Dumper
\%suspect" every now and then. Devel::Leak can also be helpful, mostly if
the program is working with complex data structures or XS.

Finally, you could force a core-dump and use strings or vi to inspect the
guts for familiar looking data.

Xho
 
J

James Willmore

On Mon, 26 Apr 2004 18:18:41 +1000, Gregory Toomey wrote:

[ ... ]
... "Memory leak" is a MS windows bug; nothing to do withe Perl.

Huh? Where on Earth did you hear/read THAT? Hell ... memory leaks have
been around for a LOOONNGG time (before Windows was even an itch in Billy
boy's pants) and *do* exist in Perl :)

Memory leaks are a result of poor programming practices - they just
*appear* to happen more often on Windoze boxes :)

--
Jim

Copyright notice: all code written by the author in this post is
released under the GPL. http://www.gnu.org/licenses/gpl.txt
for more information.

a fortune quote ...
To be is to do. -- I. Kant To do is to be. -- A. Sartre
Yabba-Dabba-Doo! -- F. Flinstone
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,902
Latest member
Elena68X5

Latest Threads

Top