What to do about memory leaks

[Martin Trautmann]

Hi all,

I got a perl script here which takes more memory than expected: It
starts at about 10 MB, which is ok. After half of the job (3000 URL
checks) it got > 500 MB, further increasing up to its termination. I've
seen several segmentation fault errors as soon as the script would end,
but don't know why. It's just a while loop, which is processed again and
again without problems, while there's no more command after the end of
this loop. Maybe the increased memory size is related to this error.

I use strict and warnings, I verified that there was nothing wrong about
the variables and open/close files.

I take extensive use of WWW::Mechanize, without any knowledge what going
on inside.

I used Devel::Leaks in order to get the info that there's much going on -
but I do not know yet what to do about this kind of information.

What's your recommendation for a perl novice how to find out about these
leaks?

Thanks,
Martin

 

[anno4000]

Hi all,

I got a perl script here which takes more memory than expected: It
starts at about 10 MB, which is ok. After half of the job (3000 URL
checks) it got > 500 MB, further increasing up to its termination. I've
seen several segmentation fault errors as soon as the script would end,
but don't know why. It's just a while loop, which is processed again and
again without problems, while there's no more command after the end of
this loop. Maybe the increased memory size is related to this error.

I use strict and warnings, I verified that there was nothing wrong about
the variables and open/close files.

I take extensive use of WWW::Mechanize, without any knowledge what going
on inside.

I used Devel::Leaks in order to get the info that there's much going on -
but I do not know yet what to do about this kind of information.

What's that? I can't find it on CPAN.

What's your recommendation for a perl novice how to find out about these
leaks?

None.

We'd have to see more of your code. Reduce it to a minimal example
that still exhibits the behavior and post that.

Anno

 

[Martin Trautmann]

What's that? I can't find it on CPAN.

The module itself is Devel::Leak
http://search.cpan.org/~ni-s/Devel-Leak-0.03/

None.

We'd have to see more of your code. Reduce it to a minimal example
that still exhibits the behavior and post that.

I don have a short example at hand - it's an interactive web site where
I have to step along some inputs up to the result, then start again.

I'll try to reproduce it while submitting it to a local server and find
out more about it...

The script is not that long, if I should mail it to you completely.

Thanks,
Martin

 

[Martin Trautmann]

Check line 17.

[http://search.cpan.org/~ni-s/Devel-Leak-0.03/|Devel::Leak]s

my perl was NOT build with -DDEBUGGING and compilation warns me about an
existing sv_dump. I don't know what I may expect from this Utitlity - I
get the memory warnings, but I don't know yet what to do with them.

What should I learn from the result of Devel::Leak::CheckSV($handle);

new 0x18ebd08 :

Thanks,
Martin

 

[anno4000]

The module itself is Devel::Leak
http://search.cpan.org/~ni-s/Devel-Leak-0.03/

Aha. Please be careful, typos like that frustrate those who try to help
you.

I don have a short example at hand - it's an interactive web site where
I have to step along some inputs up to the result, then start again.

I'll try to reproduce it while submitting it to a local server and find
out more about it...

The script is not that long, if I should mail it to you completely.

No, not to me personally.

Read the posting guidelines that come up here twice a week. It has
recommendations on how to prepare an example script. When you have
a short demo of the problem, post it here. You'll probably find
the solution while preparing the demo, it often happens.

Anno

 

[Martin Trautmann]

Aha. Please be careful, typos like that frustrate those who try to help
you.

Sorry about that - I may be used too much to wiki styles, changing suffix or
conjugation for better readability

Read the posting guidelines that come up here twice a week. It has
recommendations on how to prepare an example script. When you have
a short demo of the problem, post it here. You'll probably find
the solution while preparing the demo, it often happens.

Ok, I´ll try a combination of a cgi perl script and a mechanize script,
asking the same page again and again and will make these scripts more
and more complex until they will show the leaking - if they do.


I hoped that there are other modules that would give more information
what's going on. Devel::Leak does show me that something is leaking -
now I´d like to see further info what to do about it. This package from
2000 is somehow dead and unmaintained (unfortunately the owner also died
himself), while there's little docu for the novice what to do with this
kind of info.

 

[baalbek]

I take extensive use of WWW::Mechanize, without any knowledge what going
on inside.

Do remember that WWW::Mechanize caches the fetched web pages by default,
thus using a lot of memory if you access a lot of web pages.

In order not to cache the downloaded result, use this constructor:

my $agent = WWW::Mechanize->new(stack_depth => 1);

that is, stack_depth is set to one, so no caching.

Hope this helps.

~Christian

 

[Martin Trautmann]

In order not to cache the downloaded result, use this constructor:

my $agent = WWW::Mechanize->new(stack_depth => 1);

Thanks!
Martin

 

[Clenna Lumina]

_
Martin Trautmann ([email protected]) wrote on VLXXI September MCMXCIII in
<URL)
() Hi all,
()
() I got a perl script here which takes more memory than expected: It
() starts at about 10 MB, which is ok. After half of the job (3000
URL () checks) it got > 500 MB, further increasing up to its
termination. I've () seen several segmentation fault errors as soon
as the script would end, () but don't know why. It's just a while
loop, which is processed again and () again without problems, while
there's no more command after the end of () this loop. Maybe the
increased memory size is related to this error. ()
() I use strict and warnings, I verified that there was nothing
wrong about () the variables and open/close files.
()
() I take extensive use of WWW::Mechanize, without any knowledge
what going () on inside.
()
() I used Devel::Leaks in order to get the info that there's much
going on - () but I do not know yet what to do about this kind of
information. ()
() What's your recommendation for a perl novice how to find out
about these () leaks?


Check line 17.

Check quoting delimiters.