Image::Magick memory leak question

S

Stan Brown

OK, I've got this memory leak down to a 3 line example:


my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
$image->Read($l_tmpfile);
undef $image;

In a loop leaks memory.

Can anyone tell me if I'm doing somethign wrong here?
 
B

Ben Morrow

Stan Brown said:
OK, I've got this memory leak down to a 3 line example:


my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
$image->Read($l_tmpfile);
undef $image;

In a loop leaks memory.

Can anyone tell me if I'm doing somethign wrong here?

Not by my reading of the Image::Magick docs. I never have liked
imagemagick: the couple of looks I took at the source a while ago did
*not* inspire me with confidence. I could find no record at all, for
most of the API, of who was responsible for freeing the various bits
of memory.

When I'm doing something like this I usually use system() and the
NetPBM utilities; you may prefer GD or Image::Imlib2.

Alternatively, if you can find a convenient place in your loop to
re-exec() yourself, that will cut the leaks short.

Ben
 
S

Stan Brown

Not by my reading of the Image::Magick docs. I never have liked
imagemagick: the couple of looks I took at the source a while ago did
*not* inspire me with confidence. I could find no record at all, for
most of the API, of who was responsible for freeing the various bits
of memory.
When I'm doing something like this I usually use system() and the
NetPBM utilities; you may prefer GD or Image::Imlib2.
Alternatively, if you can find a convenient place in your loop to
re-exec() yourself, that will cut the leaks short.

Thanks, you have been a big help on this!

I'm a bit hesitant to uses systen() as in the long term, I would like to be
able to capture images at a failry high rate of speed.

In addition, I spent about 10 minutes trying to figure out how to do the
equivelant commnad from the comand line using the Imagemagick tools, and
never did get it to work, so building the system() call for that would eb
problematic.

But, as you have pointed out, I do have other tools to choose from. I think
thta I will satrt examining them.

BTW, adding @$image = ();, right before the undef $image; call seems to
have drasticly lowered the ammount of memory that it's leaking.

Does that make any sens at all? I would hhave thoughtthat the undef would
have deleted all parts of the $image object. Am I wrong?
 
S

Stan Brown

With some module objects, who knows? You are undefining a reference to the
object, which cetainly frees the object (the array) if there are no other
references to it. However, it may be that the object's code keeps some hidden
reference to instances of itself (circular?) which then preserves all the
elements in the array when you undefine it. When you instead set the array
contents to none, you dereference and destroy all the actual elements in the
array. So the amount of leak is limited to just the empty array object's memory
usage alone, probably a much smaller leak.. I hope this makes sense. In any
case, I suggest considering saving required states, exiting, and re-running
the script itself periodically, to keep any remaining leakiness under control.
Yes, it make perfect sense. I was naievly believing that since I had left
the subroutine, and the $image object was "local" to it, that all traces of
it would eb retruned to the free pool, as would be done, for non malloced
memory in C.

This seems to be a pretty basic weakness of perl. I wonder if perl6
addresses this?
 
M

Martien Verbruggen

[snip of other material, which is not relevant to this point]
BTW, adding @$image = ();, right before the undef $image; call seems to
have drasticly lowered the ammount of memory that it's leaking.

It shouldn't make a difference.

An ImageMagick object is a blessed reference to an array of
ImageMagick objects. If you undef the object, then all other elements
in the data structure should automatically be freed up. getting rid of
them explicitly by doing @$im = (); _should_ not make a difference.

If it does, then there is a bug in ImageMagick, and the authors should
be told.

When you report the bug, make sure you try the latest version, and
include information about the versions of Perl, IM and your
platform/OS. It would also be a good idea to mention how you measure
that there is a memory leak, and how large it is.

Martien
 
B

Ben Morrow

Stan Brown said:
In <[email protected]>
(e-mail address removed) (William Herrera) writes:
Yes, it make perfect sense. I was naievly believing that since I had left
the subroutine, and the $image object was "local" to it, that all traces of
it would eb retruned to the free pool, as would be done, for non malloced
memory in C.

This seems to be a pretty basic weakness of perl. I wonder if perl6
addresses this?

No, this is a basic weakness of ImageMagick, or of C. Perl will
correctly tell the Image::Magick object to free itself at the end of
the scope: if it fails to do so correctly, this is not Perl's fault.

Ben
 
S

Stan Brown

[snip of other material, which is not relevant to this point]
BTW, adding @$image = ();, right before the undef $image; call seems to
have drasticly lowered the ammount of memory that it's leaking.
It shouldn't make a difference.
An ImageMagick object is a blessed reference to an array of
ImageMagick objects. If you undef the object, then all other elements
in the data structure should automatically be freed up. getting rid of
them explicitly by doing @$im = (); _should_ not make a difference.
If it does, then there is a bug in ImageMagick, and the authors should
be told.
When you report the bug, make sure you try the latest version, and
include information about the versions of Perl, IM and your
platform/OS. It would also be a good idea to mention how you measure
that there is a memory leak, and how large it is.

Thanks for the advice.

Unfortunately, I can't get the latest version to build on the target
machine (Debian "testing").
 
S

Stan Brown

No, this is a basic weakness of ImageMagick, or of C. Perl will
correctly tell the Image::Magick object to free itself at the end of
the scope: if it fails to do so correctly, this is not Perl's fault.

Well, OK. I guess I was thinking in a perfect world, perl would be in
charge of freeing all out of scope objects. Isn't that one og Java's claims
to fame? Or not, I know littel about Java.
 
B

Ben Morrow

Stan Brown said:
Well, OK. I guess I was thinking in a perfect world, perl would be in
charge of freeing all out of scope objects. Isn't that one og Java's claims
to fame? Or not, I know littel about Java.

Yes it is, and also one of Perl's. However, most of Image::Magick is
not written in Perl, but in C, so it can leak. Similarly, stuff
written for Java using the JNI (the Java equivalent of XS) can leak,
as well.

Ben
 
M

Martien Verbruggen

Well, OK. I guess I was thinking in a perfect world, perl would be in
charge of freeing all out of scope objects. Isn't that one og Java's claims
to fame? Or not, I know littel about Java.

Perl does reference counting, and will "free" all objects that have no
references to them anymore. Image::Magick objects are not pure Perl
objects, however, but an XS wrapper around a C library. The memory
leak can be anywhere inside that XS or C code.

The ImageMagick library has had some memory leaks internally in the
past (which is why I advised using the latest library to see whether
the problem persists), which generally get fixed quite quickly.

While Image::Magick objects look like regular references to Perl
arrays, they're not. Perl only controls the freeing of memory up to
the freeing of the Image::Magick blessed object reference itself.
Everything after that is dealt with in the Image::Magick XS code, and
the ImageMagick library internally, and is (largely) out of the
control of perl, unless the author uses Perl structures in that code
and correctly keeps track of references.

It is quite easy to plug in a C module to perl that leaks memory.
However, that doesn't make it the fault of Perl that there is a leak.

The following little program (which will only report memory correctly
on Linux, really) illustrates that. it uses Inline::C instead of XS,
but the effect is the same: The code that allocates memory and doesn't
hand control of it over to perl, leaks, the code that does allow perl
to clean up, doesn't:

#!/usr/local/bin/perl
use warnings;
use strict;

use Inline 'C';

for my $i (0 .. 9)
{
my $foo = gimmePerlmemory();
print reportmem(), ", ";
}
print "\n";
for my $i (0 .. 9)
{
my $foo = gimmememory();
print reportmem(), ", ";
}
print "\n";

sub reportmem
{
open my $ps, "/proc/self/stat" or die;
seek $ps, 0, 0;
(split ' ', scalar <$ps>)[22] / 1024;
}

__END__
__C__
#include <stdlib.h>

char * gimmememory()
{
return malloc(4096);
}
SV * gimmePerlmemory()
{
return newSVpv("", 4096);
}



And the output is:

5940, 5948, 5948, 5948, 5948, 5948, 5948, 5948, 5948, 5948, 5948,
5948, 5952, 5956, 5960, 5964, 5968, 5972, 5976, 5980, 5984, 5988,

As you can see, the 4 kB of malloc'ed memory is being leaked, while
the 4 kB of newSV memory is neatly being reclaimed. The same goes for
external modules. If they don't clean up their own memory, or they
don't allow Perl to do it, the memory will be leaked.

Martien
 
S

Stan Brown

Perl does reference counting, and will "free" all objects that have no
references to them anymore. Image::Magick objects are not pure Perl
objects, however, but an XS wrapper around a C library. The memory
leak can be anywhere inside that XS or C code.
return newSVpv("", 4096);

I'm not familar with the library call. Is it provided by a perl glue
libarary?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top