Image::Magick memory leak question

Discussion in 'Perl Misc' started by Stan Brown, Nov 19, 2003.

  1. Stan Brown

    Stan Brown Guest

    OK, I've got this memory leak down to a 3 line example:


    my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
    $image->Read($l_tmpfile);
    undef $image;

    In a loop leaks memory.

    Can anyone tell me if I'm doing somethign wrong here?
    --
    "They that would give up essential liberty for temporary safety deserve
    neither liberty nor safety."
    -- Benjamin Franklin
    Stan Brown, Nov 19, 2003
    #1
    1. Advertising

  2. Stan Brown

    Ben Morrow Guest

    Stan Brown <> wrote:
    > OK, I've got this memory leak down to a 3 line example:
    >
    >
    > my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
    > $image->Read($l_tmpfile);
    > undef $image;
    >
    > In a loop leaks memory.
    >
    > Can anyone tell me if I'm doing somethign wrong here?


    Not by my reading of the Image::Magick docs. I never have liked
    imagemagick: the couple of looks I took at the source a while ago did
    *not* inspire me with confidence. I could find no record at all, for
    most of the API, of who was responsible for freeing the various bits
    of memory.

    When I'm doing something like this I usually use system() and the
    NetPBM utilities; you may prefer GD or Image::Imlib2.

    Alternatively, if you can find a convenient place in your loop to
    re-exec() yourself, that will cut the leaks short.

    Ben

    --
    perl -e'print map {/.(.)/s} sort unpack "a2"x26, pack "N"x13,
    qw/1632265075 1651865445 1685354798 1696626283 1752131169 1769237618
    1801808488 1830841936 1886550130 1914728293 1936225377 1969451372
    2047502190/' #
    Ben Morrow, Nov 19, 2003
    #2
    1. Advertising

  3. Stan Brown

    Stan Brown Guest

    In <bpgf47$fei$> Ben Morrow <> writes:


    >Stan Brown <> wrote:
    >> OK, I've got this memory leak down to a 3 line example:
    >>
    >>
    >> my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
    >> $image->Read($l_tmpfile);
    >> undef $image;
    >>
    >> In a loop leaks memory.
    >>
    >> Can anyone tell me if I'm doing somethign wrong here?


    >Not by my reading of the Image::Magick docs. I never have liked
    >imagemagick: the couple of looks I took at the source a while ago did
    >*not* inspire me with confidence. I could find no record at all, for
    >most of the API, of who was responsible for freeing the various bits
    >of memory.


    >When I'm doing something like this I usually use system() and the
    >NetPBM utilities; you may prefer GD or Image::Imlib2.


    >Alternatively, if you can find a convenient place in your loop to
    >re-exec() yourself, that will cut the leaks short.


    Thanks, you have been a big help on this!

    I'm a bit hesitant to uses systen() as in the long term, I would like to be
    able to capture images at a failry high rate of speed.

    In addition, I spent about 10 minutes trying to figure out how to do the
    equivelant commnad from the comand line using the Imagemagick tools, and
    never did get it to work, so building the system() call for that would eb
    problematic.

    But, as you have pointed out, I do have other tools to choose from. I think
    thta I will satrt examining them.

    BTW, adding @$image = ();, right before the undef $image; call seems to
    have drasticly lowered the ammount of memory that it's leaking.

    Does that make any sens at all? I would hhave thoughtthat the undef would
    have deleted all parts of the $image object. Am I wrong?

    --
    "They that would give up essential liberty for temporary safety deserve
    neither liberty nor safety."
    -- Benjamin Franklin
    Stan Brown, Nov 19, 2003
    #3
  4. Stan Brown

    Stan Brown Guest

    In <> (William Herrera) writes:

    >On Wed, 19 Nov 2003 19:57:31 +0000 (UTC), Stan Brown <> wrote:
    >>
    >>BTW, adding @$image = ();, right before the undef $image; call seems to
    >>have drasticly lowered the ammount of memory that it's leaking.
    >>
    >>Does that make any sens at all? I would hhave thoughtthat the undef would
    >>have deleted all parts of the $image object. Am I wrong?
    >>


    >With some module objects, who knows? You are undefining a reference to the
    >object, which cetainly frees the object (the array) if there are no other
    >references to it. However, it may be that the object's code keeps some hidden
    >reference to instances of itself (circular?) which then preserves all the
    >elements in the array when you undefine it. When you instead set the array
    >contents to none, you dereference and destroy all the actual elements in the
    >array. So the amount of leak is limited to just the empty array object's memory
    >usage alone, probably a much smaller leak.. I hope this makes sense. In any
    >case, I suggest considering saving required states, exiting, and re-running
    >the script itself periodically, to keep any remaining leakiness under control.

    Yes, it make perfect sense. I was naievly believing that since I had left
    the subroutine, and the $image object was "local" to it, that all traces of
    it would eb retruned to the free pool, as would be done, for non malloced
    memory in C.

    This seems to be a pretty basic weakness of perl. I wonder if perl6
    addresses this?

    --
    "They that would give up essential liberty for temporary safety deserve
    neither liberty nor safety."
    -- Benjamin Franklin
    Stan Brown, Nov 20, 2003
    #4
  5. On Wed, 19 Nov 2003 19:57:31 +0000 (UTC),
    Stan Brown <> wrote:
    > In <bpgf47$fei$> Ben Morrow <> writes:


    >>> my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
    >>> $image->Read($l_tmpfile);
    >>> undef $image;
    >>>
    >>> In a loop leaks memory.


    [snip of other material, which is not relevant to this point]

    > BTW, adding @$image = ();, right before the undef $image; call seems to
    > have drasticly lowered the ammount of memory that it's leaking.


    It shouldn't make a difference.

    An ImageMagick object is a blessed reference to an array of
    ImageMagick objects. If you undef the object, then all other elements
    in the data structure should automatically be freed up. getting rid of
    them explicitly by doing @$im = (); _should_ not make a difference.

    If it does, then there is a bug in ImageMagick, and the authors should
    be told.

    When you report the bug, make sure you try the latest version, and
    include information about the versions of Perl, IM and your
    platform/OS. It would also be a good idea to mention how you measure
    that there is a memory leak, and how large it is.

    Martien
    --
    |
    Martien Verbruggen | If at first you don't succeed, destroy all
    Trading Post Australia | evidence that you tried.
    |
    Martien Verbruggen, Nov 20, 2003
    #5
  6. Stan Brown

    Ben Morrow Guest

    Stan Brown <> wrote:
    > In <>
    > (William Herrera) writes:
    > Yes, it make perfect sense. I was naievly believing that since I had left
    > the subroutine, and the $image object was "local" to it, that all traces of
    > it would eb retruned to the free pool, as would be done, for non malloced
    > memory in C.
    >
    > This seems to be a pretty basic weakness of perl. I wonder if perl6
    > addresses this?


    No, this is a basic weakness of ImageMagick, or of C. Perl will
    correctly tell the Image::Magick object to free itself at the end of
    the scope: if it fails to do so correctly, this is not Perl's fault.

    Ben

    --
    If I were a butterfly I'd live for a day, / I would be free, just blowing away.
    This cruel country has driven me down / Teased me and lied, teased me and lied.
    I've only sad stories to tell to this town: / My dreams have withered and died.
    <=>=<=>=<=>=<=>=<=>=<=>=<=>=<=>=<=>=<=>=<=> (Kate Rusby)
    Ben Morrow, Nov 21, 2003
    #6
  7. Stan Brown

    Stan Brown Guest

    In <> Martien Verbruggen <> writes:

    >On Wed, 19 Nov 2003 19:57:31 +0000 (UTC),
    > Stan Brown <> wrote:
    >> In <bpgf47$fei$> Ben Morrow <> writes:


    >>>> my $image = Image::Magick->new(magick=>'GIF',font=>'clean');
    >>>> $image->Read($l_tmpfile);
    >>>> undef $image;
    >>>>
    >>>> In a loop leaks memory.


    >[snip of other material, which is not relevant to this point]


    >> BTW, adding @$image = ();, right before the undef $image; call seems to
    >> have drasticly lowered the ammount of memory that it's leaking.


    >It shouldn't make a difference.


    >An ImageMagick object is a blessed reference to an array of
    >ImageMagick objects. If you undef the object, then all other elements
    >in the data structure should automatically be freed up. getting rid of
    >them explicitly by doing @$im = (); _should_ not make a difference.


    >If it does, then there is a bug in ImageMagick, and the authors should
    >be told.


    >When you report the bug, make sure you try the latest version, and
    >include information about the versions of Perl, IM and your
    >platform/OS. It would also be a good idea to mention how you measure
    >that there is a memory leak, and how large it is.


    Thanks for the advice.

    Unfortunately, I can't get the latest version to build on the target
    machine (Debian "testing").
    --
    "They that would give up essential liberty for temporary safety deserve
    neither liberty nor safety."
    -- Benjamin Franklin
    Stan Brown, Nov 24, 2003
    #7
  8. Stan Brown

    Stan Brown Guest

    In <bpjl3q$qps$> Ben Morrow <> writes:


    >Stan Brown <> wrote:
    >> In <>
    >> (William Herrera) writes:
    >> Yes, it make perfect sense. I was naievly believing that since I had left
    >> the subroutine, and the $image object was "local" to it, that all traces of
    >> it would eb retruned to the free pool, as would be done, for non malloced
    >> memory in C.
    >>
    >> This seems to be a pretty basic weakness of perl. I wonder if perl6
    >> addresses this?


    >No, this is a basic weakness of ImageMagick, or of C. Perl will
    >correctly tell the Image::Magick object to free itself at the end of
    >the scope: if it fails to do so correctly, this is not Perl's fault.


    Well, OK. I guess I was thinking in a perfect world, perl would be in
    charge of freeing all out of scope objects. Isn't that one og Java's claims
    to fame? Or not, I know littel about Java.

    --
    "They that would give up essential liberty for temporary safety deserve
    neither liberty nor safety."
    -- Benjamin Franklin
    Stan Brown, Nov 24, 2003
    #8
  9. Stan Brown

    Ben Morrow Guest

    Stan Brown <> wrote:
    > In <bpjl3q$qps$> Ben Morrow
    > <> writes:
    > >Stan Brown <> wrote:
    > >> This seems to be a pretty basic weakness of perl. I wonder if perl6
    > >> addresses this?

    >
    > >No, this is a basic weakness of ImageMagick, or of C. Perl will
    > >correctly tell the Image::Magick object to free itself at the end of
    > >the scope: if it fails to do so correctly, this is not Perl's fault.

    >
    > Well, OK. I guess I was thinking in a perfect world, perl would be in
    > charge of freeing all out of scope objects. Isn't that one og Java's claims
    > to fame? Or not, I know littel about Java.


    Yes it is, and also one of Perl's. However, most of Image::Magick is
    not written in Perl, but in C, so it can leak. Similarly, stuff
    written for Java using the JNI (the Java equivalent of XS) can leak,
    as well.

    Ben

    --
    Although few may originate a policy, we are all able to judge it.
    - Pericles of Athens, c.430 B.C.
    Ben Morrow, Nov 24, 2003
    #9
  10. On Mon, 24 Nov 2003 14:15:54 +0000 (UTC),
    Stan Brown <> wrote:
    > In <bpjl3q$qps$> Ben Morrow <> writes:
    >
    >
    >>Stan Brown <> wrote:
    >>> In <>
    >>> (William Herrera) writes:
    >>> Yes, it make perfect sense. I was naievly believing that since I had left
    >>> the subroutine, and the $image object was "local" to it, that all traces of
    >>> it would eb retruned to the free pool, as would be done, for non malloced
    >>> memory in C.
    >>>
    >>> This seems to be a pretty basic weakness of perl. I wonder if perl6
    >>> addresses this?

    >
    >>No, this is a basic weakness of ImageMagick, or of C. Perl will
    >>correctly tell the Image::Magick object to free itself at the end of
    >>the scope: if it fails to do so correctly, this is not Perl's fault.

    >
    > Well, OK. I guess I was thinking in a perfect world, perl would be in
    > charge of freeing all out of scope objects. Isn't that one og Java's claims
    > to fame? Or not, I know littel about Java.


    Perl does reference counting, and will "free" all objects that have no
    references to them anymore. Image::Magick objects are not pure Perl
    objects, however, but an XS wrapper around a C library. The memory
    leak can be anywhere inside that XS or C code.

    The ImageMagick library has had some memory leaks internally in the
    past (which is why I advised using the latest library to see whether
    the problem persists), which generally get fixed quite quickly.

    While Image::Magick objects look like regular references to Perl
    arrays, they're not. Perl only controls the freeing of memory up to
    the freeing of the Image::Magick blessed object reference itself.
    Everything after that is dealt with in the Image::Magick XS code, and
    the ImageMagick library internally, and is (largely) out of the
    control of perl, unless the author uses Perl structures in that code
    and correctly keeps track of references.

    It is quite easy to plug in a C module to perl that leaks memory.
    However, that doesn't make it the fault of Perl that there is a leak.

    The following little program (which will only report memory correctly
    on Linux, really) illustrates that. it uses Inline::C instead of XS,
    but the effect is the same: The code that allocates memory and doesn't
    hand control of it over to perl, leaks, the code that does allow perl
    to clean up, doesn't:

    #!/usr/local/bin/perl
    use warnings;
    use strict;

    use Inline 'C';

    for my $i (0 .. 9)
    {
    my $foo = gimmePerlmemory();
    print reportmem(), ", ";
    }
    print "\n";
    for my $i (0 .. 9)
    {
    my $foo = gimmememory();
    print reportmem(), ", ";
    }
    print "\n";

    sub reportmem
    {
    open my $ps, "/proc/self/stat" or die;
    seek $ps, 0, 0;
    (split ' ', scalar <$ps>)[22] / 1024;
    }

    __END__
    __C__
    #include <stdlib.h>

    char * gimmememory()
    {
    return malloc(4096);
    }
    SV * gimmePerlmemory()
    {
    return newSVpv("", 4096);
    }



    And the output is:

    5940, 5948, 5948, 5948, 5948, 5948, 5948, 5948, 5948, 5948, 5948,
    5948, 5952, 5956, 5960, 5964, 5968, 5972, 5976, 5980, 5984, 5988,

    As you can see, the 4 kB of malloc'ed memory is being leaked, while
    the 4 kB of newSV memory is neatly being reclaimed. The same goes for
    external modules. If they don't clean up their own memory, or they
    don't allow Perl to do it, the memory will be leaked.

    Martien
    --
    |
    Martien Verbruggen | In the fight between you and the world, back
    Trading Post Australia | the world - Franz Kafka
    |
    Martien Verbruggen, Nov 24, 2003
    #10
  11. Stan Brown

    Stan Brown Guest

    In <> Martien Verbruggen <> writes:

    >On Mon, 24 Nov 2003 14:15:54 +0000 (UTC),
    > Stan Brown <> wrote:
    >> In <bpjl3q$qps$> Ben Morrow <> writes:
    >>
    >>
    >>>Stan Brown <> wrote:
    >>>> In <>
    >>>> (William Herrera) writes:
    >>>> Yes, it make perfect sense. I was naievly believing that since I had left
    >>>> the subroutine, and the $image object was "local" to it, that all traces of
    >>>> it would eb retruned to the free pool, as would be done, for non malloced
    >>>> memory in C.
    >>>>
    >>>> This seems to be a pretty basic weakness of perl. I wonder if perl6
    >>>> addresses this?

    >>
    >>>No, this is a basic weakness of ImageMagick, or of C. Perl will
    >>>correctly tell the Image::Magick object to free itself at the end of
    >>>the scope: if it fails to do so correctly, this is not Perl's fault.

    >>
    >> Well, OK. I guess I was thinking in a perfect world, perl would be in
    >> charge of freeing all out of scope objects. Isn't that one og Java's claims
    >> to fame? Or not, I know littel about Java.


    >Perl does reference counting, and will "free" all objects that have no
    >references to them anymore. Image::Magick objects are not pure Perl
    >objects, however, but an XS wrapper around a C library. The memory
    >leak can be anywhere inside that XS or C code.


    > return newSVpv("", 4096);


    I'm not familar with the library call. Is it provided by a perl glue
    libarary?

    --
    "They that would give up essential liberty for temporary safety deserve
    neither liberty nor safety."
    -- Benjamin Franklin
    Stan Brown, Nov 26, 2003
    #11
  12. Stan Brown

    Ben Morrow Guest

    Stan Brown <> wrote:
    > In <> Martien Verbruggen
    > <> writes:
    > >
    > > return newSVpv("", 4096);

    >
    > I'm not familar with the library call. Is it provided by a perl glue
    > libarary?


    perldoc perlapi

    --
    $.=1;*g=sub{print@_};sub r($$\$){my($w,$x,$y)=@_;for(keys%$x){/main/&&next;*p=$
    $x{$_};/(\w)::$/&&(r($w.$1,$x.$_,$y),next);$y eq\$p&&&g("$w$_")}};sub t{for(@_)
    {$f&&($_||&g(" "));$f=1;r"","::",$_;$_&&&g(chr(0012))}};t #
    $J::u::s::t, $a::n::eek:::t::h::e::r, $P::e::r::l, $h::a::c::k::e::r, $.
    Ben Morrow, Nov 26, 2003
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eric McDaniel

    Image::Magick and Tk::Photo

    Eric McDaniel, Aug 30, 2003, in forum: Perl
    Replies:
    2
    Views:
    1,582
    Joseph Brenner
    Sep 20, 2003
  2. Weapons of Mass Destruction

    Trying to get Image::Magick to work on a remote server

    Weapons of Mass Destruction, Nov 15, 2003, in forum: Perl
    Replies:
    3
    Views:
    530
    Gunnar Hjalmarsson
    Nov 15, 2003
  3. Tommy Butler

    Image::Magick "geometry" atribute ??

    Tommy Butler, Jul 20, 2003, in forum: Perl Misc
    Replies:
    2
    Views:
    166
  4. Rob Bell
    Replies:
    4
    Views:
    207
    Martien Verbruggen
    Sep 10, 2003
  5. Jack
    Replies:
    2
    Views:
    204
    John W. Krahn
    Jan 25, 2008
Loading...

Share This Page