Leak in Win32::ChangeNotify?

Discussion in 'Perl Misc' started by Thomas Kratz, Dec 28, 2004.

  1. Thomas Kratz

    Thomas Kratz Guest

    The code below is a simple example of using Win32::ChangeNotify. AFAIK
    this is how it should be used according to the docs.
    If you run this you will see a steady increase in the used nonpage pool
    memory of the perl process in the Win32 performance monitor (Category
    'Process', counter 'Pool Nonpaged Bytes'). Changing the file will result
    in the notification message printed to screen as well as in more memory
    leaking. After the second change it will go berserk and if you don't
    terminate the process quickly it will eat up all the systems nonpaged pool
    memory and thus cause serious trouble elsewhere in the system.

    Moving the object creation into the while loop and deleting the line with
    '$notify->reset()' works around the problem.

    Can somebody confirm this? If yes I will file a bug report on rt.cpan.org.

    Tested with perl 5.8.4 (custom build) and libwin 0.19.1
    (Win32::ChangeNotify 1.02) under WinXP and Win2003 Server.

    Thomas

    === code start ===
    use strict;
    use warnings;

    $| = 1;

    use File::Spec::Functions qw/catfile/;
    use Win32::ChangeNotify;

    my $dir = 'd:/tmp/perl/chng';
    my $notify_fn = catfile($dir, 'bla.txt');

    my $last = 0;
    my $notify = Win32::ChangeNotify->new($dir, 0, 'LAST_WRITE');
    die 'cannot create notify object'
    unless ref($notify) eq 'Win32::ChangeNotify';

    while ( 1 ) {

    my $rc = $notify->wait(100);
    $notify->reset();

    last if $rc == -1;

    my @stat = stat($notify_fn);

    if ( $rc == 1 and $stat[9] > $last ) {

    print "$notify_fn has changed\n";

    $last = $stat[9];
    }
    }
    === code end ===

    --
    $/=$,,$_=<DATA>,s,(.*),$1,see;__END__
    s,^(.*\043),,mg,@_=map{[split'']}split;{#>J~.>_an~>>e~......>r~
    $_=$_[$%][$"];y,<~>^,-++-,?{$/=--$|?'"':#..u.t.^.o.P.r.>ha~.e..
    '%',s,(.),\$$/$1=1,,$;=$_}:/\w/?{y,_, ,,#..>s^~ht<._..._..c....
    print}:y,.,,||last,,,,,,$_=$;;eval,redo}#.....>.e.r^.>l^..>k^.-
    Thomas Kratz, Dec 28, 2004
    #1
    1. Advertising

  2. Thomas Kratz

    Matt Garrish Guest

    "Thomas Kratz" <> wrote in message
    news:41d170e7$0$776$-rheiner.de...
    > The code below is a simple example of using Win32::ChangeNotify. AFAIK
    > this is how it should be used according to the docs.
    > If you run this you will see a steady increase in the used nonpage pool
    > memory of the perl process in the Win32 performance monitor (Category
    > 'Process', counter 'Pool Nonpaged Bytes'). Changing the file will result
    > in the notification message printed to screen as well as in more memory
    > leaking. After the second change it will go berserk and if you don't
    > terminate the process quickly it will eat up all the systems nonpaged pool
    > memory and thus cause serious trouble elsewhere in the system.
    >
    > Moving the object creation into the while loop and deleting the line with
    > '$notify->reset()' works around the problem.
    >
    > Can somebody confirm this? If yes I will file a bug report on rt.cpan.org.
    >


    Strange. The first time I ran the script the nonpaged memory slowly grew
    (nothing like what you describe, though). I let it run up from ~10100K to
    ~11500K before killing script (at which point it dropped right back to where
    it started), but on subsequent restarts could not get the script to leak
    again (it stayed steady at ~10100 no matter what I did to the file).

    This was on XP Pro using v5.8.2.

    Matt
    Matt Garrish, Dec 29, 2004
    #2
    1. Advertising

  3. Thomas Kratz

    Thomas Kratz Guest

    Matt Garrish wrote:
    >
    > Strange. The first time I ran the script the nonpaged memory slowly grew
    > (nothing like what you describe, though). I let it run up from ~10100K to
    > ~11500K before killing script (at which point it dropped right back to where
    > it started), but on subsequent restarts could not get the script to leak
    > again (it stayed steady at ~10100 no matter what I did to the file).
    >
    > This was on XP Pro using v5.8.2.


    Do you really mean 10100KB. The value should be about 1-2K. 10MB of
    nonpaged memory is way too much.

    I get the same results as yesterday on various machines. The strange thing
    is, that after the second modification of the file it will take some time
    before the nonepaged pool usage freaks out.
    I think I will have a look into the xs code of Win32::ChangeNotify and
    Win32::IPC on the weekend. Perhaps I'll get an idea, what could go wrong here.

    Always creating a new object works fine as a workaround, so there is no
    urgent need for a fix.

    Thomas

    --
    $/=$,,$_=<DATA>,s,(.*),$1,see;__END__
    s,^(.*\043),,mg,@_=map{[split'']}split;{#>J~.>_an~>>e~......>r~
    $_=$_[$%][$"];y,<~>^,-++-,?{$/=--$|?'"':#..u.t.^.o.P.r.>ha~.e..
    '%',s,(.),\$$/$1=1,,$;=$_}:/\w/?{y,_, ,,#..>s^~ht<._..._..c....
    print}:y,.,,||last,,,,,,$_=$;;eval,redo}#.....>.e.r^.>l^..>k^.-
    Thomas Kratz, Dec 29, 2004
    #3
  4. Thomas Kratz

    Thomas Kratz Guest

    Re: Leak in Win32::ChangeNotify? [solved]

    Thomas Kratz wrote:

    > === code start ===
    > use strict;
    > use warnings;
    >
    > $| = 1;
    >
    > use File::Spec::Functions qw/catfile/;
    > use Win32::ChangeNotify;
    >
    > my $dir = 'd:/tmp/perl/chng';
    > my $notify_fn = catfile($dir, 'bla.txt');
    >
    > my $last = 0;
    > my $notify = Win32::ChangeNotify->new($dir, 0, 'LAST_WRITE');
    > die 'cannot create notify object'
    > unless ref($notify) eq 'Win32::ChangeNotify';
    >
    > while ( 1 ) {
    >
    > my $rc = $notify->wait(100);
    > $notify->reset();
    >
    > last if $rc == -1;
    >
    > my @stat = stat($notify_fn);
    >
    > if ( $rc == 1 and $stat[9] > $last ) {
    >
    > print "$notify_fn has changed\n";
    >
    > $last = $stat[9];
    > }
    > }
    > === code end ===


    The problem was calling reset on the object even if there was no signalled
    change event. Moving the "$notify->reset();" to the end of the "if ( $rc
    == 1..." block, ended the leaking.

    After looking at the XS code, it was clear that I used the module the
    wrong way. Each call to reset() causes a Win32 API call to
    FindNextChangeNotification and that only makes sense if the last Find was
    successful.

    So: no bug, wrong usage!

    Thomas

    --
    $/=$,,$_=<DATA>,s,(.*),$1,see;__END__
    s,^(.*\043),,mg,@_=map{[split'']}split;{#>J~.>_an~>>e~......>r~
    $_=$_[$%][$"];y,<~>^,-++-,?{$/=--$|?'"':#..u.t.^.o.P.r.>ha~.e..
    '%',s,(.),\$$/$1=1,,$;=$_}:/\w/?{y,_, ,,#..>s^~ht<._..._..c....
    print}:y,.,,||last,,,,,,$_=$;;eval,redo}#.....>.e.r^.>l^..>k^.-
    Thomas Kratz, Dec 29, 2004
    #4
  5. Thomas Kratz

    Matt Garrish Guest

    "Thomas Kratz" <> wrote in message
    news:41d283b9$0$782$-rheiner.de...
    > Matt Garrish wrote:
    >>
    >> Strange. The first time I ran the script the nonpaged memory slowly grew
    >> (nothing like what you describe, though). I let it run up from ~10100K to
    >> ~11500K before killing script (at which point it dropped right back to
    >> where it started), but on subsequent restarts could not get the script to
    >> leak again (it stayed steady at ~10100 no matter what I did to the file).
    >>
    >> This was on XP Pro using v5.8.2.

    >
    > Do you really mean 10100KB. The value should be about 1-2K. 10MB of
    > nonpaged memory is way too much.
    >


    Yup, but I believe we were looking at different things. I was watching the
    nonpaged kernel memory in the task manager under the Performance tab. It
    generally sits around 10100 - 10300 when the machine is idle.

    Matt
    Matt Garrish, Dec 29, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Daniel Berger

    [ANN] win32-changenotify

    Daniel Berger, May 8, 2004, in forum: Ruby
    Replies:
    0
    Views:
    81
    Daniel Berger
    May 8, 2004
  2. Daniel Berger

    [ANN] win32-changenotify 0.3.0

    Daniel Berger, Jan 25, 2005, in forum: Ruby
    Replies:
    3
    Views:
    77
    Daniel Berger
    Feb 4, 2005
  3. Replies:
    7
    Views:
    93
    Daniel Berger
    Dec 15, 2006
  4. Martin DeMello

    win32::changenotify and multiple events

    Martin DeMello, Feb 28, 2007, in forum: Ruby
    Replies:
    0
    Views:
    92
    Martin DeMello
    Feb 28, 2007
  5. perl coder
    Replies:
    0
    Views:
    120
    perl coder
    Feb 16, 2004
Loading...

Share This Page