Leak in Win32::ChangeNotify?

[Thomas Kratz]

The code below is a simple example of using Win32::ChangeNotify. AFAIK
this is how it should be used according to the docs.
If you run this you will see a steady increase in the used nonpage pool
memory of the perl process in the Win32 performance monitor (Category
'Process', counter 'Pool Nonpaged Bytes'). Changing the file will result
in the notification message printed to screen as well as in more memory
leaking. After the second change it will go berserk and if you don't
terminate the process quickly it will eat up all the systems nonpaged pool
memory and thus cause serious trouble elsewhere in the system.

Moving the object creation into the while loop and deleting the line with
'$notify->reset()' works around the problem.

Can somebody confirm this? If yes I will file a bug report on rt.cpan.org.

Tested with perl 5.8.4 (custom build) and libwin 0.19.1
(Win32::ChangeNotify 1.02) under WinXP and Win2003 Server.

Thomas

=== code start ===
use strict;
use warnings;

$| = 1;

use File::Spec::Functions qw/catfile/;
use Win32::ChangeNotify;

my $dir = 'd:/tmp/perl/chng';
my $notify_fn = catfile($dir, 'bla.txt');

my $last = 0;
my $notify = Win32::ChangeNotify->new($dir, 0, 'LAST_WRITE');
die 'cannot create notify object'
unless ref($notify) eq 'Win32::ChangeNotify';

while ( 1 ) {

my $rc = $notify->wait(100);
$notify->reset();

last if $rc == -1;

my @stat = stat($notify_fn);

if ( $rc == 1 and $stat[9] > $last ) {

print "$notify_fn has changed\n";

$last = $stat[9];
}
}
=== code end ===

--
$/=$,,$_=<DATA>,s,(.*),$1,see;__END__
s,^(.*\043),,mg,@_=map{[split'']}split;{#>J~.>_an~>>e~......>r~
$_=$_[$%][$"];y,<~>^,-++-,?{$/=--$|?'"':#..u.t.^.o.P.r.>ha~.e..
'%',s,(.),\$$/$1=1,,$;=$_}:/\w/?{y,_, ,,#..>s^~ht<._..._..c....
print}:y,.,,||last,,,,,,$_=$;;eval,redo}#.....>.e.r^.>l^..>k^.-

 

[Matt Garrish]

The code below is a simple example of using Win32::ChangeNotify. AFAIK
this is how it should be used according to the docs.
If you run this you will see a steady increase in the used nonpage pool
memory of the perl process in the Win32 performance monitor (Category
'Process', counter 'Pool Nonpaged Bytes'). Changing the file will result
in the notification message printed to screen as well as in more memory
leaking. After the second change it will go berserk and if you don't
terminate the process quickly it will eat up all the systems nonpaged pool
memory and thus cause serious trouble elsewhere in the system.

Moving the object creation into the while loop and deleting the line with
'$notify->reset()' works around the problem.

Can somebody confirm this? If yes I will file a bug report on rt.cpan.org.

Strange. The first time I ran the script the nonpaged memory slowly grew
(nothing like what you describe, though). I let it run up from ~10100K to
~11500K before killing script (at which point it dropped right back to where
it started), but on subsequent restarts could not get the script to leak
again (it stayed steady at ~10100 no matter what I did to the file).

This was on XP Pro using v5.8.2.

Matt

 

[Thomas Kratz]

Strange. The first time I ran the script the nonpaged memory slowly grew
(nothing like what you describe, though). I let it run up from ~10100K to
~11500K before killing script (at which point it dropped right back to where
it started), but on subsequent restarts could not get the script to leak
again (it stayed steady at ~10100 no matter what I did to the file).

This was on XP Pro using v5.8.2.

Do you really mean 10100KB. The value should be about 1-2K. 10MB of
nonpaged memory is way too much.

I get the same results as yesterday on various machines. The strange thing
is, that after the second modification of the file it will take some time
before the nonepaged pool usage freaks out.
I think I will have a look into the xs code of Win32::ChangeNotify and
Win32::IPC on the weekend. Perhaps I'll get an idea, what could go wrong here.

Always creating a new object works fine as a workaround, so there is no
urgent need for a fix.

Thomas

--
$/=$,,$_=<DATA>,s,(.*),$1,see;__END__
s,^(.*\043),,mg,@_=map{[split'']}split;{#>J~.>_an~>>e~......>r~
$_=$_[$%][$"];y,<~>^,-++-,?{$/=--$|?'"':#..u.t.^.o.P.r.>ha~.e..
'%',s,(.),\$$/$1=1,,$;=$_}:/\w/?{y,_, ,,#..>s^~ht<._..._..c....
print}:y,.,,||last,,,,,,$_=$;;eval,redo}#.....>.e.r^.>l^..>k^.-

 

[Thomas Kratz]

=== code start ===
use strict;
use warnings;

$| = 1;

use File::Spec::Functions qw/catfile/;
use Win32::ChangeNotify;

my $dir = 'd:/tmp/perl/chng';
my $notify_fn = catfile($dir, 'bla.txt');

my $last = 0;
my $notify = Win32::ChangeNotify->new($dir, 0, 'LAST_WRITE');
die 'cannot create notify object'
unless ref($notify) eq 'Win32::ChangeNotify';

while ( 1 ) {

my $rc = $notify->wait(100);
$notify->reset();

last if $rc == -1;

my @stat = stat($notify_fn);

if ( $rc == 1 and $stat[9] > $last ) {

print "$notify_fn has changed\n";

$last = $stat[9];
}
}
=== code end ===

The problem was calling reset on the object even if there was no signalled
change event. Moving the "$notify->reset();" to the end of the "if ( $rc
== 1..." block, ended the leaking.

After looking at the XS code, it was clear that I used the module the
wrong way. Each call to reset() causes a Win32 API call to
FindNextChangeNotification and that only makes sense if the last Find was
successful.

So: no bug, wrong usage!

Thomas

--
$/=$,,$_=<DATA>,s,(.*),$1,see;__END__
s,^(.*\043),,mg,@_=map{[split'']}split;{#>J~.>_an~>>e~......>r~
$_=$_[$%][$"];y,<~>^,-++-,?{$/=--$|?'"':#..u.t.^.o.P.r.>ha~.e..
'%',s,(.),\$$/$1=1,,$;=$_}:/\w/?{y,_, ,,#..>s^~ht<._..._..c....
print}:y,.,,||last,,,,,,$_=$;;eval,redo}#.....>.e.r^.>l^..>k^.-

 

[Matt Garrish]

Do you really mean 10100KB. The value should be about 1-2K. 10MB of
nonpaged memory is way too much.

Yup, but I believe we were looking at different things. I was watching the
nonpaged kernel memory in the task manager under the Performance tab. It
generally sits around 10100 - 10300 when the machine is idle.

Matt