Weird GC behaviour or memory leak with NArray::to_na?

J

Jeff

I have what appears to be a memory leak in my code, and I'm able to
reproduce it using the following test code:

require 'narray'

a=Array.new(100,2)
200000.times{|x|
#c=a.collect{|b| b+1} #This is OK.
#a.collect!{|b| b+1} #This is OK.
#a=a.collect{|b| b+1} #This is OK.
#c=Array(a) #This is OK.
d=NArray.to_na(a) #This results in huge memory bloat
}

Running this code shows my memory footprint increase monotonically
from ~2M to >20M... I killed the process at this point. Any of the
other commented lines would run with a stable memory footprint.

However, if I add ObjectSpace::garbage_collect into the loop,
everything is fine.

Now to complicate matters further:

require 'narray'
a=Array.new(100,2)
200000.times{|x|
d=a.collect{|b| NArray.to_na([b,b])} #memory bloat despite explicit
GC!
ObjectSpace::garbage_collect
}

This exhibits the growing memory footprint problem...... Help? Note:
I realize this example may be a little contrived, but my code does
this plus some more, requiring (IMHO) this approach.

Can someone explain this behaviour? I use the .to_na constructor a
lot in my code, and really need to sort this out, otherwise I cannot
run my code for longer than brief periods before my system comes
crashing on down.

Thanks, Jeff
 
M

Masahiro TANAKA

I have what appears to be a memory leak in my code, and I'm able to
reproduce it using the following test code:

require 'narray'

a=Array.new(100,2)
200000.times{|x|
#c=a.collect{|b| b+1} #This is OK.
#a.collect!{|b| b+1} #This is OK.
#a=a.collect{|b| b+1} #This is OK.
#c=Array(a) #This is OK.
d=NArray.to_na(a) #This results in huge memory bloat
}

Running this code shows my memory footprint increase monotonically
from ~2M to >20M... I killed the process at this point. Any of the
other commented lines would run with a stable memory footprint.

Hmm...

% ruby -v
ruby 1.8.1 (2003-12-25) [i686-linux]

% cat testna.rb
require 'narray'

a=Array.new(100,2)
system "ps v"
5.times do
200000.times do |x|
d=NArray.to_na(a)
end
system "ps v | grep ruby | grep -v grep"
end

% ruby testna.rb
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
5400 pts/3 S 0:00 253 570 2445 1600 0.2 ruby testna.rb
5400 pts/3 S 0:08 257 570 17565 16736 2.1 ruby testna.rb
5400 pts/3 S 0:16 257 570 25265 24436 3.1 ruby testna.rb
5400 pts/3 S 0:25 257 570 33605 32776 4.2 ruby testna.rb
5400 pts/3 S 0:33 257 570 41301 40472 5.2 ruby testna.rb
5400 pts/3 S 0:41 257 570 49481 48484 6.3 ruby testna.rb


Thanks, it is a bug.

--- na_array.c 2003/03/03 11:15:28 1.5
+++ na_array.c 2004/02/19 04:14:29
@@ -107,6 +107,7 @@
for (i=0; r-->0; i++) {
shape = mdai->item[r].shape;
}
+ xfree(mdai->type);
xfree(mdai->item);
xfree(mdai);
return shape;

Then,

% ruby testna.rb
5964 pts/3 R 0:00 253 570 2445 1604 0.2 ruby testna.rb
5964 pts/3 S 0:02 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 S 0:03 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 S 0:05 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 R 0:07 260 570 7733 6920 0.9 ruby testna.rb
5964 pts/3 S 0:09 260 570 7733 6920 0.9 ruby testna.rb

Masahiro Tanaka
 
J

Jeff

This problem was due to a memory leak in NArray 0.5.7. Updating to
0.5.7p2 fixed the problem.

Jeff
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top