Memory Leak (again)

[Horacio Sanson]

There is a little memory leak in my script I am not able to spot. I have tried
everything I found about detecting leaks but no luck.

I first tried all the recomendations found here

http://theexciter.com/articles/finding-leaking-ruby-objects?commented=1#c000092

I can see the number of objects but the total number of objects does not
increment indefinitely but the script keeps consuming memory.

I tried this code:

##################################33
Entry = Struct.new( "MEntry", :c, :mem )
class MEntry; def to_s() "#{c} : #{mem}"; end; end

GroupEntry = Struct.new( "GroupEntry", :c, :mem, :total )
class GroupEntry; def to_s() "#{c} x#{total}(#{mem})"; end; end

def profile_mem
groups = {}
ObjectSpace.each_object { |x|
e = nil
begin
e = MEntry.new( x.class, Marshal::dump(x).size )
rescue TypeError # undumpable
e = MEntry.new( x.class, 0 )
end
if groups.has_key? e.c
groups[e.c].mem += e.mem
groups[e.c].total += 1
else
groups[e.c] = GroupEntry.new( e.c, e.mem, 1 )
end
}
File.open( "mem_log", "a+" ) { |file|
total = 0
groups.to_a.sort_by { |e| e[1].mem }.each { |e|
file << "#{e[1]} ";
total += e[1].mem
}
file << "TOTAL == #{total}"
file << "\n"
}
end


This code gives a list of all objects in ObjectSpace and their memory usage.
Also I log the total memory usage.

The log indicates that my scripts uses between 3 to 4 megs of memory
constantly, that is, it fluctuates between these values but the windows
manager indicates that the memory increase all the time and never goes down.

Running the script for a day consumes all 2G ram of my machine and causes the
script to bail out.

So are there any other tips on how to detect memory leaks?? Ruby Memory
Validator is only available to accepted beta testers so is not an option for
me.

I use Windows XP pro with the One-Click Ruby installer (1.8.2) and
ActiveRecord (1.13.2).

Any ideas are welcome...

regards,
Horacio

 

[Stephen Kellett]

So are there any other tips on how to detect memory leaks?? Ruby Memory
Validator is only available to accepted beta testers so is not an option for
me.

What makes you think you would not be accepted?

Stephen

 

[Robert Klemme]

There is a little memory leak in my script I am not able to spot. I
have tried everything I found about detecting leaks but no luck.

I first tried all the recomendations found here

http://theexciter.com/articles/finding-leaking-ruby-objects?commented=1#c000092

I can see the number of objects but the total number of objects does
not increment indefinitely but the script keeps consuming memory.

I tried this code:

##################################33
Entry = Struct.new( "MEntry", :c, :mem )
class MEntry; def to_s() "#{c} : #{mem}"; end; end

GroupEntry = Struct.new( "GroupEntry", :c, :mem, :total )
class GroupEntry; def to_s() "#{c} x#{total}(#{mem})"; end; end

def profile_mem
groups = {}
ObjectSpace.each_object { |x|
e = nil
begin
e = MEntry.new( x.class, Marshal::dump(x).size )
rescue TypeError # undumpable
e = MEntry.new( x.class, 0 )
end
if groups.has_key? e.c
groups[e.c].mem += e.mem
groups[e.c].total += 1
else
groups[e.c] = GroupEntry.new( e.c, e.mem, 1 )
end
}
File.open( "mem_log", "a+" ) { |file|
total = 0
groups.to_a.sort_by { |e| e[1].mem }.each { |e|
file << "#{e[1]} ";
total += e[1].mem
}
file << "TOTAL == #{total}"
file << "\n"
}
end


This code gives a list of all objects in ObjectSpace and their memory
usage. Also I log the total memory usage.

Your script does not determine the amount of individual objects. By doing
Marshal::dump(x).size you calculate the memory needed by a complete graph
of objects that is accessible from x. You'll count object size at least
twice. Also, you will see growing mem usage if you create a bunch of
objects and then stuff them in several collections.

The log indicates that my scripts uses between 3 to 4 megs of memory
constantly, that is, it fluctuates between these values but the
windows manager indicates that the memory increase all the time and
never goes down.

That's probably a better measure.

Running the script for a day consumes all 2G ram of my machine and
causes the script to bail out.

Difficult to tell what the problem is without seeing the script. A likely
cause is that you somehow store objects in some statically (i.e. from a
constant) accessible container.

Since the number of objects stays put another likely cause of memory
consumption is a String which gets appended to all the time. Do you maybe
store some kind of log info in mem?

Kind regards

robert

 

[Eero Saynatkari]

There is a little memory leak in my script I am not able to spot. I have
tried
everything I found about detecting leaks but no luck.

I first tried all the recomendations found here

http://theexciter.com/articles/finding-leaking-ruby-objects?commented=1#c000092

I can see the number of objects but the total number of objects does not
increment indefinitely but the script keeps consuming memory.

I tried this code:

##################################33
Entry = Struct.new( "MEntry", :c, :mem )
class MEntry; def to_s() "#{c} : #{mem}"; end; end

GroupEntry = Struct.new( "GroupEntry", :c, :mem, :total )
class GroupEntry; def to_s() "#{c} x#{total}(#{mem})"; end; end

def profile_mem
groups = {}
ObjectSpace.each_object { |x|
e = nil
begin
e = MEntry.new( x.class, Marshal::dump(x).size )
rescue TypeError # undumpable
e = MEntry.new( x.class, 0 )
end
if groups.has_key? e.c
groups[e.c].mem += e.mem
groups[e.c].total += 1
else
groups[e.c] = GroupEntry.new( e.c, e.mem, 1 )
end
}
File.open( "mem_log", "a+" ) { |file|
total = 0
groups.to_a.sort_by { |e| e[1].mem }.each { |e|
file << "#{e[1]} ";
total += e[1].mem
}
file << "TOTAL == #{total}"
file << "\n"
}
end


This code gives a list of all objects in ObjectSpace and their memory
usage.
Also I log the total memory usage.

The log indicates that my scripts uses between 3 to 4 megs of memory
constantly, that is, it fluctuates between these values but the windows
manager indicates that the memory increase all the time and never goes
down.

The above logger seems to be reasonably good (although it
will not count Symbols and Fixnums) so there is a chance
this might be caused by your platform.

Running the script for a day consumes all 2G ram of my machine and
causes the
script to bail out.

So are there any other tips on how to detect memory leaks?? Ruby Memory
Validator is only available to accepted beta testers so is not an option
for
me.

I use Windows XP pro with the One-Click Ruby installer (1.8.2) and
ActiveRecord (1.13.2).

Any ideas are welcome...

If you can publish the script or a minimal version exhibiting
the same behaviour, we can try to troubleshoot it (or just run
it on different platforms in case the problem is caused by WinXP).

regards,
Horacio

E