How to find out memory leak?

P

pluskid

Hi, all!

I'm having memory leak problem with my program, but can't find where's
the leak.
I have a program written in pure-Ruby. It implemented a Chinese word
segment
algorithm. I call `segment(text)' to get the result. But each time I
call `segment',
the memory usage of my program (Ruby) increased several handred K-
bytes -- the
number of bytes of increment is roughly equal during each call.

I can see from outside (using `ps' or `top') that the memory is
leaking. But I can't
find where the leaking goes. I tried to use ObjectSpace.each_object to
find out
what leaks. I followed this article

http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails

and compare the objects after each call to `segment'. What surprise me
is that
there's no object leaked between each call. The number of objects
after each
call is roughly equal.

I also tried bleak_house:

http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html

I don't precisely how to use it. I just dump a snapshot after each
call to `segment'
and use `bleak' to analyze the result. However, it also shows no leak.
But if
there's no leak, where goes the memory?

Here's the result of using the `pmap' tool to inspect the running Ruby
process:

933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
08048000 4K r-x-- /usr/bin/ruby1.8
08049000 4K rw--- /usr/bin/ruby1.8
0804a000 33412K rw--- [ anon ]
b74a8000 7016K rw--- [ anon ]
b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8c000 1192K rw--- [ anon ]
b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
b7e00000 12K rw--- [ anon ]
b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
b7e33000 156K rw--- [ anon ]
b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
b7e5e000 4K rw--- [ anon ]
b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
b7e75000 8K rw--- [ anon ]
b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
b7f36000 64K rw--- [ anon ]
b7f48000 8K rw--- [ anon ]
b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f51000 8K rw--- [ anon ]
b7f53000 104K r-x-- /lib/ld-2.7.so
b7f6d000 8K rw--- /lib/ld-2.7.so
bfdea000 116K rw--- [ stack ]
ffffe000 4K r-x-- [ anon ]
total 44568K

The section "0804a000 33412K rw--- [ anon ]" increased after each
call. I've no idea now. Do you have any suggestion for me?
 
J

Jano Svitok

Hi, all!

I'm having memory leak problem with my program, but can't find where's
the leak.
I have a program written in pure-Ruby. It implemented a Chinese word
segment
algorithm. I call `segment(text)' to get the result. But each time I
call `segment',
the memory usage of my program (Ruby) increased several handred K-
bytes -- the
number of bytes of increment is roughly equal during each call.

I can see from outside (using `ps' or `top') that the memory is
leaking. But I can't
find where the leaking goes. I tried to use ObjectSpace.each_object to
find out
what leaks. I followed this article

http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails

and compare the objects after each call to `segment'. What surprise me
is that
there's no object leaked between each call. The number of objects
after each
call is roughly equal.

I also tried bleak_house:

http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html

I don't precisely how to use it. I just dump a snapshot after each
call to `segment'
and use `bleak' to analyze the result. However, it also shows no leak.
But if
there's no leak, where goes the memory?

Here's the result of using the `pmap' tool to inspect the running Ruby
process:

933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
08048000 4K r-x-- /usr/bin/ruby1.8
08049000 4K rw--- /usr/bin/ruby1.8
0804a000 33412K rw--- [ anon ]
b74a8000 7016K rw--- [ anon ]
b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8c000 1192K rw--- [ anon ]
b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
b7e00000 12K rw--- [ anon ]
b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
b7e33000 156K rw--- [ anon ]
b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
b7e5e000 4K rw--- [ anon ]
b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
b7e75000 8K rw--- [ anon ]
b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
b7f36000 64K rw--- [ anon ]
b7f48000 8K rw--- [ anon ]
b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f51000 8K rw--- [ anon ]
b7f53000 104K r-x-- /lib/ld-2.7.so
b7f6d000 8K rw--- /lib/ld-2.7.so
bfdea000 116K rw--- [ stack ]
ffffe000 4K r-x-- [ anon ]
total 44568K

The section "0804a000 33412K rw--- [ anon ]" increased after each
call. I've no idea now. Do you have any suggestion for me?
Click to expand...

This is what I have read here or somewhere else, and I an not
knowledgeable enough to confirm it, anyway:
Ruby allocs space from the OS in large chunks (initially 8 MB, and
each other chunk is bigger by a predefined coefficient).
Any chunk can be returned to the OS only if it is empty - so if your
memory is fragmented, ruby will have to alloc new chunks
in order to satisfy ruby script being run.

If this is true, it means: the order and size of your allocations is important.
Especially order of the allocations that you later release relative to
those that you don't.
 
C

Chiyuan Zhang

If that is the case, that might be quite random, istead
of seeing the increment of every call roughly equal. :(

2008/2/25 said:
Hi, all!

I'm having memory leak problem with my program, but can't find where's
the leak.
I have a program written in pure-Ruby. It implemented a Chinese word
segment
algorithm. I call `segment(text)' to get the result. But each time I
call `segment',
the memory usage of my program (Ruby) increased several handred K-
bytes -- the
number of bytes of increment is roughly equal during each call.

I can see from outside (using `ps' or `top') that the memory is
leaking. But I can't
find where the leaking goes. I tried to use ObjectSpace.each_object to
find out
what leaks. I followed this article

http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails

and compare the objects after each call to `segment'. What surprise me
is that
there's no object leaked between each call. The number of objects
after each
call is roughly equal.

I also tried bleak_house:

http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html

I don't precisely how to use it. I just dump a snapshot after each
call to `segment'
and use `bleak' to analyze the result. However, it also shows no leak.
But if
there's no leak, where goes the memory?

Here's the result of using the `pmap' tool to inspect the running Ruby
process:

933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
08048000 4K r-x-- /usr/bin/ruby1.8
08049000 4K rw--- /usr/bin/ruby1.8
0804a000 33412K rw--- [ anon ]
b74a8000 7016K rw--- [ anon ]
b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8c000 1192K rw--- [ anon ]
b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
b7e00000 12K rw--- [ anon ]
b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
b7e33000 156K rw--- [ anon ]
b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
b7e5e000 4K rw--- [ anon ]
b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
b7e75000 8K rw--- [ anon ]
b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
b7f36000 64K rw--- [ anon ]
b7f48000 8K rw--- [ anon ]
b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f51000 8K rw--- [ anon ]
b7f53000 104K r-x-- /lib/ld-2.7.so
b7f6d000 8K rw--- /lib/ld-2.7.so
bfdea000 116K rw--- [ stack ]
ffffe000 4K r-x-- [ anon ]
total 44568K

The section "0804a000 33412K rw--- [ anon ]" increased after each
call. I've no idea now. Do you have any suggestion for me?
Click to expand...


This is what I have read here or somewhere else, and I an not
knowledgeable enough to confirm it, anyway:
Ruby allocs space from the OS in large chunks (initially 8 MB, and
each other chunk is bigger by a predefined coefficient).
Any chunk can be returned to the OS only if it is empty - so if your
memory is fragmented, ruby will have to alloc new chunks
in order to satisfy ruby script being run.

If this is true, it means: the order and size of your allocations is important.
Especially order of the allocations that you later release relative to
those that you don't.
Click to expand...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Cannot allocate memory (Errno::ENOMEM) 1
Memory leak in pthread_cancel with a detached thread. 2
Runtime Error for virtual derived class destructor 1
Segfault - How do i find whats causing it? 3
Find memory leak in 1.9 app 5
Memory corruption on freeing a pointer to pointer 172
FFI Memory Leak 4
mysql gem memory leak 1

Members online

No members online now.
Total: 43 (members: 0, guests: 43)
Robots: 179

Forum statistics

Threads
473,766
Messages
2,569,569
Members
45,043
Latest member
CannalabsCBDReview

Latest Threads

Top