Memory leaks, maybe. need help.

[doron]

I am writing a server using ruby 1.8.6 patchlevel 287. It connects to
another server, fetches and process information.
Sometimes it needs to process a lot of information. It uses net/http
and REXML libraries.
My problem is that at some point the process memory usage grows from
20MB to 300MB.

I used gdb and rb_object_counts to look at the number of objects it
uses.
While the memory usage is low i see it uses 20K Strings, 1k or Arrays
etc.
While the memory usage is high i see it uses 1M Strings, 200k or
Arrays etc.
After a while, when the load is over, it again uses only 20K string
and 1K Arrays. It seems that GC worked fine.
But the memory usage I see using top is still around 300MB.

Does it mean I have memory leaks?
I read somewhere that "MRI's garbage collector never returns memory to
the system". that can explain what I see, but is it true? If so, is
there any solution?

 

[Roger Pack]

Does it mean I have memory leaks?
I read somewhere that "MRI's garbage collector never returns memory to
the system". that can explain what I see, but is it true? If so, is
there any solution?

MRI returns memory to the system if it is alloc'ed and free'ed
besides that it allocates memory in ever increasing heap sizes, which
means that they have less and less of a chance to be reclaimed [though
in 1.9 they are 2K each so less of a problem...I guess].
Anyway, to the point--try 1.9 or 1.8.x with MBARI patches they are much
better with memory [or jruby].
-=r

 

[Charles Oliver Nutter]

MRI returns memory to the system if it is alloc'ed and free'ed
besides that it allocates memory in ever increasing heap sizes, which
means that they have less and less of a chance to be reclaimed [though
in 1.9 they are 2K each so less of a problem...I guess].
Anyway, to the point--try 1.9 or 1.8.x with MBARI patches they are much
better with memory [or jruby].

I'd definitely recommend trying JRuby. There's a few knobs you can
turn to keep memory under a specific size (default is to let it grow
no larger than 512MB heap, but many apps will run in a lot less) which
means it won't use up all available memory unless you tell it to. And
though it uses more memory out of the box, if you have a large dataset
or heavy object churn and high loads, JRuby does extremely well.

- Charlie