Garbage Collection

[Brett Simmers]

I know that the garbage collector will start when more memory is needed,
but will it also start after the program has been running for a long
period of time? I have a daemon written in Ruby that's been running for
about 8 days now, and its memory usage has increased by about 1MB per
day. It's up to 39MB by now which isn't bad, but if this trend
continues it could get ugly after a few weeks. What I don't know is if
it's actually using all that memory or if it just hasn't run the GC yet
because the usage is still fairly low. I'm sure it wouldn't be hard to
test this by putting a few calls to GC.start in the code and restarting
it, but I'd really like to avoid restarting it if possible. Does anyone
know which conditions will trigger the GC other than malloc failing?

Brett

 

[Jano Svitok]

I know that the garbage collector will start when more memory is needed,
but will it also start after the program has been running for a long
period of time? I have a daemon written in Ruby that's been running for
about 8 days now, and its memory usage has increased by about 1MB per
day. It's up to 39MB by now which isn't bad, but if this trend
continues it could get ugly after a few weeks. What I don't know is if
it's actually using all that memory or if it just hasn't run the GC yet
because the usage is still fairly low. I'm sure it wouldn't be hard to
test this by putting a few calls to GC.start in the code and restarting
it, but I'd really like to avoid restarting it if possible. Does anyone
know which conditions will trigger the GC other than malloc failing?

http://redhanded.hobix.com/inspect/theFullyUpturnedBin.html is
interesting reading. Other than that, make sure you don't have left
references... there are tools to check it, on windows there's Ruby
Memory Validator by softwareverify, and some opensource ones... IIRC
Eric Hodel did something some time ago (mem_inspect)

http://www.softwareverify.com/ruby/customBuild/memtrack/index.html
http://www.softwareverify.com/ruby/memory/index.html

 

[Tim Hunter]

I know that the garbage collector will start when more memory is
needed, but will it also start after the program has been running for
a long period of time? I have a daemon written in Ruby that's been
running for about 8 days now, and its memory usage has increased by
about 1MB per day. It's up to 39MB by now which isn't bad, but if
this trend continues it could get ugly after a few weeks. What I
don't know is if it's actually using all that memory or if it just
hasn't run the GC yet because the usage is still fairly low. I'm sure
it wouldn't be hard to test this by putting a few calls to GC.start in
the code and restarting it, but I'd really like to avoid restarting it
if possible. Does anyone know which conditions will trigger the GC
other than malloc failing?
Brett

This article may help.

http://whytheluckystiff.net/articles/theFullyUpturnedBin.html

 

[Joel VanderWerf]

I know that the garbage collector will start when more memory is needed,
but will it also start after the program has been running for a long
period of time? I have a daemon written in Ruby that's been running for
about 8 days now, and its memory usage has increased by about 1MB per
day. It's up to 39MB by now which isn't bad, but if this trend
continues it could get ugly after a few weeks. What I don't know is if
it's actually using all that memory or if it just hasn't run the GC yet
because the usage is still fairly low. I'm sure it wouldn't be hard to
test this by putting a few calls to GC.start in the code and restarting
it, but I'd really like to avoid restarting it if possible. Does anyone
know which conditions will trigger the GC other than malloc failing?
Brett

What ruby version?

I have the same problem with 1.8.6, but not with 1.8.4.

I'm pretty sure GC will have run by now. The threshold starts out at
around 8Mb (GC runs when the total requested allocation since the last
GC exceeds the threshold).

(It's a good idea to have a DRb backdoor to a process like this, so that
you can connect to it and issue commands, not just GC.start.)

 

[Brett Simmers]

What ruby version?

I have the same problem with 1.8.6, but not with 1.8.4.

I'm pretty sure GC will have run by now. The threshold starts out at
around 8Mb (GC runs when the total requested allocation since the last
GC exceeds the threshold).

(It's a good idea to have a DRb backdoor to a process like this, so
that you can connect to it and issue commands, not just GC.start.)

I think the server it's running on has 1.8.5. Thanks to everyone for
the responses, those articles were definitely interesting. I did some
quick calculations based on which objects I know are persisting and I
think it's actually using about 900kb per day of ram so manually
activating GC wouldn't help much. I redesigned the core of the app and
the next version isn't going to have nearly as much state so that won't
be a problem any more when I deploy it next week.