Writing long-running daemons without memory leaks?

Discussion in 'Ruby' started by Toby DiPasquale, Mar 17, 2006.

  1. Hi all,

    I am in the middle of writing some code that will exist as a
    long-running process on a number of machines. I am having trouble
    keeping memory utilization down, which is important in this case,
    because other processes on these machines will have priority to use the
    majority of RAM on these boxes for other tasks. The daemon itself is
    pretty simple, just reading and writing files and keeping some small
    state, but it also needs to cache large writes from multiple clients.
    When it does this, I find that this memory is not freed.

    My question is, I've heard from numerous accounts that WEBrick also
    leaks memory, and I would suspect that Mongrel does, too (though I
    haven't confirmed that) for the same reason. I understand that Ruby's GC
    is conservative, but can anyone point me to documentation, reference,
    source code, anything really that will help me better understand how to
    write Ruby code that doesn't "leak" and can be sustained as a
    long-running process without an ever-increasing memory footprint?

    Any help would be greatly appreciated. Thanks in advance.

    P.S. I've read the Pickaxe's lone page on the subject (in the Duck
    Typing chapter) and I've seen Why's article about the same
    (http://whytheluckystiff.net/articles/theFullyUpturnedBin.html).

    P.P.S. Has Minero Aoki's Ruby book been translated to English anywhere?
    I hear that it has a whole chapter on GC...

    --
    Toby DiPasquale

    --
    Posted via http://www.ruby-forum.com/.
     
    Toby DiPasquale, Mar 17, 2006
    #1
    1. Advertising

  2. Toby DiPasquale

    Guest

    On Sat, 18 Mar 2006, Toby DiPasquale wrote:

    > Hi all,
    >
    > I am in the middle of writing some code that will exist as a
    > long-running process on a number of machines. I am having trouble
    > keeping memory utilization down, which is important in this case,
    > because other processes on these machines will have priority to use the
    > majority of RAM on these boxes for other tasks. The daemon itself is
    > pretty simple, just reading and writing files and keeping some small
    > state, but it also needs to cache large writes from multiple clients.
    > When it does this, I find that this memory is not freed.
    >
    > My question is, I've heard from numerous accounts that WEBrick also
    > leaks memory, and I would suspect that Mongrel does, too (though I
    > haven't confirmed that) for the same reason. I understand that Ruby's GC
    > is conservative, but can anyone point me to documentation, reference,
    > source code, anything really that will help me better understand how to
    > write Ruby code that doesn't "leak" and can be sustained as a
    > long-running process without an ever-increasing memory footprint?
    >
    > Any help would be greatly appreciated. Thanks in advance.
    >
    > P.S. I've read the Pickaxe's lone page on the subject (in the Duck
    > Typing chapter) and I've seen Why's article about the same
    > (http://whytheluckystiff.net/articles/theFullyUpturnedBin.html).
    >
    > P.P.S. Has Minero Aoki's Ruby book been translated to English anywhere?
    > I hear that it has a whole chapter on GC...


    i'm running dozens of ruby daemons, some of which are extremely (months at a
    time) long lived and have not seen any gc issues. can you post a specific
    (minimal) example that you find leaks memory?

    regards.

    -a
    --
    share your knowledge. it's a way to achieve immortality.
    - h.h. the 14th dali lama
     
    , Mar 17, 2006
    #2
    1. Advertising

  3. Toby DiPasquale

    Guest

    On Sat, 18 Mar 2006, Toby DiPasquale wrote:

    >
    > Sure. Here is a server and client, resp, that exhibit the behavior I am
    > referring to:


    i cannot reproduce it:


    harp:~ > cat a.rb
    require 'socket'

    class Buffer
    def initialize size
    @size = size
    @buffer = []
    @length = 0
    end
    attr_reader :length, :size
    def full? ; @length == @size end
    def empty? ; @length.zero? end
    # Lets you fill up to capacity and then returns
    # what's left over
    def fill data
    if full?
    data
    elsif @length + data.length < @size
    @buffer << data
    @length += data.length
    nil
    else
    l = @size - @length
    @buffer << data[0, l]
    data[l..-1]
    end
    end
    def expunge
    buf = @buffer.join
    @buffer.clear
    @length = 0
    buf
    end
    end # class Buffer

    def read_long s
    s.read( 4).unpack( "N")[0]
    end

    # main client handling thread logic
    client_handler = lambda do |s|
    begin
    len = read_long s
    buf = Buffer.new len
    until buf.full?
    x = s.read 4096
    break if x.nil?
    buf.fill x
    end
    puts "writing #{buf.length} bytes"
    f = File.open "/dev/null", "w"
    f.write buf.expunge
    f.close
    ensure
    s.close
    end
    end

    # main server loop
    ss = TCPServer.new '127.0.0.1', 10001
    begin
    while true
    s = ss.accept
    puts "got connection"
    Thread.start s, &client_handler
    end
    ensure
    ss.close
    end



    harp:~ > cat b.rb
    require 'socket'

    def write_long s, l
    s.write( [l].pack( "N"))
    end

    str = "a" * 65536

    t = TCPSocket.new '127.0.0.1', 10001
    write_long t, 1024 * str.length
    1024.times { t.write str }
    t.close



    harp:~ > ps aux|head -1; while true;do ruby b.rb && ps aux|grep 'ruby a.rb'|grep -v grep ;done
    USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
    ahoward 13671 12.7 23.1 239328 237956 pts/26 S 10:14 0:44 ruby a.rb
    ahoward 13671 12.7 21.2 219404 218032 pts/26 S 10:14 0:45 ruby a.rb
    ahoward 13671 12.8 22.3 230784 229396 pts/26 S 10:14 0:45 ruby a.rb
    ahoward 13671 13.0 22.3 230784 229412 pts/26 S 10:14 0:45 ruby a.rb
    ahoward 13671 13.1 22.3 230916 229416 pts/26 S 10:14 0:46 ruby a.rb
    ahoward 13671 13.2 15.9 165364 163876 pts/26 S 10:14 0:46 ruby a.rb
    ahoward 13671 13.3 22.3 230904 229424 pts/26 S 10:14 0:47 ruby a.rb
    ahoward 13671 13.4 21.5 225116 221504 pts/26 R 10:14 0:47 ruby a.rb
    ahoward 13671 13.5 28.7 296328 294940 pts/26 S 10:14 0:48 ruby a.rb
    ahoward 13671 13.6 21.5 222480 221108 pts/26 S 10:14 0:48 ruby a.rb
    ahoward 13671 13.7 22.3 230796 229400 pts/26 S 10:14 0:49 ruby a.rb
    ahoward 13671 13.8 21.8 230804 224296 pts/26 R 10:14 0:49 ruby a.rb
    ahoward 13671 13.9 27.3 296344 281256 pts/26 R 10:14 0:50 ruby a.rb
    ahoward 13671 14.0 25.4 265672 261500 pts/26 R 10:14 0:50 ruby a.rb
    ahoward 13671 14.1 22.3 230880 229384 pts/26 S 10:14 0:51 ruby a.rb
    ahoward 13671 14.2 15.9 165340 163864 pts/26 S 10:14 0:51 ruby a.rb
    ahoward 13671 14.3 20.8 230888 214376 pts/26 R 10:14 0:52 ruby a.rb
    ahoward 13671 14.4 19.3 200220 198844 pts/26 S 10:14 0:52 ruby a.rb
    ahoward 13671 14.5 21.2 230852 218484 pts/26 R 10:14 0:53 ruby a.rb
    ahoward 13671 14.6 15.9 165312 163872 pts/26 R 10:14 0:53 ruby a.rb
    ahoward 13671 14.8 28.1 290632 289256 pts/26 S 10:14 0:54 ruby a.rb
    ahoward 13671 14.8 16.0 230776 164724 pts/26 R 10:14 0:54 ruby a.rb
    ahoward 13671 14.9 21.6 230908 222412 pts/26 R 10:14 0:54 ruby a.rb
    ahoward 13671 15.0 15.0 219356 154356 pts/26 R 10:14 0:55 ruby a.rb
    ahoward 13671 15.1 16.2 230864 166464 pts/26 R 10:14 0:55 ruby a.rb
    ahoward 13671 15.2 25.7 265764 264320 pts/26 S 10:14 0:56 ruby a.rb
    ahoward 13671 15.4 22.3 230852 229388 pts/26 S 10:14 0:56 ruby a.rb
    ahoward 13671 15.4 15.9 165312 163864 pts/26 S 10:14 0:57 ruby a.rb
    ahoward 13671 15.5 15.9 165320 163880 pts/26 R 10:14 0:57 ruby a.rb
    ahoward 13671 15.6 28.5 294368 292992 pts/26 S 10:14 0:58 ruby a.rb
    ahoward 13671 15.7 22.3 230808 229388 pts/26 S 10:14 0:58 ruby a.rb
    ahoward 13671 15.8 21.7 230816 223624 pts/26 R 10:14 0:59 ruby a.rb
    ahoward 13671 15.9 15.9 165276 163872 pts/26 R 10:14 0:59 ruby a.rb
    ahoward 13671 16.0 20.6 213384 212008 pts/26 S 10:14 0:59 ruby a.rb
    ahoward 13671 16.1 22.3 230812 229396 pts/26 S 10:14 1:00 ruby a.rb
    ahoward 13671 16.2 22.3 230812 229408 pts/26 S 10:14 1:00 ruby a.rb
    ahoward 13671 16.2 15.9 165280 163872 pts/26 R 10:14 1:01 ruby a.rb
    ahoward 13671 16.3 22.3 230812 229408 pts/26 S 10:14 1:01 ruby a.rb
    ahoward 13671 16.4 19.3 200212 198836 pts/26 S 10:14 1:02 ruby a.rb
    ahoward 13671 16.5 22.3 230840 229400 pts/26 S 10:14 1:02 ruby a.rb
    ahoward 13671 16.5 15.4 159584 158204 pts/26 R 10:14 1:02 ruby a.rb
    ahoward 13671 16.7 27.4 296424 281408 pts/26 R 10:14 1:03 ruby a.rb
    ahoward 13671 16.8 19.3 203424 199044 pts/26 R 10:14 1:03 ruby a.rb
    ahoward 13671 16.9 22.1 230888 227576 pts/26 R 10:14 1:04 ruby a.rb
    ahoward 13671 17.0 25.7 265720 264108 pts/26 S 10:14 1:04 ruby a.rb
    ahoward 13671 17.0 15.9 230812 163952 pts/26 R 10:14 1:05 ruby a.rb
    ahoward 13671 17.1 27.1 296352 278484 pts/26 R 10:14 1:05 ruby a.rb
    ahoward 13671 17.2 28.1 290640 289264 pts/26 S 10:14 1:06 ruby a.rb
    ahoward 13671 17.3 15.9 165248 163860 pts/26 R 10:14 1:06 ruby a.rb
    ahoward 13671 17.4 21.8 230920 224092 pts/26 R 10:14 1:07 ruby a.rb
    ahoward 13671 17.5 22.0 228288 226912 pts/26 S 10:14 1:07 ruby a.rb
    ahoward 13671 17.5 15.9 165264 163872 pts/26 R 10:14 1:07 ruby a.rb
    ahoward 13671 17.6 22.0 228228 226852 pts/26 S 10:14 1:08 ruby a.rb
    ahoward 13671 17.7 22.3 230868 229400 pts/26 S 10:14 1:08 ruby a.rb
    ahoward 13671 17.8 14.8 153684 152304 pts/26 R 10:14 1:09 ruby a.rb
    ahoward 13671 17.9 22.3 230812 229400 pts/26 S 10:14 1:09 ruby a.rb
    ahoward 13671 18.0 21.7 225108 223732 pts/26 S 10:14 1:10 ruby a.rb
    ahoward 13671 18.1 20.6 230924 212128 pts/26 R 10:14 1:10 ruby a.rb
    ahoward 13671 18.1 14.4 149944 148564 pts/26 R 10:14 1:11 ruby a.rb
    ahoward 13671 18.3 28.7 296332 294936 pts/26 S 10:14 1:11 ruby a.rb
    ahoward 13671 18.3 19.5 212736 200952 pts/26 R 10:14 1:12 ruby a.rb
    ahoward 13671 18.4 21.1 230824 216768 pts/26 R 10:14 1:12 ruby a.rb
    ahoward 13671 18.5 22.3 230816 229404 pts/26 S 10:14 1:12 ruby a.rb


    so memory builds to about 25 and, presumably when the gc kicks in, drops to 15
    - but it certainly stays in this range.

    what os and ruby version?

    -a
    --
    share your knowledge. it's a way to achieve immortality.
    - h.h. the 14th dali lama
     
    , Mar 17, 2006
    #3
  4. unknown wrote:
    > harp:~ > ps aux|head -1; while true;do ruby b.rb && ps aux|grep

    [...]
    > ahoward 13671 18.3 19.5 212736 200952 pts/26 R 10:14 1:12 ruby
    > a.rb
    > ahoward 13671 18.4 21.1 230824 216768 pts/26 R 10:14 1:12 ruby
    > a.rb
    > ahoward 13671 18.5 22.3 230816 229404 pts/26 S 10:14 1:12 ruby
    > a.rb
    >
    >
    > so memory builds to about 25 and, presumably when the gc kicks in, drops
    > to 15
    > - but it certainly stays in this range.
    >
    > what os and ruby version?


    ruby 1.8.4 on Ubuntu Linux, kernel 2.6.12-10 on a P4 Thinkpad with 1GB
    RAM. The other systems having this problem are all also running Linux
    2.6.x with ruby 1.8.4.

    One thing about the above: you do realize that's ~250MB and ~150MB, not
    25 and 15, right?

    --
    Toby DiPasquale

    --
    Posted via http://www.ruby-forum.com/.
     
    Toby DiPasquale, Mar 17, 2006
    #4
  5. -----BEGIN PGP SIGNED MESSAGE-----

    In article <>,
    Toby DiPasquale <> wrote:
    >Hi all,
    >
    >I am in the middle of writing some code that will exist as a
    >long-running process on a number of machines. I am having trouble
    >keeping memory utilization down, which is important in this case,
    >because other processes on these machines will have priority to use the
    >majority of RAM on these boxes for other tasks. The daemon itself is
    >pretty simple, just reading and writing files and keeping some small
    >state, but it also needs to cache large writes from multiple clients.
    >When it does this, I find that this memory is not freed.
    >


    _ This is generally true of all unix processes. They do not
    return allocated memory back to the system until they exit.
    Freeing memory inside the code merely allows you to use it
    again, it does not make it available to the system.

    _ If your process is not using the memory, it may stay swapped
    out, ( look at the real and virtual sizes ). If this is really
    an issue, the standard way around it is to do the memory
    intensive part in a forked subprocess that exits after it's
    done.

    _ Booker C. Bense


    -----BEGIN PGP SIGNATURE-----
    Version: 2.6.2

    iQCVAwUBRBsqMGTWTAjn5N/lAQHVbwP/XYqiyZsWzCqsYNfzP4rCe5cEzWbKBBfU
    54A/x/DDnImzpM3MYcIG4Um2AAGFBYMrylxEeTv0Ot6RMi7Nl2NJLIzIg/6lGh1M
    6d1ojbnddmUGfJXuYDZ2/lcWDWYzYCM6hYbWGh7h6eMYtNVZizE1k9ebA4zL+H0i
    jFU8K8Aoe7k=
    =ZbII
    -----END PGP SIGNATURE-----
     
    Booker C. Bense, Mar 17, 2006
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Phil Powell
    Replies:
    12
    Views:
    741
    Phil Powell
    Feb 1, 2004
  2. George Marsaglia

    Assigning unsigned long to unsigned long long

    George Marsaglia, Jul 8, 2003, in forum: C Programming
    Replies:
    1
    Views:
    685
    Eric Sosman
    Jul 8, 2003
  3. Marco Hornung

    find memory leaks in running program

    Marco Hornung, Dec 7, 2010, in forum: Python
    Replies:
    1
    Views:
    317
    shearichard
    Dec 7, 2010
  4. ShaunJ

    m// on very long lines leaks memory

    ShaunJ, Mar 13, 2008, in forum: Perl Misc
    Replies:
    5
    Views:
    114
    Uri Guttman
    Mar 13, 2008
  5. Replies:
    4
    Views:
    128
Loading...

Share This Page