Memory leak

Discussion in 'Ruby' started by Sebastian probst Eide, May 25, 2007.

  1. Hi

    I am currently learning to use Ruby-DBI and have a really simple script
    (pasted below). The problem I am having is that the program seems to be
    leaking memory! When monitoring it in the activity monitor (MacOSX) I
    can see that the process uses 100kb more memory every second! Why does
    this happen? I am planning to write a little daemon that is going to run
    non-stop on a server and can't have it leak memory. Is it dbi that is
    leaking or is there something in my code?

    If I let it run for 24 hours the process would take 8,6gb of memory
    instead of the 3mb it started out with, and my server isn't up for that!

    Hope you have any suggestions.

    I also tried a patch to the ruby-mysql api which was supposed to handle
    some memory leak, but it didn't improve my test scenario...
    (http://railsexpress.de/blog/articles/2006/10/05/make-ruby-mysql-create-less-garbage)

    #!/usr/bin/ruby -w

    require "dbi"

    while true

    begin
    # connect to the MySQL server
    dbh = DBI.connect("DBI:Mysql:test:localhost", "root", "")
    # get server version string and display it
    row = dbh.select_one("SELECT VERSION()")
    puts "Server version: " + row[0]
    rescue DBI::DatabaseError => e
    puts "An error occurred"
    puts "Error code: #{e.err}"
    puts "Error message: #{e.errstr}"
    ensure
    # disconnect from server
    dbh.disconnect if dbh
    end
    end

    --
    Posted via http://www.ruby-forum.com/.
    Sebastian probst Eide, May 25, 2007
    #1
    1. Advertising

  2. Why is it inside a while loop?

    Aren't you just creating hundreds and thousands of db connecions?
    Peter Hickman, May 25, 2007
    #2
    1. Advertising

  3. Well, never mind... it stops "leaking" after a while. When it is at
    about the double size of what it started out at. The first version I run
    paused one second between each run so I didn't realize that till after I
    had posted my previous post. When removing the "sleep 1" (not in the
    source code I posted) I realized that the process doesn't consume more
    memory when it has reached about 6mb.

    --
    Posted via http://www.ruby-forum.com/.
    Sebastian probst Eide, May 25, 2007
    #3
  4. Peter Hickman wrote:
    > Why is it inside a while loop?
    >
    > Aren't you just creating hundreds and thousands of db connections?


    Well, now, not really. I am closing the connections I am making which
    should make it all right? Although having the connection part outside
    the loop probably would make sense... But I am going to have the program
    pause at certain intervals and didn't want to have any open
    connections... that is why I tried it like this... But opening and
    closing a connection shouldn't make the process memory consumption grow,
    should it?

    Sebastian

    --
    Posted via http://www.ruby-forum.com/.
    Sebastian probst Eide, May 25, 2007
    #4
  5. Sebastian probst Eide wrote:
    > Peter Hickman wrote:
    >> Why is it inside a while loop?
    >>
    >> Aren't you just creating hundreds and thousands of db connections?

    >
    > Well, now, not really. I am closing the connections I am making which
    > should make it all right? Although having the connection part outside
    > the loop probably would make sense... But I am going to have the program
    > pause at certain intervals and didn't want to have any open
    > connections... that is why I tried it like this... But opening and
    > closing a connection shouldn't make the process memory consumption grow,
    > should it?
    >

    Well... ofcourse you were right. Moving the loop into the begin clause
    made the program a lot faster and lowered the memory consumption.


    --
    Posted via http://www.ruby-forum.com/.
    Sebastian probst Eide, May 25, 2007
    #5
  6. On 25.05.2007 16:27, Sebastian probst Eide wrote:
    > Well, never mind... it stops "leaking" after a while. When it is at
    > about the double size of what it started out at. The first version I run
    > paused one second between each run so I didn't realize that till after I
    > had posted my previous post. When removing the "sleep 1" (not in the
    > source code I posted) I realized that the process doesn't consume more
    > memory when it has reached about 6mb.


    IIRC DBI also supports the block form for opening and closing, so you
    should rather do

    DBI.connect("DBI:Mysql:test:localhost", "root", "") do |dbh|
    ....
    end

    robert
    Robert Klemme, May 25, 2007
    #6
  7. If my understanding is correct the garbage collector will only jump in
    when things get dangerously low or when there is a pause(?) in the
    process. So a tight loop can generate a large backlog of garbage that
    needs to be cleaned up.


    Others will probably be able to inform this better than I.
    Peter Hickman, May 25, 2007
    #7
  8. Peter Hickman wrote:
    > If my understanding is correct the garbage collector will only jump in
    > when things get dangerously low or when there is a pause(?) in the
    > process. So a tight loop can generate a large backlog of garbage that
    > needs to be cleaned up.

    Ok, good to know! Thanks Peter!

    Best regards
    Sebastian

    --
    Posted via http://www.ruby-forum.com/.
    Sebastian probst Eide, May 25, 2007
    #8
  9. > IIRC DBI also supports the block form for opening and closing, so you
    > should rather do
    >
    > DBI.connect("DBI:Mysql:test:localhost", "root", "") do |dbh|
    > ...
    > end
    >
    > robert


    Thanks for the tip!
    I'll use that instead.

    --
    Posted via http://www.ruby-forum.com/.
    Sebastian probst Eide, May 25, 2007
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?Y3liZXJzdHJpa2U=?=

    datagrid memory leak?

    =?Utf-8?B?Y3liZXJzdHJpa2U=?=, Jan 3, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    465
    =?Utf-8?B?Y3liZXJzdHJpa2U=?=
    Jan 3, 2005
  2. s.subbarayan

    Dynamic memory allocation and memory leak...

    s.subbarayan, Mar 18, 2005, in forum: C Programming
    Replies:
    10
    Views:
    698
    Eric Sosman
    Mar 22, 2005
  3. Richard Heathfield

    Leak or no leak ??

    Richard Heathfield, Jul 10, 2006, in forum: C Programming
    Replies:
    4
    Views:
    349
    Richard Heathfield
    Jul 10, 2006
  4. cham
    Replies:
    5
    Views:
    767
  5. Mark Probert
    Replies:
    4
    Views:
    325
    Mark Probert
    Feb 9, 2005
Loading...

Share This Page