XML-RPC Memory Problem

[Torsten Senf]

Hi,

I use a simple XML-RPC Server, which should print out Content from some Files via XML-RPC.
Here is the source of the server:

require "xmlrpc/server"

s = XMLRPC::Server.new(port="8080")

class MyFile

# shows only the tail of the file
def showlog(a)
line_a = []
f = File.open(a,"r")
f.seek(-100, IO::SEEK_END)
f.seek(-30000, IO::SEEK_CUR)
line_a = f.readlines.reverse
f.close
return line_a
end

# shows the whole file
def showconfig(a)
f = File.open(a,"r")
f_a = f.readlines
f.close
return f_a
end
end


s.add_handler("file", MyFile.new)

s.serve

The problem is now, when I call the server, it gives the contents of the
files (ok), but the memory usage of the process increases from call to call
and the memory would not be get free. I have no idea why. How can I
guarantee, that the memory usage from the server will be decreased after
calling a methode via XML RPC?

I hope you know what I mean and please apologize my bad english.

Regards

Torsten Senf

 

[Michael Neumann]

Hi,

I use a simple XML-RPC Server, which should print out Content from some Files via XML-RPC.
Here is the source of the server:

require "xmlrpc/server"

s = XMLRPC::Server.new(port="8080")

class MyFile

# shows only the tail of the file
def showlog(a)
line_a = []
f = File.open(a,"r")
f.seek(-100, IO::SEEK_END)
f.seek(-30000, IO::SEEK_CUR)
line_a = f.readlines.reverse
f.close
return line_a
end

# shows the whole file
def showconfig(a)
f = File.open(a,"r")
f_a = f.readlines
f.close
return f_a
end
end


s.add_handler("file", MyFile.new)

s.serve

The problem is now, when I call the server, it gives the contents of the
files (ok), but the memory usage of the process increases from call to call
and the memory would not be get free. I have no idea why. How can I
guarantee, that the memory usage from the server will be decreased after
calling a methode via XML RPC?

You could insert GC.start, which invokes the garbage-collector. Is
memory consuption increasing monotonically, or does it increase and then
stay stable? If the first case applies, you (or XML-RPC) has a memory leak.

Regards,

Michael

 

[Torsten Senf]

Hi,

I use a simple XML-RPC Server, which should print out Content from some
Files via XML-RPC. Here is the source of the server:

require "xmlrpc/server"

s = XMLRPC::Server.new(port="8080")

class MyFile

# shows only the tail of the file
def showlog(a)
line_a = []
f = File.open(a,"r")
f.seek(-100, IO::SEEK_END)
f.seek(-30000, IO::SEEK_CUR)
line_a = f.readlines.reverse
f.close
return line_a
end

# shows the whole file
def showconfig(a)
f = File.open(a,"r")
f_a = f.readlines
f.close
return f_a
end
end


s.add_handler("file", MyFile.new)

s.serve

The problem is now, when I call the server, it gives the contents of the
files (ok), but the memory usage of the process increases from call to call
and the memory would not be get free. I have no idea why. How can I
guarantee, that the memory usage from the server will be decreased after
calling a methode via XML RPC?

You could insert GC.start, which invokes the garbage-collector. Is
memory consuption increasing monotonically, or does it increase and then
stay stable? If the first case applies, you (or XML-RPC) has a memory leak.

The memory increasing monotonically. Where can be the memory leak? In my
above short code?

Torsten Senf

 

[Michael Neumann]

The memory increasing monotonically. Where can be the memory leak? In my
above short code?

I don't think so. How long do you run it? And how much memory does it
use? An initial monotonical increasement is normal... Could you write a
small script that I can run on my machine?

Regards,

Michael

 

[Torsten Senf]

I don't think so. How long do you run it? And how much memory does it
use? An initial monotonical increasement is normal... Could you write a
small script that I can run on my machine?

You got the server code in an older mail. Here is a client which should
work. Try it out with some different LogFiles. (different in size)

require "xmlrpc/client"

server = XMLRPC::Client.new("localhost","/RPC2", port=8080)

result = server.call("file.showlog", "/var/log/messages")
puts result

After using GC.stat; GC.enable in a methode it seems that the memory usage
do not increase and I can also view a small decreasing sometimes but not to the
initial memory. The initial memory is about 4MB. After calling the server
10 times the memory usage is about 7MB.


Torsten Senf

 

[Michael Neumann]

You got the server code in an older mail. Here is a client which should
work. Try it out with some different LogFiles. (different in size)

require "xmlrpc/client"

server = XMLRPC::Client.new("localhost","/RPC2", port=8080)

result = server.call("file.showlog", "/var/log/messages")
puts result

After using GC.stat; GC.enable in a methode it seems that the memory usage
do not increase and I can also view a small decreasing sometimes but not to the
initial memory. The initial memory is about 4MB. After calling the server
10 times the memory usage is about 7MB.

Ah okay, that's (IMHO) just normal in Ruby, so no memory leak. Not sure
whether this can be tuned, but I don't think so.
Try, whether it will permanently stay above 8MB, I claim it won't.

Regards,

Michael

 

[Nicholas Van Weerdenburg]

Is it maybe a threading issue, and the GC never getting a chance to
run based on XMLRPC internals?

Was there any root cause identified? Or is this a magic solution by
inserting GS calls?

Regards,
Nick