Memory concerns wrt garbage collection etc

S

Srijayanth Sridhar

Hello,

I have a script which basically launches several threads, each of them
parsing a web page from a different link using Hpricot. I am not sure
how many threads get spawned, could be in the hundreds. The question I
have is, assuming we have a simple scenario where:

doc=Hpricot.new(open("http://somewhere/"))
# a bunch of processing
doc=Hpricot.new(open("http://somewhere.else/"))
# some more processing here.


When does the memory from the first instance get released? As soon as
no references point to that chunk of memory or right at the end of the
scope of wherever that object was instantiated?

Can someone please tell me as I have to my script running for months
perhaps and I don't know if its going to sit there eating memory for
no reason :)

Thank you,

Jayanth
 
R

Roger Pack

doc=Hpricot.new(open("http://somewhere/"))
# a bunch of processing
doc=Hpricot.new(open("http://somewhere.else/"))
# some more processing here.


When does the memory from the first instance get released? As soon as
no references point to that chunk of memory or right at the end of the
scope of wherever that object was instantiated?

Theoretically as soon as you redefine doc you'll no longer have any
references to it so it will be collected when the GC "kicks in" or when
you call GC.start--whichever comes first.
GL.
-=R
 
R

Robert Klemme

2008/5/28 Srijayanth Sridhar said:
I have a script which basically launches several threads, each of them
parsing a web page from a different link using Hpricot. I am not sure
how many threads get spawned, could be in the hundreds.

I doubt the usefulness of this. Since Ruby uses green threads chances
are that your overall throughput could be better if you limit the
number of threads. This also has the effect of reducing the amount of
memory needed at a point in time. I'd seriously question this design.
If you have multiple cores in your system and want to take advantage
of them using multiple processes might also be an option.

Kind regards

robert
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top