Manual Memory Management and Automatic Garbage Collection

[Tridib Bandopadhyay]

Hello all

I am trying to do a research on Manual Memory Management and Automatic
Garbage Collection on Ruby.So i need to play with Garbage Collector code
of Ruby.I am using Ruby 1.8.6

I am also trying to figure out in Ruby when the Garbage Collector starts
working.But didn't got any clue.

I read somewhere in wiki about some systems that uses this design of
Memory Allocation.

Can any one help me in some of my queries:-

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

After I get my first step done I hope I will have many questions to
share with you all.

Regards...Eagerly waiting for a reply at the earliest

Tridib

 

[Jeremy Bopp]

Hello all

I am trying to do a research on Manual Memory Management and Automatic
Garbage Collection on Ruby.So i need to play with Garbage Collector code
of Ruby.I am using Ruby 1.8.6

I am also trying to figure out in Ruby when the Garbage Collector starts
working.But didn't got any clue.

I read somewhere in wiki about some systems that uses this design of
Memory Allocation.

Can any one help me in some of my queries:-

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

After I get my first step done I hope I will have many questions to
share with you all.

Regards...Eagerly waiting for a reply at the earliest

Have you tried sprinkling some printf statements throughout gc.c
yourself yet? If the diagnostic instrumentation for gc.c is hard to
find or does not exist, adding your own can at least help you make some
progress.

To start, you could add a printf with the function name at the beginning
of each function in gc.c. Then run your build of ruby with your test
scripts to trigger garbage collection and see what gets printed. That
should give you a lead on where to add more printf calls to gc.c in
order to gather more detailed information.

-Jeremy

 

[Robert Klemme]

I am trying to do a research on Manual Memory Management and Automatic
Garbage Collection on Ruby.So i need to play with Garbage Collector code
of Ruby.I am using Ruby 1.8.6

I am also trying to figure out in Ruby when the Garbage Collector starts
working.But didn't got any clue.

There were some threads here recently about GC. What specific
questions were left unanswered?

I read somewhere in wiki about some systems that uses this design of
Memory Allocation.

Can any one help me in some of my queries:-

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

You know CPP (C preprocessor) defines and macros, do you? Please look
for "GC_NOTIFY".

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

I haven't looked closely but I would start at the end of the file and
walk my way through reading from rb_gc_start(). You should hit all
main structures pretty soon.

After I get my first step done I hope I will have many questions to
share with you all.

OK, let's hear them.

Regards...Eagerly waiting for a reply at the earliest

Please don't push it. It won't increase your chances of getting answers anyway.
http://www.catb.org/~esr/faqs/smart-questions.html#urgent

Cheers

robert

 

[Ammar Ali]

Hello all

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

The GC_NOTIFY flag is not available in ruby 1.8, it's in 1.9 only. If
you have to use ruby 1.8, you can do it yourself by adding printf
calls at the begining an end of the rb_gc_start function in gc.c.
Under 1.9 the function where GC start is called gc_collect, or at
least that's where GC_NOTIFY is placed.

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

I think this was suggested before, it will answer many of your questions.

http://timetobleed.com/garbage-collection-slides-from-la-ruby-conference/

Regards,
Ammar

 

[Tridib Bandopadhyay]

Jeremy Bopp wrote in post #966541:

Have you tried sprinkling some printf statements throughout gc.c
yourself yet? If the diagnostic instrumentation for gc.c is hard to
find or does not exist, adding your own can at least help you make some
progress.

To start, you could add a printf with the function name at the beginning
of each function in gc.c. Then run your build of ruby with your test
scripts to trigger garbage collection and see what gets printed. That
should give you a lead on where to add more printf calls to gc.c in
order to gather more detailed information.

-Jeremy

Yes I have tried that long back.. for example in this code fragment--

VALUE
rb_newobj()
{
VALUE obj;
printf("newobj"); /*My own line*/
if (!freelist) garbage_collect();

obj = (VALUE)freelist;
freelist = freelist->as.free.next;
MEMZERO((void*)obj, RVALUE, 1);
#ifdef GC_DEBUG
RANY(obj)->file = ruby_sourcefile;
RANY(obj)->line = ruby_sourceline;
#endif
return obj;
}

But when I am making this change and trying to compile my ruby. It is
bombing out with a message Segmentation failure and it is printing my
print statement numerous times before bombing out.I also tried in my
other functions.

Also can you tell me when we compile the ruby with 'make' command how
does it changes the gc.c file in the executable file.

 

[Tridib Bandopadhyay]

Ammar Ali wrote in post #966575:

The GC_NOTIFY flag is not available in ruby 1.8, it's in 1.9 only. If
you have to use ruby 1.8, you can do it yourself by adding printf
calls at the begining an end of the rb_gc_start function in gc.c.
Under 1.9 the function where GC start is called gc_collect, or at
least that's where GC_NOTIFY is placed.

Regards,
Ammar

I tried in there also.Here is the code.--


rb_gc_start()
{
rb_gc();
return Qnil;
printf("Hello!!")
}

It is compiling but is not showing the hello message when i am running
the code.Here is my ruby code--

print "Please enter number 1 : ";
val1 = gets;
print "Please enter number 2 : ";
val2 = gets;
print "Answer : " , (val1.to_i + val2.to_i), "\n";
ct = 1;
while(ct<100)
a=[];
na = 0;
while(na<1000)
a[na]=na;
na = na+1;
end;
print ct;
ct=ct+1;
end;

How to see that hello message and how to know when Garbage Collector is
doing its work.

 

[Tridib Bandopadhyay]

On One function void rb_gc() I added one printf statement of my own.

And when I type the command -

/ruby -e 'GC.start()'

Then it is showing me my print statement.But I want to see my print
statement when i am running one of my ruby codes.

Regards

Tridib

 

[Ammar Ali]

I tried in there also.Here is the code.--


rb_gc_start()
{
=C2=A0 =C2=A0rb_gc();
=C2=A0 =C2=A0return Qnil;
=C2=A0 =C2=A0printf("Hello!!")
}

It will never work. You're printing after return!

Regards,
Ammar

 

[Tridib Bandopadhyay]

Ammar Ali wrote in post #966734:

It will never work. You're printing after return!

Regards,
Ammar

I shifted the printf statement above the return statement.Here is the =

output I am getting--

/ruby doug.rb
Please enter number 1 : 3
Please enter number 2 : 4
Answer : 7
1234567891011121314151617181920212223242526272829303132333435363738394041=
4243444546474849505152535455565758596061626364656667686970717273747576777=
8798081828384858687888990919293949596979899-bash-3.00$

Regards
Tridib

-- =

Posted via http://www.ruby-forum.com/.=

 

[Walton Hoops]

I shifted the printf statement above the return statement.Here is the
output I am getting--

./ruby doug.rb
Please enter number 1 : 3
Please enter number 2 : 4
Answer : 7
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899-bash-3.00$

Most likely Garbage Collection is not being run for your (rather short)
script. Does your message print when you run GC.start()? If so your
message is working fine. Try writing something that creates a log of
garbage to be collected. I think something like:

i=1
while (true)
puts "message #{i}"
i+=1
end

should do the trick. Of course, that particular program will never exit.

 

[Tridib Bandopadhyay]

Walton Hoops wrote in post #966746:
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899-bash-3.00$

Most likely Garbage Collection is not being run for your (rather short)
script. Does your message print when you run GC.start()? If so your
message is working fine. Try writing something that creates a log of
garbage to be collected. I think something like:

i=1
while (true)
puts "message #{i}"
i+=1
end

should do the trick. Of course, that particular program will never
exit.

Yes When I can see my message when I run my GC.start().So When does the
garbage collector comes into play?.When we call them or isn't it
automatic after each object declaration it will always come for
cleaning.

But can't I see the message when I am running a program of my own.

Secondly how does the make command do interpret the gc.c file into
*.ext?Is there any mechanism?.

Regards

Tridib

 

[Jeremy Bopp]

Jeremy Bopp wrote in post #966541:



Yes I have tried that long back.. for example in this code fragment--

VALUE
rb_newobj()
{
VALUE obj;
printf("newobj"); /*My own line*/
if (!freelist) garbage_collect();

obj = (VALUE)freelist;
freelist = freelist->as.free.next;
MEMZERO((void*)obj, RVALUE, 1);
#ifdef GC_DEBUG
RANY(obj)->file = ruby_sourcefile;
RANY(obj)->line = ruby_sourceline;
#endif
return obj;
}

But when I am making this change and trying to compile my ruby. It is
bombing out with a message Segmentation failure and it is printing my
print statement numerous times before bombing out.I also tried in my
other functions.

Ruby creates plenty of objects during its normal operation, so this
function will be called frequently. It makes sense then that you'll see
your output many times. What doesn't make sense is the segmentation fault.

I would bet that you had more changes than just that single line in gc.c
when you built your modified version of Ruby. If that's the case, it's
likely that one of your other changes is the culprit. I suggest that
you try again from a completely fresh set of sources and add only that
single line to see if you can avoid the problem.

Also can you tell me when we compile the ruby with 'make' command how
does it changes the gc.c file in the executable file.

I'm not sure I understand what you're asking unless you're really asking
for the details of how the compiler works. Generally, the compiler is
used to compile each source file into a corresponding object (*.o) file.
Once all object files are created, they are then linked into the ruby
executable. In this case, gc.c would be compiled into gc.o which would
ultimately be linked into ruby.

The make program is usually smart enough to only rebuild the .o files
whose corresponding .c files were updated since the last build run, so
if you only change gc.c, only gc.c should be recompiled. Then the ruby
program should be recreated by linking the new gc.o file in along with
all the other .o files.

None of that really matters for you though. All you need to do is make
your changes to gc.c and then run make. You should see some output
during the make process that indicates that gc.c is being compiled and
that ultimately a new ruby file is created. Try not to get sidetracked
on the details.

-Jeremy

 

[Jeremy Bopp]

Yes When I can see my message when I run my GC.start().So When does the
garbage collector comes into play?.When we call them or isn't it
automatic after each object declaration it will always come for
cleaning.

Garbage collection is expensive, so the garbage collector's operation is
usually postponed until there is a large enough unit of work to warrant
its cost of operation. The nice thing about this is that the GC doesn't
have to run at all for sufficiently simple scripts. This will make such
scripts much faster than if you run the GC as a result of every object's
allocation.

But can't I see the message when I am running a program of my own.

If you want to see the GC run on its own, you need to cause enough
objects and memory to be allocated to trigger it. I can't personally
say what that trigger is, but it's probably a safe bet that allocating
some large number of objects is one such trigger. Walton's example does
that by creating a new string in each iteration of the while loop.

Secondly how does the make command do interpret the gc.c file into
*.ext?Is there any mechanism?.

I covered this to some extent in another response in this thread.
Obviously, there is a mechanism or you wouldn't get a ruby program to
run with your scripts in the first place. ;-) The C compiler is used
under the covers, and the make program automates the process.

Why do you ask? Do you suspect that your changes are not making it into
your ruby program for some reason?

From the sound of things, you don't really have much of a grasp on the
theory of garbage collection. You might find your task much easier if
you go read up more on the general topic and perhaps some discussions of
particular implementations. You should find plenty of that information
if you search around for Java garbage collection for instance. Without
more of a background in the theory, it's going to take you much more
time to understand Ruby's implementation.

-Jeremy

 

[Tridib Bandopadhyay]

Thank You all for your responses...I am getting a clear view of the Ruby
garbage collector and the compiler...

Regards

Tridib

 

[Tridib Bandopadhyay]

I tried this code fragment..

i=1
while (true)
puts "message #{i}"
i+=1
end

But this one is ending with a message Infinity but I can't see my own
print statement when the Garbage Collector is coming...

Regards

Tridib

 

[Tridib Bandopadhyay]

I can see the garbage collector coming when I type.

/ruby -e 'GC.start()'

But I want to know while a code is running the printf statement written
by me should appear.

Is there any other code to make gc show my message.

This code is running for infinite time..but not showing the message.


i=1
while (true)
puts "message #{i}"
i+=1
end

What are the other ways?

I need to convince myself that when there is memory allocation the
Garbage collector comes for cleaning with a printf statement stating its
work.

Regards
Tridib

 

[Jeremy Bopp]

I can see the garbage collector coming when I type.

./ruby -e 'GC.start()'

But I want to know while a code is running the printf statement written
by me should appear.

Is there any other code to make gc show my message.

This code is running for infinite time..but not showing the message.


i=1
while (true)
puts "message #{i}"
i+=1
end

What are the other ways?

I need to convince myself that when there is memory allocation the
Garbage collector comes for cleaning with a printf statement stating its
work.

Maybe there are optimizations for String objects that circumvent parts
of regular garbage collection. Have you tried looking at the memory
consumption of your ruby process as you run the while loop? Is memory
usage growing over time, or does it plateau at some point?

Try using your ruby executable to run some non-trivial projects
available online. Once you find one that triggers garbage collection
the way you want it, you could try to boil that down to something more
concise.

Another suggestion would be to pick up a template library such as erb
and use that to process some large template files with random data
repeatedly. That should hopefully cause some memory growth with
something more than simple strings.

-Jeremy

 

[Tridib Bandopadhyay]

Jeremy Bopp wrote in post #967674:

Maybe there are optimizations for String objects that circumvent parts
of regular garbage collection. Have you tried looking at the memory
consumption of your ruby process as you run the while loop? Is memory
usage growing over time, or does it plateau at some point?

The plateau is at the same point.Even if I ran the code for at-least
half and Hour.

Another suggestion would be to pick up a template library such as erb
and use that to process some large template files with random data
repeatedly. That should hopefully cause some memory growth with
something more than simple strings.

Sorry..I didn't understand this part..

Regards

Tridib

 

[Jeremy Bopp]

Jeremy Bopp wrote in post #967674:

Sorry..I didn't understand this part..

Use the ERB class:

http://rdoc.info/docs/ruby-stdlib/1.8.7/ERB

Create a handful of files to be used as templates of the form required
for that class. Ensure that the templates have data you can replace.
Process the templates with generated data in a loop.

We're basically making what should hopefully be a non-trivial form of
the while loop you've been using that prints a string with a number
embedded. Hopefully, this will allocate a sufficient number of objects
in a way that will eventually trigger garbage collection. If it doesn't
work, get creative and try to figure out some other way to lots of
non-trivial objects.

-Jeremy

 

[Tridib Bandopadhyay]

I have some questions::--

1. Is it possible to Manually free memory(without calling GC.start) by
extending it with C? As i have tried it and its of no use for my work.

2. Can I add a new file and try to manually free memory(disabling GC)?

Regards

Tridib