Chasing a garbage collection bug

T

Thomas Sondergaard

I just discovered that I have a GC related bug, or that is to say it doesn't
seem to occur when I disable the GC with GC.disable.

Are there any special GC-related bug finding techniques that are worth
mentioning?

Cheers,

Thomas
 
T

Thomas Sondergaard

Are there any special GC-related bug finding techniques that are worth
mentioning?

For starters, is there a way to make the GC print some information about the
object it reaps?

Cheers,

Thomas
 
S

Simon Strandgaard

For starters, is there a way to make the GC print some information about the
object it reaps?

Cheers,

Thomas


Are you writing an extension for Ruby in C ?

If so then you can add a printf statement in the free-instance function.
Like this C++ example:

void MyclassFree(Myclass *p) {
printf("we got killed\n");
delete p;
}

VALUE MyclassAlloc(VALUE self) {
Myclass* p = new Myclass();
return Data_Wrap_Struct(self, 0, MyclassFree, p);
}

void Myclass::RubyInit() {
VALUE module = rb_define_module("Embed");
VALUE h = rb_define_class_under(module, "Myclass", rb_cObject);
rb_define_alloc_func(h, MyclassAlloc);
}
 
J

Joel VanderWerf

T

Thomas Sondergaard

Q1) I am wondering about the rb_gc_register_address/rb_gc_unregister_address
function pairs.

The both take a VALUE pointer rather than a VALUE. Does that mean that to
unregister I have to pass it the exact same VALUE pointer, or will another
VALUE pointer pointing to the same object be equivalent. I'm asking because
I might rb_gc_register_address and rb_gc_unregister_address multiple times
for the same VALUE, but only once with each VALUE pointer.

Here's a scenario:

VALUE array = rb_ary_new(); // At some point this array is created
somewhere in ruby

.... a little later ...

// I need to pass a reference of the object out of ruby's reach
// so I 'lock' the object with rb_gc_register_address and keep the arrayRef
// pointer with me somewhere safe
VALUE *arrayRef = ALLOC(VALUE);
*arrayRef = array;
rb_gc_register_address(arrayRef)

..... a little later ...

// I need to pass another reference out and do the exact same thing again
VALUE *arrayRef2 = ALLOC(VALUE);
*arrayRef2 = array;
rb_gc_register_address(arrayRef2)

.... a little later again ...

// The second non-ruby object referencing the ruby object is dead and gone
and I
// unregister using the same pointer as I registered with
rb_gc_unregister_address(arrayRef2);
// I then free the pointer that I allocated, assuming that is my
responsibility
free(arrayRef2);

.... a little later again ....

// I do it again because the first non-ruby object referencing the ruby
object is gone
rb_gc_unregister_address(arrayRef);
free(arrayRef);

// At this point the ruby object should be ready to be garbage collected
unless there
// are still references to it in ruby.


Is this the way to do it?



Q2) Another thing that I am wondering about. Can the GC be invoked in the
middle of my C code or does that only happen when control is in the "guts of
ruby". What prevents the VALUEs I have on the stack from being GC'ed in that
case?


/Thomas
 
T

ts

T> Here's a scenario:

The best is to simplify your code to a few lines, and *post* it
completely.

T> Q2) Another thing that I am wondering about. Can the GC be invoked in the
T> middle of my C code or does that only happen when control is in the "guts of
T> ruby". What prevents the VALUEs I have on the stack from being GC'ed in that
T> case?

The GC can't be invoked in the middle of your C code, if you don't call
ruby functions.


Guy Decoux
 
S

Sean O'Dell

Thomas said:
Q1) I am wondering about the rb_gc_register_address/rb_gc_unregister_address
function pairs.

The both take a VALUE pointer rather than a VALUE. Does that mean that to
unregister I have to pass it the exact same VALUE pointer, or will another
VALUE pointer pointing to the same object be equivalent. I'm asking because
I might rb_gc_register_address and rb_gc_unregister_address multiple times
for the same VALUE, but only once with each VALUE pointer.

It uses the address, not the value.
Here's a scenario:

VALUE array = rb_ary_new(); // At some point this array is created
somewhere in ruby

... a little later ...

// I need to pass a reference of the object out of ruby's reach
// so I 'lock' the object with rb_gc_register_address and keep the arrayRef
// pointer with me somewhere safe
VALUE *arrayRef = ALLOC(VALUE);
*arrayRef = array;
rb_gc_register_address(arrayRef)
[...snip...]

Is this the way to do it?

That way would work fine, I think, but it seems a little convoluted.

If your C/C++ object has a VALUE member, you need only register the
address of that just once, and unregister it in the destructor. As long
as your C/C++ object is around, the VALUE is registered and the object
it is tied to will not go away.

Also, if the VALUE you need is just temporary for while you're in a C
call, just use a regular VALUE stack variable; they're protected (see
below) while in-use and cleaned up automatically when you're done with them.

Inversely, if you're trying to tie an C/C++ object/data to a Ruby
object, just use malloc to allocated it and associate it with a Ruby
object with Data_Wrap_Struct and provide a free function (use the normal
free/malloc routines).
Q2) Another thing that I am wondering about. Can the GC be invoked in the
middle of my C code or does that only happen when control is in the "guts of
ruby". What prevents the VALUEs I have on the stack from being GC'ed in that
case?

The GC can be invoked whenever you call any Ruby function. The Ruby GC
scans the stack for VALUE variables and "marks" any it finds which point
to valid objects. So long as the current stack pointer is
deeper/further into the stack than your VALUE variable, it is protected.
When your C/C++ function returns, it is eligible for collection and
will be left unmarked during the next GC sweep.

One gotcha I learned about this is, the object must really be on the
stack. Compilers these days all optimize, so to be sure your variable
actually lives on the stack and not in a register somewhere, all VALUEs
that you want marked as in-use, you must declare it as volatile, i.e.:

volatile VALUE tempArray;


Sean O'Dell
 
M

Mauricio Fernández

One gotcha I learned about this is, the object must really be on the
stack. Compilers these days all optimize, so to be sure your variable
actually lives on the stack and not in a register somewhere, all VALUEs
that you want marked as in-use, you must declare it as volatile, i.e.:

volatile VALUE tempArray;

It would probably make sense to

#define SVALUE volatile VALUE

--
_ _
| |__ __ _| |_ ___ _ __ ___ __ _ _ __
| '_ \ / _` | __/ __| '_ ` _ \ / _` | '_ \
| |_) | (_| | |_\__ \ | | | | | (_| | | | |
|_.__/ \__,_|\__|___/_| |_| |_|\__,_|_| |_|
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

'Ooohh.. "FreeBSD is faster over loopback, when compared to Linux
over the wire". Film at 11.'
-- Linus Torvalds
 
T

Thomas Sondergaard

If your C/C++ object has a VALUE member, you need only register the
address of that just once, and unregister it in the destructor. As long
as your C/C++ object is around, the VALUE is registered and the object
it is tied to will not go away.

That is exactly what I am doing I just wanted to spell out the scenario as
it may happen when there are more than one C++ wrapped reference to a ruby
object.

I'll try that volatile thing and see if it makes a difference.

Cheers,

Thomas
 
S

Sean O'Dell

Thomas said:
That is exactly what I am doing I just wanted to spell out the scenario as
it may happen when there are more than one C++ wrapped reference to a ruby
object.

I'll try that volatile thing and see if it makes a difference.

Hmm ... I'm confused about that statement.

If the VALUE is a member of a class, then it's probably NOT on the
stack, so volatile won't help you there. In this case, just use
rb_gc_register_address/rb_gc_unregister_address in your
constructor/destructor methods for your C++ class.

Sean O'Dell
 
T

Thomas Sondergaard

Hmm ... I'm confused about that statement.

Well that wont do..
If the VALUE is a member of a class, then it's probably NOT on the
stack, so volatile won't help you there. In this case, just use
rb_gc_register_address/rb_gc_unregister_address in your
constructor/destructor methods for your C++ class.

1) I am using rb_gc_register/unregister in my C++ wrapper.

2) I have now added the volatile keyword in all the functions where I use an
automatic variable.


Here's the relevant bit of the C++ wrapper:

public __gc class Object : public DynamicLanguageSupport::IObject {
public:
explicit Object(VALUE handle) {
_handle = ALLOC(VALUE);
*_handle = handle;
rb_gc_register_address(_handle);
}

~Object() {
rb_gc_unregister_address(_handle);
free(_handle);
}

// stuff for invoking ruby methods from .net removed

private:
VALUE __nogc *_handle;
};

Does it look reasonable?

Cheers,

Thomas
 
S

Sean O'Dell

Thomas said:
Well that wont do..




1) I am using rb_gc_register/unregister in my C++ wrapper.

2) I have now added the volatile keyword in all the functions where I use an
automatic variable.


Here's the relevant bit of the C++ wrapper:

public __gc class Object : public DynamicLanguageSupport::IObject {
public:
explicit Object(VALUE handle) {
_handle = ALLOC(VALUE);
*_handle = handle;
rb_gc_register_address(_handle);
}

~Object() {
rb_gc_unregister_address(_handle);
free(_handle);
}

// stuff for invoking ruby methods from .net removed

private:
VALUE __nogc *_handle;
};

Does it look reasonable?

You don't need to ALLOC the VALUE though. You can just make it a
regular member of your class and register the &address of it. When you
get the &address of a member variable, it's true location in memory is
given, and as far as I know that won't ever change during the lifetime
of your object, so you can really just do it like this:

public __gc class Object : public DynamicLanguageSupport::IObject {
public:
explicit Object(VALUE handle) {
rb_gc_register_address(&_handle);
}

~Object() {
rb_gc_unregister_address(&_handle);
}

// stuff for invoking ruby methods from .net removed

private:
VALUE __nogc _handle;
};


Aside from that, it sounds like you have got the knack of it!

Sean O'Dell
 
T

Thomas Sondergaard

You don't need to ALLOC the VALUE though. You can just make it a
regular member of your class and register the &address of it. When you
get the &address of a member variable, it's true location in memory is
given, and as far as I know that won't ever change during the lifetime
of your object, so you can really just do it like this:

That need not be true for .net objects. If you hold references to .net
objects in native code you need to tell the CLR to pin the object.

Thomas
 
T

Thomas Sondergaard

~Object() {
I just thought of something. This .net finalizer is called asynchronously
and since ruby is not thread safe that might cause problem if ruby is
traversing the linked list that the finalizer above is removing an element
from. Tomorrow I will change the finalizer so it insteads adds the _handle
pointer to an ArrayList that I can read from the "ruby thread" when I feel
like it. A very good time to do this would be on the GC thread just before
the actual GC'ing starts.

Is there a gc_start_hook that I can call? I imagine something like this:

GC.gc_start_hook {
// calls rb_gc_unregister() for all the elements in the array list I
mentioned above
RubyDotNet::GcUtil.unregister_all()
}

Cheers,

Thomas
 
S

Sean O'Dell

Thomas said:
That need not be true for .net objects. If you hold references to .net
objects in native code you need to tell the CLR to pin the object.

Heh ... my last MS compiler is from MSVC 4.0, circa 1995. That hasn't
been installed in probably 2 years or better, either.

Is there a Wiki I can go edit somewhere to include all these little
details regarding extensions, etc.? This stuff has really been tough to
get a handle on because all the formal documentation I've read in books
and online is all very simplified, and this stuff bites people in the
behind, but I've got a good handle on things and I'd really like to find
a place where I can go insert all this triviata.

Anyone have a URL to a somewhat official Wiki on this topic?

Sean O'Dell
 
T

Thomas Sondergaard

Is there a Wiki I can go edit somewhere to include all these little
details regarding extensions, etc.? This stuff has really been tough to
get a handle on because all the formal documentation I've read in books
and online is all very simplified, and this stuff bites people in the
behind, but I've got a good handle on things and I'd really like to find
a place where I can go insert all this triviata.

Excellent idea. I for one had many questions after having read the chapter
on extensions in the pick axe and perused the README.EXT from the
distribution.
 
S

Sean O'Dell

Thomas said:
I just thought of something. This .net finalizer is called asynchronously
and since ruby is not thread safe that might cause problem if ruby is
traversing the linked list that the finalizer above is removing an element
from. Tomorrow I will change the finalizer so it insteads adds the _handle
pointer to an ArrayList that I can read from the "ruby thread" when I feel
like it. A very good time to do this would be on the GC thread just before
the actual GC'ing starts.

Is there a gc_start_hook that I can call? I imagine something like this:

GC.gc_start_hook {
// calls rb_gc_unregister() for all the elements in the array list I
mentioned above
RubyDotNet::GcUtil.unregister_all()
}

What you could do is create sort of a "global collector" object that is
a C++ object which collects VALUEs and "marks" them when the GC is invoked.

Create this object, and then wrap it with Data_Wrap_Struct. Register
the VALUE returned as a global, so the collection never goes away so
long as the Ruby environment is loaded.

In your constructors, add their VALUE members to the collector. In your
destructors, remove them. Block on a mutex for both actions.

When your collector object's free is called, go ahead and free the
collection object; Ruby is going away anyway at that point, so there's
little you can do about possible dangling VALUEs in living C++ objects.

When the mark function is called, block on the mutex and "mark" all the
VALUEs it is holding.

That will keep the VALUEs you are maintaining as members of your C++
objects around so long as the objects themselves are around. When their
destructor (aka finalizer) is called, they will tell the collector
object to stop marking them, and the GC will collect them eventually.
All access to the list of VALUEs that the collector manipulates blocks
on a single mutex, so it doesn't matter what thread your objects "die" from.

Sean O'Dell
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top