rb_yield(), break, and C extensions

[Noah Easterly]

So, I'm working on a C extension.

One of my C functions uses a callback function, so when I wrap that
function in a method, I pass it a callback function that just wraps a
call to rb_yield().

This works great, except that when the block passed to the method
includes a 'break' statement, the end of the C function is bypassed,
and so it can't do its own cleanup.

What I'd like to do is "catch" the break somehow, so I can tell my C
function to exit and cleanup before restoring control to the ruby
script.

Is there any way to do this? I realize it could be a Bad Thing if
abused, but this is for good, I swear.

I guess I could try to pull all the memory allocation/file opening/etc
out to the containing object, but
[1] I'm not sure it's possible.
[2] It seems kind of awkward.
[3] This wouldn't solve the problem if I was linking in to an existing
object file that I didn't have source for.

Here's some toy code, if I didn't get my point across:
void func( int n, int (*callback)(void) )
{
void * mem = malloc(sizeof(int));
FILE * file = fopen("filename", "r");
int i;
// ... do stuff
for (i = 0; i < n; i++) { if (callback() < 0) { goto cleanup; } }
// ... do more stuff
cleanup:
if (mem) { free(mem); }
if (file) { fclose(file); }
}static int
meth_callback(void)
{
rb_yield(Qnil);
return 0;
}
static VALUE
yyy_meth(VALUE self, VALUE count)
{
func(FIX2INT(count), meth_callback);
return Qnil;
}require 'ext'
YYY.new.meth(100) { break }

 

[Kent Sibilev]

So, I'm working on a C extension.

One of my C functions uses a callback function, so when I wrap that
function in a method, I pass it a callback function that just wraps a
call to rb_yield().

This works great, except that when the block passed to the method
includes a 'break' statement, the end of the C function is bypassed,
and so it can't do its own cleanup.

You should wrap your rb_yield call with rb_ensure. From README.EXT:

VALUE rb_ensure(VALUE (*func1)(), void *arg1, void (*func2)(), void *arg2)

Calls the function func1 with arg1 as the argument, then calls func2
with arg2 if execution terminated. The return value from
rb_ensure() is that of func1.

 

[Noah Easterly]

You should wrap your rb_yield call with rb_ensure. From README.EXT:

VALUE rb_ensure(VALUE (*func1)(), void *arg1, void (*func2)(), void *arg2)

Calls the function func1 with arg1 as the argument, then calls func2
with arg2 if execution terminated. The return value from
rb_ensure() is that of func1.

Hmm.... this doesn't seem to have the desired effect. I'm pretty sure
rb_ensure() is just for handling exceptions, not breaks.

According to ruby.h and the Pickaxe, the function def'n has changed a
bit too. From Programming Ruby(2nd ed):

VALUE rb_ensure(VALUE(*body)(), VALUE args, VALUE(*ensure)(), VALUE
eargs)

Executes body with the given args. Whether or not an exception is
raised, execute ensure with the given eargs after body has completed.

So it seems that rb_ensure is just for exceptions, not 'break'. I
tried tweaking my code to use it anyway, but the ensure call is still
bypassed when there is a break. For example:
void func( int n, int (*callback)(void) )
{
printf("before callback...\n");
if (callback() < 0)
goto cleanup;
printf("after callback...\n");
cleanup:
printf("in cleanup...\n");
}static void
my_ensure(void * ptr)
{
*(int *)ptr = -1;
}
static int
meth_callback(void)
{
int retval = 0;
rb_ensure(rb_yield, Qnil, my_ensure, &retval);
return retval;
}
// ...
require 'ext'
YYY.new.meth(100) { } # outputs "before callback...\n", "in cleanup...
\n"
YYY.new.meth(100) { break } # outputs "before callback...\n"

Which makes sense if, when there is no break, the my_ensure func gets
called (as it should), so processing skips to cleanup, and when there
is a break, the ensure func does not get called, and processing skips
back to the ruby script.

I suppose I could switch to having people 'raise BreakException',
rather than 'break' within the block, but that seems like its hobbling
the ruby language.

Thanks for the idea, though!

 

[Rick DeNatale]

Hmm.... this doesn't seem to have the desired effect. I'm pretty sure
rb_ensure() is just for handling exceptions, not breaks.

According to ruby.h and the Pickaxe, the function def'n has changed a
bit too. From Programming Ruby(2nd ed):

VALUE rb_ensure(VALUE(*body)(), VALUE args, VALUE(*ensure)(), VALUE
eargs)

Executes body with the given args. Whether or not an exception is
raised, execute ensure with the given eargs after body has completed.

So it seems that rb_ensure is just for exceptions, not 'break'. I
tried tweaking my code to use it anyway, but the ensure call is still
bypassed when there is a break. For example:

void func( int n, int (*callback)(void) )
{
printf("before callback...\n");
if (callback() < 0)
goto cleanup;
printf("after callback...\n");
cleanup:
printf("in cleanup...\n");
}
static void
my_ensure(void * ptr)
{
*(int *)ptr = -1;
}
static int
meth_callback(void)
{
int retval = 0;
rb_ensure(rb_yield, Qnil, my_ensure, &retval);
return retval;
}
// ...

require 'ext'
YYY.new.meth(100) { } # outputs "before callback...\n", "in cleanup...
\n"
YYY.new.meth(100) { break } # outputs "before callback...\n"

Which makes sense if, when there is no break, the my_ensure func gets
called (as it should), so processing skips to cleanup, and when there
is a break, the ensure func does not get called, and processing skips
back to the ruby script.

I suppose I could switch to having people 'raise BreakException',
rather than 'break' within the block, but that seems like its hobbling
the ruby language.

Thanks for the idea, though!

It sounds like you want to do the equivalent of turning the given
block into a lambda. The break is causing a return from the block.

Here's some ruby code which shows the problem and the solution:

rick@frodo:/public/rubyscripts$ cat lambda.rb
def with_yield
puts "before yield"
yield
puts "got back"
end

def with_call(&block)
puts "before call"
block.call
puts "got back"
end

def with_lambda(&block)
puts "before lambda call"
(lambda &block).call
puts "got back"
end

with_yield {puts "in block"; break}
puts
with_call {puts "in block"; break}
puts
with_lambda {puts "in block"; break}
rick@frodo:/public/rubyscripts$ ruby lambda.rb
before yield
in block

before call
in block

before lambda call
in block
got back


Now just turn that into C code.

You need rb_scan_args with a format of "&" on the last argument to get
the block. rb_funcall2 to call the lambda method which is private

And you probably also want to use rb_ensure in case the block raises
and exception as well.

 

[Kent Sibilev]

Hmm.... this doesn't seem to have the desired effect. I'm pretty sure
rb_ensure() is just for handling exceptions, not breaks.

According to ruby.h and the Pickaxe, the function def'n has changed a
bit too. From Programming Ruby(2nd ed):

VALUE rb_ensure(VALUE(*body)(), VALUE args, VALUE(*ensure)(), VALUE
eargs)

Executes body with the given args. Whether or not an exception is
raised, execute ensure with the given eargs after body has completed.

So it seems that rb_ensure is just for exceptions, not 'break'. I
tried tweaking my code to use it anyway, but the ensure call is still
bypassed when there is a break. For example:

void func( int n, int (*callback)(void) )
{
printf("before callback...\n");
if (callback() < 0)
goto cleanup;
printf("after callback...\n");
cleanup:
printf("in cleanup...\n");
}
static void
my_ensure(void * ptr)
{
*(int *)ptr = -1;
}
static int
meth_callback(void)
{
int retval = 0;
rb_ensure(rb_yield, Qnil, my_ensure, &retval);
return retval;
}
// ...

require 'ext'
YYY.new.meth(100) { } # outputs "before callback...\n", "in cleanup...
\n"
YYY.new.meth(100) { break } # outputs "before callback...\n"

What I meant was that your cleanup code should be in my_ensure
function and not in func itself. Just to prove my point:

$ cat t.rb
def test
puts 'pre'
yield
puts 'post'
ensure
puts 'ensure'
end

test {break}

$ ruby t.rb
pre
ensure

 

[Kent Sibilev]

What I meant was that your cleanup code should be in my_ensure
function and not in func itself. Just to prove my point:

$ cat t.rb
def test
puts 'pre'
yield
puts 'post'
ensure
puts 'ensure'
end

test {break}

$ ruby t.rb
pre
ensure

OK, maybe this code will clean things up:

$ cat t.rb
require 'rubygems'
require 'inline'

class MyTest
inline do |builder|
builder.prefix <<-EOC
static VALUE
func(VALUE arg)
{
printf("pre\\n");
rb_yield(arg);
printf("post\\n");
return Qnil;
}
static VALUE
cleanup(VALUE arg)
{
printf("ensure\\n");
return Qnil;
}
EOC
builder.c <<-EOC
void my_func()
{
rb_ensure(func, Qnil, cleanup, Qnil);
}
EOC
end
end

MyTest.new.my_func {break}

$ ruby t.rb
pre
ensure
$

 

[Ryan Davis]

You should wrap your rb_yield call with rb_ensure. From README.EXT:

VALUE rb_ensure(VALUE (*func1)(), void *arg1, void (*func2)(), void
*arg2)

Calls the function func1 with arg1 as the argument, then calls func2
with arg2 if execution terminated. The return value from
rb_ensure() is that of func1.

Right, for an example of this, take a look at the inline'd code in
image_science. I just had to wrap all this stuff up in the same way.

 

[Rick DeNatale]

What I meant was that your cleanup code should be in my_ensure
function and not in func itself. Just to prove my point:

$ cat t.rb
def test
puts 'pre'
yield
puts 'post'
ensure
puts 'ensure'
end

test {break}

$ ruby t.rb
pre
ensure

Yes, this is better than my suggestion to wrap the block in a lambda
since it covers not only returns but exceptions as well.

 

[Noah Easterly]

OK, maybe this code will clean things up:
[...]

Now I understand. So, the rb_ensure() only makes sure that it's third
argument is called, not that anything after that in the function
containing rb_ensure() is run. That's what I thought it was doing -
which was leading me astray.

So the original definition of func (which allocated memory, used the
callback, freed memory, and didn't use ruby.h) won't work, I need to
divide that into 3 parts (one to alloc, one that uses the callback,
and one to free), and do something like:

static VALUE
my_meth(VALUE self)
{
MyType * my_type = AllocMyType();

rb_ensure(DoStuff, my_type, FreeMyType, my_type);
return Qnil;
}

Thanks for bearing with me and explaining.

 

[ts]

N> So the original definition of func (which allocated memory, used the
N> callback, freed memory, and didn't use ruby.h) won't work, I need to
N> divide that into 3 parts (one to alloc, one that uses the callback,
N> and one to free), and do something like:

Well, the best is to let ruby manage the memory (with its GC), but if you
have an original C function that you *can't* modify and you need to give
it a callback, you can still write *something* like this


moulon% cat a.c
#include <ruby.h>

void func( int n, int (*callback)(void) )
{
int i;

for (i = 0; i < n; i++) { if (callback() < 0) { goto cleanup; } }
rb_warn("normal exit");
cleanup:
rb_warn("cleanup");
return;
}

static int
meth_callback(void)
{
int state;

rb_protect(rb_yield, 0, &state);
if (state) {
rb_thread_local_aset(rb_thread_current(), rb_intern("_test"),
INT2NUM(state));
return -1;
}
return 0;
}

static VALUE
yyy_meth(VALUE self, VALUE count)
{
VALUE res;

rb_thread_local_aset(rb_thread_current(), rb_intern("_test"), INT2NUM(0));
func(FIX2INT(count), meth_callback);
res = rb_thread_local_aref(rb_thread_current(), rb_intern("_test"));
if (FIXNUM_P(res) && NUM2INT(res) != 0) {
rb_thread_local_aset(rb_thread_current(), rb_intern("_test"), INT2NUM(0));
rb_warn("rb_jump_tag");
rb_jump_tag(NUM2INT(res));
}
return Qnil;
}


void Init_a()
{
VALUE a_cM;

a_cM = rb_define_class("YYY", rb_cObject);
rb_define_method(a_cM, "meth", yyy_meth, 1);
}
moulon%

moulon% cat b.rb
#!/usr/bin/ruby
require 'a'

def m(a)
puts "before"
a.meth(6) { return }
puts "after"
end

puts "before m"
m(YYY.new)
puts "after m"
moulon%

moulon% ./b.rb
before m
before
/b.rb:6: warning: cleanup
/b.rb:6: warning: rb_jump_tag
after m
moulon%


Guy Decoux

 

[Nobuyoshi Nakada]

Hi,

At Mon, 9 Apr 2007 09:40:07 +0900,
Noah Easterly wrote in [ruby-talk:247204]:

So the original definition of func (which allocated memory, used the
callback, freed memory, and didn't use ruby.h) won't work, I need to
divide that into 3 parts (one to alloc, one that uses the callback,
and one to free), and do something like:

You can use rb_protect() and rb_jump_tag() too.

static VALUE
my_meth(VALUE self)
{
MyType * my_type = AllocMyType();

int status;
VALUE result = rb_protect(DoStuff, (VALUE)my_type, &status);
FreeMyType(my_type);
if (status) rb_jump_tag(status);
return result;

}

The way to do all in one function is not provieded.