C ext / threading / callback issue, ruby-1.9.1p129

Discussion in 'Ruby' started by reynhout@quesera.com, Jul 13, 2009.

  1. Guest

    Hi,

    I've never written a C extension for Ruby before, and I'm struggling with
    some demo code which uses C and/or Ruby pthreads, and a C or Ruby callback
    (address passed as an arg into the C lib).

    There's a watcher thread (in C) that waits for an event (the creation of
    a file), then triggers whichever callback is specified.

    Triggering the Ruby callback causes MRI to crash. The C callback works...
    but when it completes, all Ruby threads appear to suspend (control is
    never returned to Ruby / the threads are never rejoined?)

    I'm guessing that the MRI crash is a dereferencing bug?? I haven't figured
    that out obviously, but I have a feeling that the second problem (threads
    never rejoin?) will show up after the crash is resolved.

    I put the code up on github: http://github.com/reynhout/callme/ . There
    is a bunch more description there, too.

    There are two versions, C_ext (described above), and FFI_lib which uses
    ruby-ffi to achieve the same goals ... but is also failing similarly.
    I worked on the FFI_lib one first, then backed down to a straight C ext
    trying to simplify it and make the bugs easier to find.

    I'm pretty sure that if I can get the C_ext version working, the FFI_lib
    one will fall into place. The ultimate goal is a Ruby library for Apache
    ZooKeeper. The rest of the code is going great, but many ZK functions
    depend on this sort of mechanism..

    I hope the bugs are obvious to anyone who has written a C ext before.
    I'd greatly appreciate any ideas or pointers (no pun intended :)..

    http://github.com/reynhout/callme/

    Thank you!
    Andrew


    --
     
    , Jul 13, 2009
    #1
    1. Advertising

  2. Roger Pack Guest


    > There's a watcher thread (in C) that waits for an event (the creation of
    > a file), then triggers whichever callback is specified.


    Hmm. MRI isn't "used" to other threads running--are you running 1.8.6
    or something else?
    =r
    --
    Posted via http://www.ruby-forum.com/.
     
    Roger Pack, Jul 14, 2009
    #2
    1. Advertising

  3. Guest

    On Tue, Jul 14, 2009 at 10:57:07AM +0900, Roger Pack wrote:
    >
    > > There's a watcher thread (in C) that waits for an event (the creation of
    > > a file), then triggers whichever callback is specified.

    >
    > Hmm. MRI isn't "used" to other threads running--are you running 1.8.6
    > or something else?


    I'm using 1.9.1p129, but I tried the FFI version under 1.8.6 too, and
    got similar MRI crashes and/or stuck threads. I did wrap the callback
    invocation inside rb_thread_blocking_region, though I'm not sure it's
    necessary for this sample code.

    I haven't written a 1.8.6 version of the C ext (the syntax/semantics
    are slightly different than 1.9), but I'd be very surprised to see
    anything change -- these failures are consistent with MRI through
    multiple versions and approaches.

    This code DOES work correctly under JRuby. JRuby has a different
    threading model, of course, but I don't understand it well enough to
    determine why it fails under MRI.

    Sample code and details at: http://github.com/reynhout/callme/

    Thanks again,
    Andrew


    --
     
    , Jul 14, 2009
    #3
  4. Roger Pack Guest


    > I'm using 1.9.1p129, but I tried the FFI version under 1.8.6 too, and
    > got similar MRI crashes and/or stuck threads. I did wrap the callback
    > invocation inside rb_thread_blocking_region, though I'm not sure it's
    > necessary for this sample code.


    Does the callback use ruby code from within the thread blocking region?
    =r
    --
    Posted via http://www.ruby-forum.com/.
     
    Roger Pack, Jul 14, 2009
    #4
  5. Roger Pack wrote:
    > Does the callback use ruby code from within the thread blocking region?


    Yes, e.g.:

    rb_thread_blocking_region((rb_blocking_function_t *)w->callback, tmp_x,
    RUBY_UBF_IO, 0);

    w->callback is the address of the Ruby proc. The address is correct
    afaict, but note that this does crash MRI before the callback appears to
    execute (dereferencing failure?). When w->callback is the address of
    the C function, the callback runs, but the Ruby threads apparently
    suspend.

    Both callbacks in this sample code are very simple -- they just print
    the int value of tmp_x, which is the number of seconds since the watcher
    thread was started.

    Thanks,
    Andrew
    --
    Posted via http://www.ruby-forum.com/.
     
    Andrew Reynhout, Jul 14, 2009
    #5
  6. Luis Lavena Guest

    On Jul 14, 10:48 am, Andrew Reynhout <> wrote:
    > Roger Pack wrote:
    > > Does the callback use ruby code from within the thread blocking region?

    >
    > Yes, e.g.:
    >
    > rb_thread_blocking_region((rb_blocking_function_t *)w->callback, tmp_x,
    > RUBY_UBF_IO, 0);
    >
    > w->callback is the address of the Ruby proc.  The address is correct
    > afaict, but note that this does crash MRI before the callback appears to
    > execute (dereferencing failure?).  When w->callback is the address of
    > the C function, the callback runs, but the Ruby threads apparently
    > suspend.
    >
    > Both callbacks in this sample code are very simple -- they just print
    > the int value of tmp_x, which is the number of seconds since the watcher
    > thread was started.
    >


    Ruby-FFI issues, more related to callbacks can be asked at Ruby-ffi
    mailing list:

    http://kenai.com/projects/ruby-ffi/lists

    If memory plays well, I saw something related to callbacks not long
    ago there:

    http://kenai.com/projects/ruby-ffi/lists/dev/archive

    Sometimes the API hasn't been updated across the implementations.

    --
    Luis Lavena
     
    Luis Lavena, Jul 14, 2009
    #6
  7. Luis Lavena wrote:
    > Ruby-FFI issues, more related to callbacks can be asked at Ruby-ffi
    > mailing list:


    Sorry, I didn't mean to confuse the issue by mentioning FFI. I did try
    it with FFI initially (and have been discussing the problem on the
    ruby-ffi list), but I've completely removed FFI from the current version
    of the code. It's a straight C ext with no dependencies except gcc,
    MRI, and mkmf.

    Thanks,
    Andrew
    --
    Posted via http://www.ruby-forum.com/.
     
    Andrew Reynhout, Jul 14, 2009
    #7
  8. Roger Pack Guest


    > rb_thread_blocking_region((rb_blocking_function_t *)w->callback, tmp_x,
    > RUBY_UBF_IO, 0);


    Hmm. This question might be more easily answered by the core folk, but
    perhaps you're assigning the value of w->callback to a ruby proc object,
    when what you wanted was perhaps a pointer to a C method.

    So the question is how to call proc objects in C?
    I've never done it, but glancing at README.EXT appears to be something
    like rb_iterate or rb_block_call (the latter for 1.9 esp.)

    Not sure of that's right but it's about as far as my know-how goes.
    GL.
    =r
    --
    Posted via http://www.ruby-forum.com/.
     
    Roger Pack, Jul 15, 2009
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jindal, Pankaj

    ['ext.IsDOMString', 'ext.SplitQName']

    Jindal, Pankaj, Dec 22, 2004, in forum: Python
    Replies:
    0
    Views:
    330
    Jindal, Pankaj
    Dec 22, 2004
  2. Replies:
    11
    Views:
    613
  3. sa 125
    Replies:
    3
    Views:
    131
    Markus Fischer
    Jun 2, 2009
  4. J. D.
    Replies:
    12
    Views:
    174
    Roger Pack
    Jun 1, 2009
  5. Lee Hinman

    Threads + Forks in Ruby 1.9.1p129

    Lee Hinman, Jun 23, 2009, in forum: Ruby
    Replies:
    5
    Views:
    171
    Eleanor McHugh
    Jun 23, 2009
Loading...

Share This Page