[ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7 and1.9

Discussion in 'Ruby' started by Charles Oliver Nutter, Oct 31, 2008.

  1. The JRuby team is proud to announce the release of FFI for Ruby 1.8.6/7
    and 1.9!

    FFI (gem install ffi) is a library for programmatically loading dynamic
    libraries, binding functions within them, and calling those functions
    from Ruby code. Here's a quick sample of binding and calling the getpid
    C library function:


    require 'ffi'

    module GetPid
    extend FFI::Library

    attach_function :getpid, [], :uint
    end

    puts GetPid.getpid


    Here's another, calling qsort and passing a Ruby block as a C callback:


    require 'ffi'

    module LibC
    extend FFI::Library
    callback :qsort_cmp, [ :pointer, :pointer ], :int
    attach_function :qsort, [ :pointer, :int, :int, :qsort_cmp ], :int
    end

    p = MemoryPointer.new:)int, 2)
    p.put_array_of_int32(0, [ 2, 1 ])
    puts "Before qsort #{p.get_array_of_int32(0, 2).join(', ')}"
    LibC.qsort(p, 2, 4) do |p1, p2|
    i1 = p1.get_int32(0)
    i2 = p2.get_int32(0)
    i1 < i2 ? -1 : i1 > i2 ? 1 : 0
    end
    puts "After qsort #{p.get_array_of_int32(0, 2).join(', ')}"


    I posted a blog entry with a longer description of the library,
    additional examples, and links to some other documentation and posts.
    Docs are a little slim at this point, so feel free to experiment and
    update the JRuby wiki page:

    http://wiki.jruby.org/wiki/Calling_C_from_JRuby

    I'm sure docs from here will filter back into the library and out into
    the general cosmos.

    Finally, there's no need to write a C extension to call C libraries, and
    the same FFI code will work in Ruby 1.8.6/7, Ruby 1.9, JRuby 1.1.4+, and
    Rubinius (though Rubinius has no callback support yet).

    Don't be an extension stooge! Use FFI!

    - Charlie
    Charles Oliver Nutter, Oct 31, 2008
    #1
    1. Advertising

  2. Charles Oliver Nutter

    Thomas Hurst Guest

    Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    * Charles Oliver Nutter () wrote:

    > The JRuby team is proud to announce the release of FFI for Ruby
    > 1.8.6/7 and 1.9!


    > Finally, there's no need to write a C extension to call C libraries,
    > and the same FFI code will work in Ruby 1.8.6/7, Ruby 1.9, JRuby
    > 1.1.4+, and Rubinius (though Rubinius has no callback support yet).


    Awesome. I've used DL to link up some custom libs to a Ruby service,
    will give FFI a go and see how it compares :)

    Are things like structs likely to be supported in future, ala Python
    ctypes?

    --
    Thomas 'Freaky' Hurst
    http://hur.st/
    Thomas Hurst, Nov 1, 2008
    #2
    1. Advertising

  3. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Thomas Hurst wrote:
    > * Charles Oliver Nutter () wrote:
    >
    >> The JRuby team is proud to announce the release of FFI for Ruby
    >> 1.8.6/7 and 1.9!

    >
    >> Finally, there's no need to write a C extension to call C libraries,
    >> and the same FFI code will work in Ruby 1.8.6/7, Ruby 1.9, JRuby
    >> 1.1.4+, and Rubinius (though Rubinius has no callback support yet).

    >
    > Awesome. I've used DL to link up some custom libs to a Ruby service,
    > will give FFI a go and see how it compares :)
    >
    > Are things like structs likely to be supported in future, ala Python
    > ctypes?


    Actually structs are already supported! See the blog post, and I believe
    there's some examples shipped with the gem. There needs to be more docs,
    certainly, and hopefully they'll get some TLC soon.

    Also, I forgot to call out Evan Phoenix for coming up with the API and
    initial design, and he or someone else on Rubinus wrote up the
    templating/header-file-processing stuff as well. And of course a huge
    thanks to Wayne Meissner for implementing FFI not just once (for JRuby)
    but twice (for C Ruby). His work will mean a huge leap forward in
    cross-impl portability.

    - Charlie
    Charles Oliver Nutter, Nov 1, 2008
    #3
  4. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    T24gU2F0LCBOb3YgMSwgMjAwOCBhdCA2OjQwIEFNLCBDaGFybGVzIE9saXZlciBOdXR0ZXIKPGNo
    YXJsZXMubnV0dGVyQHN1bi5jb20+IHdyb3RlOgo+IEFjdHVhbGx5IHN0cnVjdHMgYXJlIGFscmVh
    ZHkgc3VwcG9ydGVkISBTZWUgdGhlIGJsb2cgcG9zdCwgYW5kIEkgYmVsaWV2ZQo+IHRoZXJlJ3Mg
    c29tZSBleGFtcGxlcyBzaGlwcGVkIHdpdGggdGhlIGdlbS4KCllvdXIgYmxvZyBpcyBwcmVhdHkg
    a25vd24gYnV0IGZvciBjbGFyaXR5OgpodHRwOi8vYmxvZy5oZWFkaXVzLmNvbS8yMDA4LzEwL2Zm
    aS1mb3ItcnVieS1ub3ctYXZhaWxhYmxlLmh0bWwKOi0pCgotLSAKUmFkb3OzYXcgQnWzYXQKCmh0
    dHA6Ly9yYWRhcmVrLmpvZ2dlci5wbCAtIG3zaiBibG9nCg==
    Rados³aw Bu³at, Nov 1, 2008
    #4
  5. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Radosław Bułat wrote:
    > On Sat, Nov 1, 2008 at 6:40 AM, Charles Oliver Nutter
    > <> wrote:
    >> Actually structs are already supported! See the blog post, and I believe
    >> there's some examples shipped with the gem.

    >
    > Your blog is preaty known but for clarity:
    > http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
    > :)


    Oh right, a link would have been useful. Thank you :)

    - Charlie
    Charles Oliver Nutter, Nov 1, 2008
    #5
  6. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Charles Oliver Nutter wrote:
    > I posted a blog entry with a longer description of the library,
    > additional examples, and links to some other documentation and posts.


    And then I completely forgot to include the blog post URL...

    http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html

    - Charlie
    Charles Oliver Nutter, Nov 1, 2008
    #6
  7. Charles Oliver Nutter

    Luc Heinrich Guest

    Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    On 1 nov. 08, at 11:19, Charles Oliver Nutter wrote:

    > And then I completely forgot to include the blog post URL...


    Looks cool, great work.

    Two questions:

    - Are variadic functions supported?
    - Do you have any idea or measurements of the overhead of calling
    through FFI as opposed to using a compiled extension?

    --
    Luc Heinrich -
    Luc Heinrich, Nov 1, 2008
    #7
  8. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Luc Heinrich wrote:
    > On 1 nov. 08, at 11:19, Charles Oliver Nutter wrote:
    >
    >> And then I completely forgot to include the blog post URL...

    >
    > Looks cool, great work.
    >
    > Two questions:
    >
    > - Are variadic functions supported?
    > - Do you have any idea or measurements of the overhead of calling
    > through FFI as opposed to using a compiled extension?


    Wayne answers the latter, sorta, on his followup blog post:

    http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html

    He doesn't have specific numbers for performance at the moment, but the
    short story is that FFI introduces a bit of overhead; ultimately I
    believe that the overhead gets lost in the flow of a Ruby application,
    especially when you're tossing units of work across like SQL queries or
    arrays. Wayne probably can fill in more details on what the actual
    overhead is like.

    And I'd also expect that any small amount of overhead is vastly
    outweighed by the ability to write an FFI-based library once and use it
    across implementations.

    - Charlie
    Charles Oliver Nutter, Nov 1, 2008
    #8
  9. Charles Oliver Nutter

    Luc Heinrich Guest

    Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    On 1 nov. 08, at 12:16, Charles Oliver Nutter wrote:

    > He doesn't have specific numbers for performance at the moment, but
    > the short story is that FFI introduces a bit of overhead; ultimately
    > I believe that the overhead gets lost in the flow of a Ruby
    > application, especially when you're tossing units of work across
    > like SQL queries or arrays.


    # --- [begin unscientific test] --------

    require 'rubygems'
    require 'benchmark'
    require 'zlib'
    require 'ffi'
    require 'dl/import'

    module Zlib_ffi
    extend FFI::Library
    attach_function :zlib_version, :zlibVersion, [], :string
    end

    module Zlib_dl
    extend DL::Importable
    dlload "libz.dylib"
    extern "const char* zlibVersion()"
    end

    puts Zlib.zlib_version
    puts Zlib_ffi.zlib_version
    puts Zlib_dl.zlibVersion

    Benchmark.bm(3) do |bm|
    bm.report("ext") { 500_000.times { Zlib.zlib_version } }
    bm.report("ffi") { 500_000.times { Zlib_ffi.zlib_version } }
    bm.report("dl") { 500_000.times { Zlib_dl.zlibVersion } }
    end

    # --- [end unscientific test] --------

    This gives the following results:

    1.2.3
    1.2.3
    1.2.3
    user system total real
    ext 1.050000 0.320000 1.370000 ( 1.373800)
    ffi 2.160000 0.660000 2.820000 ( 2.818966)
    dl 3.500000 1.060000 4.560000 ( 4.552789)

    All this using MacPorts MRI 1.8.7-p72 under OS X 10.5.5. The observed
    overhead is slightly over 2x for ffi, probably not a big deal unless
    ffi calls are used in tight loops I guess.


    PS: haven't seen any trace of variadic function support in the code.

    --
    Luc Heinrich -
    Luc Heinrich, Nov 1, 2008
    #9
  10. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    V2hlcmUgY2FuIEkgZmlsZSBhbiBpc3N1ZT8gSSBoYXZlIHRyb3VibGUgdG8gYnVpbGQgaW4gd2l0
    aCBydWJ5MS45IG9uIDY0Yml0LgoKLS0gClJhZG9zs2F3IEJ1s2F0CgpodHRwOi8vcmFkYXJlay5q
    b2dnZXIucGwgLSBt82ogYmxvZwo=
    Rados³aw Bu³at, Nov 1, 2008
    #10
  11. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    PiBUaGlzIGdpdmVzIHRoZSBmb2xsb3dpbmcgcmVzdWx0czoKPgo+IDEuMi4zCj4gMS4yLjMKPiAx
    LjIuMwo+ICAgICAgICAgdXNlciAgICAgc3lzdGVtICAgICAgdG90YWwgICAgICAgIHJlYWwKPiBl
    eHQgIDEuMDUwMDAwICAgMC4zMjAwMDAgICAxLjM3MDAwMCAoICAxLjM3MzgwMCkKPiBmZmkgIDIu
    MTYwMDAwICAgMC42NjAwMDAgICAyLjgyMDAwMCAoICAyLjgxODk2NikKPiBkbCAgIDMuNTAwMDAw
    ICAgMS4wNjAwMDAgICA0LjU2MDAwMCAoICA0LjU1Mjc4OSkKPgoKVWJ1bnR1IDguMTAgNjRiaXQK
    CiQgcnVieSAtLXZlcnNpb24gJiYgcnVieSBmZmlfYmVuY2gucmIKcnVieSAxLjguNyAoMjAwOC0w
    OC0xMSBwYXRjaGxldmVsIDcyKSBbeDg2XzY0LWxpbnV4XQoxLjIuMy4zCjEuMi4zLjMKMS4yLjMu
    MwogICAgICAgICB1c2VyICAgICBzeXN0ZW0gICAgICB0b3RhbCAgICAgICAgcmVhbApleHQgIDAu
    MzIwMDAwICAgMC4wNzAwMDAgICAwLjM5MDAwMCAoICAwLjM5Njc3NCkKZmZpICAwLjc3MDAwMCAg
    IDAuMTIwMDAwICAgMC44OTAwMDAgKCAgMC44OTUwOTMpCmRsICAgMi4wOTAwMDAgICAwLjI3MDAw
    MCAgIDIuMzYwMDAwICggIDIuMzY1MDI5KQoKCgoKLS0gClJhZG9zs2F3IEJ1s2F0CgpodHRwOi8v
    cmFkYXJlay5qb2dnZXIucGwgLSBt82ogYmxvZwo=
    Rados³aw Bu³at, Nov 1, 2008
    #11
  12. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Luc Heinrich wrote:
    > user system total real
    > ext 1.050000 0.320000 1.370000 ( 1.373800)
    > ffi 2.160000 0.660000 2.820000 ( 2.818966)
    > dl 3.500000 1.060000 4.560000 ( 4.552789)
    >
    > All this using MacPorts MRI 1.8.7-p72 under OS X 10.5.5. The observed
    > overhead is slightly over 2x for ffi, probably not a big deal unless ffi
    > calls are used in tight loops I guess.


    Seems like that pretty well seals the deal for ffi over dl, at the very
    least. I'm also glad to see FFI wasn't even that bad, especially
    considering it hasn't received any optimization.

    - Charlie
    Charles Oliver Nutter, Nov 2, 2008
    #12
  13. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Radosław Bułat wrote:
    >> This gives the following results:
    >>
    >> 1.2.3
    >> 1.2.3
    >> 1.2.3
    >> user system total real
    >> ext 1.050000 0.320000 1.370000 ( 1.373800)
    >> ffi 2.160000 0.660000 2.820000 ( 2.818966)
    >> dl 3.500000 1.060000 4.560000 ( 4.552789)
    >>

    >
    > Ubuntu 8.10 64bit
    >
    > $ ruby --version && ruby ffi_bench.rb
    > ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
    > 1.2.3.3
    > 1.2.3.3
    > 1.2.3.3
    > user system total real
    > ext 0.320000 0.070000 0.390000 ( 0.396774)
    > ffi 0.770000 0.120000 0.890000 ( 0.895093)
    > dl 2.090000 0.270000 2.360000 ( 2.365029)


    Seems like about the same ratio...probably can be improved too!

    - Charlie
    Charles Oliver Nutter, Nov 2, 2008
    #13
  14. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Radosław Bułat wrote:
    > Where can I file an issue? I have trouble to build in with ruby1.9 on 64bit.


    Hop on the ruby-ffi project mailing lists, and I guess you can file a
    bug in the tracker there too:

    http://kenai.com/projects/ruby-ffi

    - Charlie
    Charles Oliver Nutter, Nov 2, 2008
    #14
  15. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    On Sat, Nov 01, 2008 at 08:16:39AM +0900, Charles Oliver Nutter wrote:
    > The JRuby team is proud to announce the release of FFI for Ruby 1.8.6/7
    > and 1.9!
    >
    > FFI (gem install ffi) is a library for programmatically loading dynamic
    > libraries, binding functions within them, and calling those functions
    > from Ruby code. Here's a quick sample of binding and calling the getpid
    > C library function:


    Interesting. On what kind of architectures is the binding part working ? I'm
    using dyncall to do the actual interfacing work (http://www.dyncall.org/) in a
    DL-replacement library, but my problem is that dyncall does not like Linux-PPC.
    What are you using on your side ?

    Sylvain
    Sylvain Joyeux, Nov 3, 2008
    #15
  16. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    Sylvain Joyeux wrote:
    > On Sat, Nov 01, 2008 at 08:16:39AM +0900, Charles Oliver Nutter wrote:
    >> The JRuby team is proud to announce the release of FFI for Ruby 1.8.6/7
    >> and 1.9!
    >>
    >> FFI (gem install ffi) is a library for programmatically loading dynamic
    >> libraries, binding functions within them, and calling those functions
    >> from Ruby code. Here's a quick sample of binding and calling the getpid
    >> C library function:

    >
    > Interesting. On what kind of architectures is the binding part working ? I'm
    > using dyncall to do the actual interfacing work (http://www.dyncall.org/) in a
    > DL-replacement library, but my problem is that dyncall does not like Linux-PPC.
    > What are you using on your side ?


    Ruby FFI uses libffi, as does JNA which ships with JRuby. I'm not
    certain about libffi specifically. but JNA claims to support OSX (ppc,
    x86, x86_64), linux (x86, amd64), FreeBSD/OpenBSD (x86, amd64), Solaris
    (x86, amd64, sparc, sparcv9) and Windows (x86, amd64).

    I'm not sure if linux-ppc support is not provided because it's not
    supported or because nobody has a linux-ppc machine to build on. The
    latter has been the case for several entries on the list; I myself was
    the build monkey for Solaris/AMD64 and Linux/AMD64 for a short time,
    before which there was no shipped support.

    There's certainly one way to find out...gem install ffi. Report back
    here or on ruby-ffi mailing lists what you learn :)

    http://kenai.com/projects/ruby-ffi

    - Charlie
    Charles Oliver Nutter, Nov 3, 2008
    #16
  17. Charles Oliver Nutter

    Guest

    Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    On Mon, 3 Nov 2008 05:01:11 -0500, Charles Oliver Nutter wrote:

    >I'm not sure if linux-ppc support is not provided because it's not
    >supported or because nobody has a linux-ppc machine to build on. The
    >latter has been the case for several entries on the list; I myself was
    >the build monkey for Solaris/AMD64 and Linux/AMD64 for a short time,
    >before which there was no shipped support.
    >
    >There's certainly one way to find out...gem install ffi. Report back
    >here or on ruby-ffi mailing lists what you learn :)


    Ubuntu 8.10 provides libffi on ppc. The gem builds, but a simple ruby
    program hangs (rather uses 100% CPU).

    I can investigate further (e.g building my own libffi), but that may
    not happen immediately.

    -jh
    , Nov 3, 2008
    #17
  18. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    >> Interesting. On what kind of architectures is the binding part working ? I'm
    >> using dyncall to do the actual interfacing work (http://www.dyncall.org/) in a
    >> DL-replacement library, but my problem is that dyncall does not like Linux-PPC.
    >> What are you using on your side ?

    >
    > Ruby FFI uses libffi, as does JNA which ships with JRuby. I'm not
    > certain about libffi specifically. but JNA claims to support OSX (ppc,
    > x86, x86_64), linux (x86, amd64), FreeBSD/OpenBSD (x86, amd64), Solaris
    > (x86, amd64, sparc, sparcv9) and Windows (x86, amd64).

    I did not know about libffi ... I'll have to look if I should not replace
    dyncall by libffi then. Thanks for the info.

    > I'm not sure if linux-ppc support is not provided because it's not
    > supported or because nobody has a linux-ppc machine to build on.

    I guess it is a bit of both. Given that libffi is supported on Linux/PPC64, I
    guess having it on Linux/PPC should not be that much of a problem -- but still,
    I think the calling convention can be slightly different between the two
    architectures.

    Sylvain
    Sylvain Joyeux, Nov 3, 2008
    #18
  19. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    We (JRuby and Rubinius contributors) designed in the ability to
    specify calling convention for a given library bound through FFI.
    Largely this was needed to support Win32's stdcall, but if Linux/PPC
    has a different convention and libffi supports it, then FFi will too
    (though we may have to add it to the list of accepted conventions).

    I'll check on the syntax we're supporting and get back to ya.

    - Charlie (mobile)

    On Nov 3, 2008, at 12:56, Sylvain Joyeux <
    > wrote:


    >>> Interesting. On what kind of architectures is the binding part
    >>> working ? I'm
    >>> using dyncall to do the actual interfacing work (http://www.dyncall.org/
    >>> ) in a
    >>> DL-replacement library, but my problem is that dyncall does not
    >>> like Linux-PPC.
    >>> What are you using on your side ?

    >>
    >> Ruby FFI uses libffi, as does JNA which ships with JRuby. I'm not
    >> certain about libffi specifically. but JNA claims to support OSX
    >> (ppc,
    >> x86, x86_64), linux (x86, amd64), FreeBSD/OpenBSD (x86, amd64),
    >> Solaris
    >> (x86, amd64, sparc, sparcv9) and Windows (x86, amd64).

    > I did not know about libffi ... I'll have to look if I should not
    > replace
    > dyncall by libffi then. Thanks for the info.
    >
    >> I'm not sure if linux-ppc support is not provided because it's not
    >> supported or because nobody has a linux-ppc machine to build on.

    > I guess it is a bit of both. Given that libffi is supported on Linux/
    > PPC64, I
    > guess having it on Linux/PPC should not be that much of a problem --
    > but still,
    > I think the calling convention can be slightly different between the
    > two
    > architectures.
    >
    > Sylvain
    >
    >
    Charles Oliver Nutter, Nov 3, 2008
    #19
  20. Re: [ANN] FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7and 1.9

    wrote:
    > On Mon, 3 Nov 2008 05:01:11 -0500, Charles Oliver Nutter wrote:
    >
    >> I'm not sure if linux-ppc support is not provided because it's not
    >> supported or because nobody has a linux-ppc machine to build on. The
    >> latter has been the case for several entries on the list; I myself was
    >> the build monkey for Solaris/AMD64 and Linux/AMD64 for a short time,
    >> before which there was no shipped support.
    >>
    >> There's certainly one way to find out...gem install ffi. Report back
    >> here or on ruby-ffi mailing lists what you learn :)

    >
    > Ubuntu 8.10 provides libffi on ppc. The gem builds, but a simple ruby
    > program hangs (rather uses 100% CPU).
    >
    > I can investigate further (e.g building my own libffi), but that may
    > not happen immediately.


    Hey, please do. Sounds like it *should* work, at least. I sure don't
    have a PPC box to test on and I don't think anyone else involved in Ruby
    FFI does either.

    - Charlie
    Charles Oliver Nutter, Nov 3, 2008
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. H5N1
    Replies:
    0
    Views:
    428
  2. Wayne Meissner

    [ANN] FFI 0.2.0

    Wayne Meissner, Dec 3, 2008, in forum: Ruby
    Replies:
    19
    Views:
    216
    Paul Brannan
    Dec 5, 2008
  3. John Croisant
    Replies:
    0
    Views:
    95
    John Croisant
    Oct 25, 2009
  4. Aston

    Ruby::DL vs Ruby::FFI

    Aston, Mar 8, 2010, in forum: Ruby
    Replies:
    5
    Views:
    173
    Chuck Remes
    Mar 11, 2010
  5. Picky Mendoza
    Replies:
    3
    Views:
    90
    Karthikeyan A K
    May 24, 2010
Loading...

Share This Page