embed Ruby or Lua

Discussion in 'Ruby' started by Gaspard Bucher, Oct 20, 2007.

  1. I am writing a real-time data processing tool (data in ---> pattern
    recognition ---> midi out) called rubyk (site: http://rubyk.org trac:
    http://dev.rubyk.org). The first idea was to embed ruby (thus the name
    rubyk).

    I did some testing with a first "pure ruby" implementation of the
    signal sending mechanism and it works fine but is erratic (very hard
    to have stable tempo), probably due to ruby's GC messing in the way.

    So I wrote a C++ core and thought I would embed ruby. But then the
    garbage collector's "inconvenient pause" would happen again (with less
    objects though).

    Will ruby work fine for real-time midi ? Or am I better off using Lua
    ? I really like ruby and would prefer going this route, but I do not
    want to be limited to a "rubato" style of music.

    So the questions are:

    1. I need to loop at a minimum of 300Hz. Will ruby GC be fast enough ?
    2. Is there a way to disable GC without leaking memory ?
    3. Would Lua be a better fit ?
    4. Python uses reference counting so no "mark & sweep" pause. But I do
    not want to use Python, so please do not tell me Python would be
    great.

    Many thanks for your answers.

    Gaspard
     
    Gaspard Bucher, Oct 20, 2007
    #1
    1. Advertising

  2. Gaspard Bucher wrote:
    --snip--
    > 1. I need to loop at a minimum of 300Hz. Will ruby GC be fast enough ?
    > 2. Is there a way to disable GC without leaking memory ?
    > 3. Would Lua be a better fit ?
    > 4. Python uses reference counting so no "mark & sweep" pause. But I do
    > not want to use Python, so please do not tell me Python would be
    > great.


    Testing on old and slow computer would provide your answers.
    If garbage collection disrupts tempo, it can be disabled
    GC.disable
    Enable and start between midi songs
    GC.enable
    GC.start
    GC is not known to leak memory, AFAIK.


    Gerald
     
    Gerald Murray, Oct 21, 2007
    #2
    1. Advertising

  3. Gaspard Bucher wrote:
    > I am writing a real-time data processing tool (data in ---> pattern
    > recognition ---> midi out) called rubyk (site: http://rubyk.org trac:
    > http://dev.rubyk.org). The first idea was to embed ruby (thus the name
    > rubyk).
    >
    > I did some testing with a first "pure ruby" implementation of the
    > signal sending mechanism and it works fine but is erratic (very hard
    > to have stable tempo), probably due to ruby's GC messing in the way.
    >
    > So I wrote a C++ core and thought I would embed ruby. But then the
    > garbage collector's "inconvenient pause" would happen again (with less
    > objects though).
    >
    > Will ruby work fine for real-time midi ? Or am I better off using Lua
    > ? I really like ruby and would prefer going this route, but I do not
    > want to be limited to a "rubato" style of music.
    >
    > So the questions are:
    >
    > 1. I need to loop at a minimum of 300Hz. Will ruby GC be fast enough ?
    > 2. Is there a way to disable GC without leaking memory ?
    > 3. Would Lua be a better fit ?
    > 4. Python uses reference counting so no "mark & sweep" pause. But I do
    > not want to use Python, so please do not tell me Python would be
    > great.
    >
    > Many thanks for your answers.
    >
    > Gaspard
    >
    >


    I'm not quite sure *exactly* what it is you're trying to do here, but my
    rough guess is that it's real-time multimedia. Judging by some of the
    other things on the site -- support vector machines, BLAS, etc. -- I
    think you probably know enough to code the whole thing in C or C++, and
    if you can do so, that's what I'd recommend.

    In fact, I'd recommend getting one of the Linux audio distributions,
    like JAD or Studio 64, and getting the real "hard real-time" (as hard
    real-time as Linux can get, anyhow) libraries up and running. I've
    mostly looked at Studio 64 because I have a 64-bit machine, but JAD is
    also good. Both are Debian Linux based and have special kernel patches
    and other software for low-latency audio processing.

    Lua is probably a good bit faster than Ruby, but I don't know how much
    of the application has to be in the scripting language and how much in
    close-to-the-metal C/C++. I really don't think you want a
    garbage-collected language for this, although I'm sure it's possible to
    use one if the heavy lifting is done in a low-level library.

    Speaking of low-level libraries, since you're looking at BLAS, LAPACK,
    support vector machines, etc., you might want to install Atlas
    (Automatically Tuned Linear Algebra Subroutines). They just released
    3.8.0 and it's hand tuned to most modern chips. On my dual-core Athlon64
    X2 2200+ I've gotten close to 10 GFLOPS on 32-bit operations!
     
    M. Edward (Ed) Borasky, Oct 21, 2007
    #3
  4. --=-CGHQ1ss7Na5OBFn4Ljkq
    Content-Type: multipart/alternative; boundary="=-HlKKbAJzBQ7yugxItBpb"


    --=-HlKKbAJzBQ7yugxItBpb
    Content-Type: text/plain
    Content-Transfer-Encoding: quoted-printable

    On Sun, 2007-21-10 at 15:19 +0900, M. Edward (Ed) Borasky wrote:

    > I really don't think you want a=20
    > garbage-collected language for this, although I'm sure it's possible to=20
    > use one if the heavy lifting is done in a low-level library.



    There are, in fact, hard realtime garbage collection algorithms
    available. For some reason language designers just don't seem to want
    to implement them in their languages in favour of garbage collection
    algorithms that are easier to write.

    What I'd like to see someday is a language whose runtime explicitly
    supports garbage-collection plug-ins so that you can use the appropriate
    collection algorithm for the appropriate situation.

    Going to the OP's question? Embed the Lua. Ruby is a great language
    and great for embedding in applications, even, except in the specific
    case of what you're talking about right now: anything with hard realtime
    (and even soft realtime!) requirements. Lua is a much faster language
    and has garbage collection that is far more easily tuned.

    --=20
    Michael T. Richter <> (GoogleTalk:
    )
    I have to wonder why people think that when they can't manage local
    personnel within easy strangling and shooting distance, then they can
    manage personnel thousands of miles away that have different languages,
    cultures, and business rules. (Joe Celko)

    --=-HlKKbAJzBQ7yugxItBpb
    Content-Type: text/html; charset=utf-8
    Content-Transfer-Encoding: quoted-printable

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
    <HTML>
    <HEAD>
    <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; CHARSET=3DUTF-8">
    <META NAME=3D"GENERATOR" CONTENT=3D"GtkHTML/3.12.1">
    </HEAD>
    <BODY>
    On Sun, 2007-21-10 at 15:19 +0900, M. Edward (Ed) Borasky wrote:
    <BLOCKQUOTE TYPE=3DCITE>
    <PRE>
    <FONT COLOR=3D"#000000">I really don't think you want a </FONT>
    <FONT COLOR=3D"#000000">garbage-collected language for this, although I'm s=
    ure it's possible to </FONT>
    <FONT COLOR=3D"#000000">use one if the heavy lifting is done in a low-level=
    library.</FONT>
    </PRE>
    </BLOCKQUOTE>
    <BR>
    There are, in fact, hard realtime garbage collection algorithms available.&=
    nbsp; For some reason language designers just don't seem to want to impleme=
    nt them in their languages in favour of garbage collection algorithms that =
    are easier to write.<BR>
    <BR>
    What I'd like to see someday is a language whose runtime explicitly support=
    s garbage-collection plug-ins so that you can use the appropriate collectio=
    n algorithm for the appropriate situation.<BR>
    <BR>
    Going to the OP's question?&nbsp; Embed the Lua.&nbsp; Ruby is a great lang=
    uage and great for embedding in applications, even, <B>except</B> in the sp=
    ecific case of what you're talking about right now: anything with hard real=
    time (and even soft realtime!) requirements.&nbsp; Lua is a much faster lan=
    guage and has garbage collection that is far more easily tuned.<BR>
    <BR>
    <TABLE CELLSPACING=3D"0" CELLPADDING=3D"0" WIDTH=3D"100%">
    <TR>
    <TD>
    -- <BR>
    <B>Michael T. Richter</B> &lt;<A HREF=3D"mailto:">ttmri=
    </A>&gt; (<B>GoogleTalk:</B> )<BR>
    <I>I have to wonder why people think that when they can't manage local pers=
    onnel within easy strangling and shooting distance, then they can manage pe=
    rsonnel thousands of miles away that have different languages, cultures, an=
    d business rules. (Joe Celko)</I>
    </TD>
    </TR>
    </TABLE>
    </BODY>
    </HTML>

    --=-HlKKbAJzBQ7yugxItBpb--

    --=-CGHQ1ss7Na5OBFn4Ljkq
    Content-Type: application/pgp-signature; name=signature.asc
    Content-Description: This is a digitally signed message part

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.3 (GNU/Linux)

    iD8DBQBHGvnwLqyWkKVQ54QRAkGNAJ9pnqpygTPkTHQKv9vUKxTPeXa2SQCfQ1YT
    /lkmOLIXAO1NHlgSWzPVVDc=
    =3EGv
    -----END PGP SIGNATURE-----

    --=-CGHQ1ss7Na5OBFn4Ljkq--
     
    Michael T. Richter, Oct 21, 2007
    #4
  5. >
    > I'm not quite sure *exactly* what it is you're trying to do here, but my
    > rough guess is that it's real-time multimedia. Judging by some of the
    > other things on the site -- support vector machines, BLAS, etc. -- I
    > think you probably know enough to code the whole thing in C or C++, and
    > if you can do so, that's what I'd recommend.


    All the core parts are coded in C. I need a scripting language for the
    post-processing (simple low-pass filter, decision tree, instrument
    change depending on song position, etc).

    >
    > In fact, I'd recommend getting one of the Linux audio distributions,
    > like JAD or Studio 64, and getting the real "hard real-time" (as hard
    > real-time as Linux can get, anyhow) libraries up and running. I've
    > mostly looked at Studio 64 because I have a 64-bit machine, but JAD is
    > also good. Both are Debian Linux based and have special kernel patches
    > and other software for low-latency audio processing.


    This would be great, but the machine has to include the
    sound-generating application (I bought mac-minis). If the latency is
    too bad, I might use Studio64 and send all the midi to a central
    "audio-generation" machine.


    >
    > Lua is probably a good bit faster than Ruby, but I don't know how much
    > of the application has to be in the scripting language and how much in
    > close-to-the-metal C/C++. I really don't think you want a
    > garbage-collected language for this, although I'm sure it's possible to
    > use one if the heavy lifting is done in a low-level library.

    All long-time processing (SVM, PCA) will be done I C using the
    Accelerate framework (fast BLAS and LAPACK tuned for Inter
    processors).

    > Speaking of low-level libraries, since you're looking at BLAS, LAPACK,
    > support vector machines, etc., you might want to install Atlas
    > (Automatically Tuned Linear Algebra Subroutines). They just released
    > 3.8.0 and it's hand tuned to most modern chips. On my dual-core Athlon64
    > X2 2200+ I've gotten close to 10 GFLOPS on 32-bit operations!

    I do not know if there is a difference between Accelerate framework
    and Atlas. I might install Atlas if I can gain some speed here.

    Thanks for the reply, it seems like Lua will be better for the
    post-processing I need to do.

    Gaspard
     
    Gaspard Bucher, Oct 21, 2007
    #5
  6. Gaspard Bucher wrote:
    >> I'm not quite sure *exactly* what it is you're trying to do here, but my
    >> rough guess is that it's real-time multimedia. Judging by some of the
    >> other things on the site -- support vector machines, BLAS, etc. -- I
    >> think you probably know enough to code the whole thing in C or C++, and
    >> if you can do so, that's what I'd recommend.

    >
    > All the core parts are coded in C. I need a scripting language for the
    > post-processing (simple low-pass filter, decision tree, instrument
    > change depending on song position, etc).
    >
    >> In fact, I'd recommend getting one of the Linux audio distributions,
    >> like JAD or Studio 64, and getting the real "hard real-time" (as hard
    >> real-time as Linux can get, anyhow) libraries up and running. I've
    >> mostly looked at Studio 64 because I have a 64-bit machine, but JAD is
    >> also good. Both are Debian Linux based and have special kernel patches
    >> and other software for low-latency audio processing.

    >
    > This would be great, but the machine has to include the
    > sound-generating application (I bought mac-minis). If the latency is
    > too bad, I might use Studio64 and send all the midi to a central
    > "audio-generation" machine.
    >
    >
    >> Lua is probably a good bit faster than Ruby, but I don't know how much
    >> of the application has to be in the scripting language and how much in
    >> close-to-the-metal C/C++. I really don't think you want a
    >> garbage-collected language for this, although I'm sure it's possible to
    >> use one if the heavy lifting is done in a low-level library.

    > All long-time processing (SVM, PCA) will be done I C using the
    > Accelerate framework (fast BLAS and LAPACK tuned for Inter
    > processors).
    >
    >> Speaking of low-level libraries, since you're looking at BLAS, LAPACK,
    >> support vector machines, etc., you might want to install Atlas
    >> (Automatically Tuned Linear Algebra Subroutines). They just released
    >> 3.8.0 and it's hand tuned to most modern chips. On my dual-core Athlon64
    >> X2 2200+ I've gotten close to 10 GFLOPS on 32-bit operations!

    > I do not know if there is a difference between Accelerate framework
    > and Atlas. I might install Atlas if I can gain some speed here.
    >
    > Thanks for the reply, it seems like Lua will be better for the
    > post-processing I need to do.
    >
    > Gaspard
    >
    >


    Ah ... Intel Mac Minis. I don't know Accelerate, but I know Clint Whaley
    and the rest of the Atlas people have wrung every possible gigaflop out
    of the recent Intel Core2 processors, so unless Accelerate has managed
    to find a cycle or two the Atlas team didn't spot, Atlas is probably the
    way to go.

    What are you using to generate the audio? The built-in synthesizer in
    MacOS? I have absolutely no idea how the audio on a Mac works -- as
    you've probably guessed I'm a hard-core Linux geek.
     
    M. Edward (Ed) Borasky, Oct 21, 2007
    #6
  7. Michael T. Richter wrote:
    > On Sun, 2007-21-10 at 15:19 +0900, M. Edward (Ed) Borasky wrote:
    >
    >> I really don't think you want a
    >> garbage-collected language for this, although I'm sure it's possible to
    >> use one if the heavy lifting is done in a low-level library.

    >
    >
    > There are, in fact, hard realtime garbage collection algorithms
    > available. For some reason language designers just don't seem to want
    > to implement them in their languages in favour of garbage collection
    > algorithms that are easier to write.
    >
    > What I'd like to see someday is a language whose runtime explicitly
    > supports garbage-collection plug-ins so that you can use the appropriate
    > collection algorithm for the appropriate situation.
    >
    > Going to the OP's question? Embed the Lua. Ruby is a great language
    > and great for embedding in applications, even, except in the specific
    > case of what you're talking about right now: anything with hard realtime
    > (and even soft realtime!) requirements. Lua is a much faster language
    > and has garbage collection that is far more easily tuned.



    IIRC Lua was designed from the ground up to be embeddable in C code and
    to make interfacing with C easy. Of course, it's not Forth or Scheme, so
    I probably wouldn't use it ... :)
     
    M. Edward (Ed) Borasky, Oct 21, 2007
    #7
  8. >
    > Testing on old and slow computer would provide your answers.
    > If garbage collection disrupts tempo, it can be disabled
    > GC.disable
    > Enable and start between midi songs
    > GC.enable
    > GC.start
    > GC is not known to leak memory, AFAIK.

    I wrote "leak" but I meant "grows", how much does the garbage grow if
    it is not collected ? (I will test this to see).
    >
    >
    > Gerald

    I will try this before moving to Lua. If it's fast enough and the
    memory does not grow too much between garbage collections: Ruby. Else:
    Lua.

    Gaspard
     
    Gaspard Bucher, Oct 21, 2007
    #8
  9. Gaspard Bucher

    John Joyce Guest

    If you're doing midi on a Mac, you should look into some of the Core
    Audio portion of the Cocoa framework.
    There are a lot of things done for you. If you build the app with the
    Cocoa framework, you can get scriptability with very little effort.
    If you're already writing in C, the Objective-C stuff should be easy.
    The Cocoa framework is pretty vast, and the audio stuff is too.
     
    John Joyce, Oct 21, 2007
    #9
  10. 2007/10/21, John Joyce <>:
    > If you're doing midi on a Mac, you should look into some of the Core
    > Audio portion of the Cocoa framework.

    I use the excellent RtMidi (http://www.music.mcgill.ca/~gary/rtmidi/)
    library so the code is portable. In fact it is just some headers for
    the vendor specific libraries on Windows, Linux and Mac os x. Audio is
    another story into which I might dive someday, but it's not a
    priority.

    > There are a lot of things done for you. If you build the app with the
    > Cocoa framework, you can get scriptability with very little effort.


    > If you're already writing in C, the Objective-C stuff should be easy.
    > The Cocoa framework is pretty vast, and the audio stuff is too.

    I have written some applications in Cocoa and it was nice. Now I
    prefer to use open source libraries like agar (http://libagar.org/)
    and sdl (www.libsdl.org) instead of the proprietary Cocoa, even if
    this library is really good. If I get tired of mac OS X and want to
    move to Linux (for speed or whatever good reason), I want to be free
    to do so. Rubyk is intended to be a long running project that I will
    be using in many future works...
     
    Gaspard Bucher, Oct 21, 2007
    #10
  11. Gaspard, just a couple of comments.
    I'm working on a similar soft-realtime application (autonomous drive of
    a vehicle), and I implemented the high level logic as a Ruby
    statemachine. I found that I can get a satisfying level of realtime
    (including UDP stack access) by keeping a fixed pace loop like that:

    while running do
    slp_thr = Thread.start do sleep timestep end
    #do the loop stuff here
    slp_thr.join
    end

    It works surprisingly fine until 300-350 Hz.
    Since you're running on Macs, you could also use the scheduling control
    functions of the Mach kernel. For example, something like that can be
    helpful:

    inline do |builder|
    builder.include "<mach/mach_init.h>"
    builder.include "<mach/thread_policy.h>"
    builder.include "<sched.h>"
    builder.c '
    int set_realtime(double period, double computation, double
    constraint) {
    uint32_t HZ = 2000000000;
    struct thread_time_constraint_policy ttcpolicy;
    int ret;
    ttcpolicy.period=HZ*period; // HZ/160
    ttcpolicy.computation=HZ*computation; // HZ/3300;
    ttcpolicy.constraint=HZ*constraint; // HZ/2200;
    ttcpolicy.preemptible=1;

    if ((ret=thread_policy_set(mach_thread_self(),
    THREAD_TIME_CONSTRAINT_POLICY,
    (thread_policy_t)&ttcpolicy,
    THREAD_TIME_CONSTRAINT_POLICY_COUNT)) != KERN_SUCCESS) {
    fprintf(stderr, "set_realtime() failed.\n");
    return 0;
    }
    return 1;
    }

    '
    end

    Cheers,
    Paolo
    --
    Posted via http://www.ruby-forum.com/.
     
    Paolo Bosetti, Apr 9, 2008
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Carsten Eckelmann

    Game Programming with Python, Lua and Ruby

    Carsten Eckelmann, Jan 11, 2004, in forum: Ruby
    Replies:
    3
    Views:
    183
    Shashank Date
    Jan 13, 2004
  2. Shea Martin

    embedding: ruby vs. lua

    Shea Martin, Jan 18, 2007, in forum: Ruby
    Replies:
    10
    Views:
    264
    Shea Martin
    Feb 6, 2007
  3. Replies:
    6
    Views:
    286
    M. Edward (Ed) Borasky
    May 18, 2007
  4. Dolazy
    Replies:
    1
    Views:
    139
    Luis Lavena
    Nov 15, 2008
  5. Catalin Tilimpea

    Parsing lua with Ruby

    Catalin Tilimpea, Jul 3, 2009, in forum: Ruby
    Replies:
    7
    Views:
    192
    Catalin Tilimpea
    Jul 6, 2009
Loading...

Share This Page