neuroimage software - scientific computing and visualization

Discussion in 'Ruby' started by Darren L. Weber, Feb 1, 2006.

  1. Hi,

    I've been programming with perl, a bit of python and a lot of matlab.
    I've become very curious about ruby and considering whether or not to
    take it on as my main development environment in the near future. I
    could use some good advice before I dive into the deep end.

    I'm looking at Ruby for scripting/programming of scientific computing
    for neuroimaging. This includes time-series plots and analysis, image
    volume segmentation and rendering, surface modeling, visualization and
    morphing, and animating scalar and vector quanities on surfaces. Also
    linear algrebra methods and spatial transformations, almost any
    scientific computation you can imagine, especially large scale
    statistical analysis (including permutations). Did I leave anything
    out? What about threading and distributed cluster systems? I could
    really use some tips on whether Ruby is a good choice for this work and
    where to find projects that already fit into this picture.

    So, I'm wonder if ruby has wrapped things like: VTK, ITK, The R-project
    for stats, OpenGL, wxWindows, scientific libraries (eg, GNU GSL,
    Atlas/Lapack), graphics libraries, etc. Any good advice on getting
    started with these tools would be great.

    There is some momentum in this area using python already. Why should I
    use Ruby instead of Python? One thing that I've noticed is that python
    distribution and installation methods are clumsy. If I want some
    python package for neuroimaging, I have to find and install many
    additional dependencies before I even get started. As a Debian user,
    the apt system helps, but it's still not quite what I want. Ruby has
    the gem system that appears to do the job right.

    Thanks, Darren
     
    Darren L. Weber, Feb 1, 2006
    #1
    1. Advertising

  2. Darren L. Weber

    anne001 Guest

    I think you should stick with Matlab, they have developped programs for
    years for this kind of work. The programs I have seen for MRI imaging
    are in matlab.

    In ruby, there are things missing: ex when you do a sort, you don't get
    the index of a sort as in matlab, only the sorted array. So you would
    have to write you own - Matlab like - methods.

    The other thing I miss in ruby, the ability to do matrix operations. If
    you do the mean, it is the mean of a vector, while in matlab, you can
    do a mean on any dimension array, indicating which dimension

    I don't know how much stats ruby can do, certainly the matlab stat
    package is not complete, but I can't imagine that a programming
    language would be that developed in this area.

    Another reason to stay with matlab is readability. Matlab is a higher
    level language. It sounds like you are a programmer selling a product,
    so it might be different, but I work at a university, and students
    inherit the matlab programs from one another, and then typically they
    have to figure out what the program does to get the data in correctly
    for their data, get the figures they want for their conditions... the
    fact that it is written in a higher level language seems to help.

    Ruby does have some matrix computation ability, so you should look at
    that, and see if that suits your needs.
     
    anne001, Feb 1, 2006
    #2
    1. Advertising

  3. Darren L. Weber

    Guest

    Hi Darren

    > I'm looking at Ruby for scripting/programming of scientific computing
    > for neuroimaging.


    I used Matlab for my PhD research (medical image modelling and
    analysis), and I think it was excellent for my purposes. Below is a
    list of why I liked using it, which I'll compare to my (limited)
    experience with Ruby afterwards:

    1. Matlab language syntax maps closely to the underlying concepts (i.e.
    vector and matrices).
    2. Very simple, but powerful, plotting/visualisation abilities.
    3. (Generally) efficient implementations for common matrix operations
    (multiplication, decompositions etc.). These are very well-tested.
    4. A REPL (i.e. you can experiment on a 'command line').
    5. Cross-platform (Linux, OS X, Solaris, Windows).
    6. No compiling (recent Matlabs do just-in-time compilation
    automatically).
    7. Excellent support for profiling (easily find and quantify those
    computational hot-spots).
    8. It's easy to write C (or FORTRAN) extensions when you really need
    every last ounce of speed or control over memory.
    9. Lots of other people use it, so there's an 'ecosystem' (the users
    are also working on your problem or closely-related ones).

    Comparing this to Ruby:

    1. Ruby is high-level, but it's a general purpose language; it's not
    specialised for vectors and matrices. You'd have to get a library or
    write this yourself. The Ruby syntax is 'sort-of' similar to the Matlab
    syntax, though. Well, it feels Matlab-y to me!
    2. You'd need to write or find plotting/visualisation library for Ruby.
    I've not looked, but I'd hope this is reasonably trivial.
    3. You'd need to find or write a library to let you do efficient matrix
    computations. I know there is stuff for Ruby out there, but it won't be
    as well-tested as Matlab's and it's not a first class citizen of the
    environment. Writing your own is much harder than it seems. You really
    need a library that allows you to call LAPACK routines from Ruby.
    4. Ruby has the irb REPL. This is fine.
    5. Ruby seems to be pretty cross-platform, but you'd need to check the
    same is true for the additional libraries you'd need. You might not
    care about WIndows, but what about your colleagues? You often need to
    share code with them.
    6. No compiling, but also no JIT as far as I'm aware. Not too big an
    issue if you've got a good matrix library.
    7. I've not tried profiling in Ruby, but Matlab's profiler is very
    good. If you can't profile your code and you're doing numerics, you're
    in trouble.
    8. I understand you can write Ruby extensions in C but I've not done
    it.
    9. The ecosystem for your problem domain is going to be much smaller in
    Ruby than in Matlab. Ruby's best ecosystem domain seems to be webapps
    (i.e. Rails).

    Now, the thing I haven't mentioned is price. Matlab is expensive
    (thousands of dollars for all the stuff you need). They do student
    discounts, but last time I looked you'd need to pay hundreds of dollars
    rather than thousands, so it's not a great discount. (Mathematica's
    student terms were much better, last time I looked.)

    Ruby is free. In my opinion, if you can afford Matlab (or someone else
    is affording it for you), you should have a very good reason before you
    ditch Matlab. However, if you can't afford Matlab, or have ideological
    reasons why you don't want to use it (it's not open source, for
    example), you should probably use and improve Octave, the open source
    equivalent to Matlab. However, if you need to roll your own soluation,
    I think Ruby would make a very good foundation for putting together
    your own environment for numerical computing.

    I think the real question is whether you want to do the 'fun' thing and
    start (almost) from scratch (i.e. base your environment on Ruby) or
    stand on the shoulders of giants (i.e. use Matlab or Octave). If you've
    got real problems to solve now, you should probably resist the
    temptation to do the fun thing.

    Chris
     
    , Feb 1, 2006
    #3
  4. Thanks very much for these thoughtful comments.

    Matlab is very nice and I enjoy working with it. The help system is
    very informative and all aspects of scientific computing work well. No
    wonder so many university projects work with matlab. But, I do not
    enjoy the inconvenience of running a licence manager and having my work
    come to a halt when my license expires every year. (Yes, some IT
    departments do not automatically update the licenses on every system,
    including my laptop). I also fear that sometime I may not have a
    matlab license and my work will be locked, despite my best attempts to
    create open-source tools. Most annoying is that I do not know how to
    or cannot create multi-threaded functions and I can't call matlab
    functions in shell scripts (matlab doesn't have a shebang for
    scripting).

    I can see some momentum in python for scientific computing and I have
    begun to play with python, but I find the installation of modules can
    be either easy or painful (especially dependencies). My intuitions are
    telling me that ruby could be a better option than python in the long
    term, but I want to clarify my understanding before I get carried away
    with enthusiasm.

    For a start, I've noticed these useful sites:

    http://narray.rubyforge.org/index.html.en

    http://rb-gsl.rubyforge.org/

    http://sciruby.codeforpeople.com/sr.cgi/FrontPage

    I don't see anyone using these for neuroimaging, in particular.

    Best, Darren
     
    Darren L. Weber, Feb 1, 2006
    #4
  5. Darren L. Weber

    Phil Tomson Guest

    In article <>,
    Darren L. Weber <> wrote:
    >
    >Hi,
    >
    >I've been programming with perl, a bit of python and a lot of matlab.
    >I've become very curious about ruby and considering whether or not to
    >take it on as my main development environment in the near future. I
    >could use some good advice before I dive into the deep end.
    >
    >I'm looking at Ruby for scripting/programming of scientific computing
    >for neuroimaging. This includes time-series plots and analysis, image
    >volume segmentation and rendering, surface modeling, visualization and
    >morphing, and animating scalar and vector quanities on surfaces. Also
    >linear algrebra methods and spatial transformations, almost any
    >scientific computation you can imagine, especially large scale
    >statistical analysis (including permutations). Did I leave anything
    >out? What about threading and distributed cluster systems? I could
    >really use some tips on whether Ruby is a good choice for this work and
    >where to find projects that already fit into this picture.
    >
    >So, I'm wonder if ruby has wrapped things like: VTK, ITK, The R-project
    >for stats, OpenGL, wxWindows, scientific libraries (eg, GNU GSL,
    >Atlas/Lapack), graphics libraries, etc. Any good advice on getting
    >started with these tools would be great.


    wxWindows, GSL, OpenGL are there, I believe. Not sure about the rest.

    It would be nice to get a nice Ruby->R bridge and it could probably be
    very Rubyish (maybe a DSL built on Ruby).

    >
    >There is some momentum in this area using python already. Why should I
    >use Ruby instead of Python? One thing that I've noticed is that python
    >distribution and installation methods are clumsy.


    Well, that could be your reason. The other thing to consider is which language
    you prefer. While many people will claim they are equivilent, there are
    differences both feature-wise and less tangible differences. Check out Bruce
    Eckel's recent post at Artima, for example.

    >If I want some
    >python package for neuroimaging, I have to find and install many
    >additional dependencies before I even get started. As a Debian user,
    >the apt system helps, but it's still not quite what I want. Ruby has
    >the gem system that appears to do the job right.


    You should definitely check ou the SciRuby site:
    http://sciruby.codeforpeople.com/

    You should also check out narray for high speed vector math in Ruby.

    Phil
     
    Phil Tomson, Feb 1, 2006
    #5
  6. Darren L. Weber

    Phil Tomson Guest

    In article <>,
    anne001 <> wrote:
    >I think you should stick with Matlab, they have developped programs for
    >years for this kind of work. The programs I have seen for MRI imaging
    >are in matlab.
    >
    >In ruby, there are things missing: ex when you do a sort, you don't get
    >the index of a sort as in matlab, only the sorted array. So you would
    >have to write you own - Matlab like - methods.


    I'm not sure I understand what you're saying here: are you saying you want the
    indices of the array sorted, but not the array itself?

    So are you saying that if you have an array like:
    a = ['d', 'b', 'a', 'c', 'e']
    You would like to get something like:
    a.sort_index #=> [2, 1, 3, 0, 4]

    If that's the case, then I'm sure it's doable. No built-in method that I'm
    aware of, but you could easily create one.

    >
    >The other thing I miss in ruby, the ability to do matrix operations. If
    >you do the mean, it is the mean of a vector, while in matlab, you can
    >do a mean on any dimension array, indicating which dimension


    check out narray.

    >
    >I don't know how much stats ruby can do, certainly the matlab stat
    >package is not complete, but I can't imagine that a programming
    >language would be that developed in this area.
    >
    >Another reason to stay with matlab is readability. Matlab is a higher
    >level language. It sounds like you are a programmer selling a product,
    >so it might be different, but I work at a university, and students
    >inherit the matlab programs from one another, and then typically they
    >have to figure out what the program does to get the data in correctly
    >for their data, get the figures they want for their conditions... the
    >fact that it is written in a higher level language seems to help.


    YMMV. I don't find Matlab very readable. I much prefer Ruby's syntax.

    It's been mentioned before, but it would be cool to create a kind of MatLab
    'replacement' using Ruby. One could imagine that you could take advantage of
    Ruby's metaprogramming and DSL abilities to create something that would be
    quite nice to use. (realistically, we couldn't completely replicate all matlab
    functionality, but perhaps we could focus on certain areas).

    MatLab may be 'higher level' but that's only in the sense of representing
    mathematical operations. In another sense Ruby is higher-level because it
    would allow you to create a DSL for representing mathematical operations
    (similar to MatLab).

    Phil
     
    Phil Tomson, Feb 1, 2006
    #6
  7. Darren L. Weber

    Phil Tomson Guest

    In article <>,
    <> wrote:
    >Hi Darren
    >
    >> I'm looking at Ruby for scripting/programming of scientific computing
    >> for neuroimaging.

    >
    >I used Matlab for my PhD research (medical image modelling and
    >analysis), and I think it was excellent for my purposes. Below is a
    >list of why I liked using it, which I'll compare to my (limited)
    >experience with Ruby afterwards:
    >
    >1. Matlab language syntax maps closely to the underlying concepts (i.e.
    >vector and matrices).
    >2. Very simple, but powerful, plotting/visualisation abilities.
    >3. (Generally) efficient implementations for common matrix operations
    >(multiplication, decompositions etc.). These are very well-tested.
    >4. A REPL (i.e. you can experiment on a 'command line').
    >5. Cross-platform (Linux, OS X, Solaris, Windows).
    >6. No compiling (recent Matlabs do just-in-time compilation
    >automatically).
    >7. Excellent support for profiling (easily find and quantify those
    >computational hot-spots).
    >8. It's easy to write C (or FORTRAN) extensions when you really need
    >every last ounce of speed or control over memory.
    >9. Lots of other people use it, so there's an 'ecosystem' (the users
    >are also working on your problem or closely-related ones).
    >
    >Comparing this to Ruby:
    >
    >1. Ruby is high-level, but it's a general purpose language; it's not
    >specialised for vectors and matrices. You'd have to get a library or
    >write this yourself. The Ruby syntax is 'sort-of' similar to the Matlab
    >syntax, though. Well, it feels Matlab-y to me!
    >2. You'd need to write or find plotting/visualisation library for Ruby.
    >I've not looked, but I'd hope this is reasonably trivial.


    There are a few things out there. There's a gnuplot interface, for example.

    >3. You'd need to find or write a library to let you do efficient matrix
    >computations. I know there is stuff for Ruby out there, but it won't be
    >as well-tested as Matlab's and it's not a first class citizen of the
    >environment. Writing your own is much harder than it seems. You really
    >need a library that allows you to call LAPACK routines from Ruby.


    Seems to me that narray has most of what you need here.

    >4. Ruby has the irb REPL. This is fine.
    >5. Ruby seems to be pretty cross-platform, but you'd need to check the
    >same is true for the additional libraries you'd need. You might not
    >care about WIndows, but what about your colleagues? You often need to
    >share code with them.


    No, they've all bought iBooks or PowerBooks by now ;-)

    >6. No compiling, but also no JIT as far as I'm aware. Not too big an
    >issue if you've got a good matrix library.


    narray is implemented in C.

    >7. I've not tried profiling in Ruby, but Matlab's profiler is very
    >good. If you can't profile your code and you're doing numerics, you're
    >in trouble.


    Ruby has reasonably good profiling ability.

    >8. I understand you can write Ruby extensions in C but I've not done
    >it.


    It's quite easy. It's even easier if you use Ruby::Inline.

    Here's an tantalizing idea that I'll throw out: C is one thing and, yes, it's
    fast.... but what if you could actually map some algorithm or math operation to
    _hardware_ and thus get a large speed advantage even over C? I'm working on
    something like this... details forthcoming. The idea is very similar to
    Ruby::Inline. It will allow seamless communication between your Ruby program
    and an FPGA board acting as hardware accelerator (these boards are available
    for less than $200 now).

    >9. The ecosystem for your problem domain is going to be much smaller in
    >Ruby than in Matlab. Ruby's best ecosystem domain seems to be webapps
    >(i.e. Rails).


    Maybe that's the most successful domain related to Ruby up to this point,
    however there are people using Ruby for scientific apps - see the SciRuby site:
    http://sciruby.codeforpeople.com/

    Ruby isn't just for webapps. In fact, until the last 18 months or so, people
    were wondering if Ruby would be a player in the webapp space - how quickly
    things change, eh?

    >
    >Now, the thing I haven't mentioned is price. Matlab is expensive
    >(thousands of dollars for all the stuff you need). They do student
    >discounts, but last time I looked you'd need to pay hundreds of dollars
    >rather than thousands, so it's not a great discount. (Mathematica's
    >student terms were much better, last time I looked.)
    >
    >Ruby is free. In my opinion, if you can afford Matlab (or someone else
    >is affording it for you), you should have a very good reason before you
    >ditch Matlab. However, if you can't afford Matlab, or have ideological
    >reasons why you don't want to use it (it's not open source, for
    >example), you should probably use and improve Octave, the open source
    >equivalent to Matlab. However, if you need to roll your own soluation,
    >I think Ruby would make a very good foundation for putting together
    >your own environment for numerical computing.


    Yes, and part of the philosophical argument as I see it is that tools like
    MatLab are becoming 'required' for doing research (many people think so
    anyway), kind of like using LaTeX for writing papers. However, LaTeX is free.
    It would be great to have a completely free tool for research since
    computational methods are becoming central to so much research.

    >
    >I think the real question is whether you want to do the 'fun' thing and
    >start (almost) from scratch (i.e. base your environment on Ruby) or
    >stand on the shoulders of giants (i.e. use Matlab or Octave). If you've
    >got real problems to solve now, you should probably resist the
    >temptation to do the fun thing.


    However, I think you'd be surprised at how much is already available in Ruby.
    Check it out and see if what you need might already be there (or be almost
    there). Also, get on SciRuby and post requests.

    Phil
     
    Phil Tomson, Feb 1, 2006
    #7
  8. Darren L. Weber

    Guest

    Hi Darren

    > But, I do not
    > enjoy the inconvenience of running a licence manager and having my work
    > come to a halt when my license expires every year. (Yes, some IT
    > departments do not automatically update the licenses on every system,
    > including my laptop). I also fear that sometime I may not have a
    > matlab license and my work will be locked, despite my best attempts to
    > create open-source tools. Most annoying is that I do not know how to
    > or cannot create multi-threaded functions and I can't call matlab
    > functions in shell scripts (matlab doesn't have a shebang for
    > scripting).


    All valid points, except that you can call matlab functions on the UNIX
    (and Windows?) command line (though probably not with a shebang, but I
    don't see that as essential).

    > For a start, I've noticed these useful sites:
    > http://narray.rubyforge.org/index.html.en
    > http://rb-gsl.rubyforge.org/
    > http://sciruby.codeforpeople.com/sr.cgi/FrontPage


    Thanks for the pointers.

    Chris
     
    , Feb 2, 2006
    #8
  9. Darren L. Weber

    Guest

    Phil

    > narray is implemented in C.


    I took a quick look at its API and it only seemed to offer the LU
    matrix decomposition; which seems a bit minimal.

    > It would be great to have a completely free tool for research since
    > computational methods are becoming central to so much research.


    Agreed.

    Chris
     
    , Feb 2, 2006
    #9
  10. Darren L. Weber

    Guest

    On Fri, 3 Feb 2006 wrote:

    > Hi Darren
    >
    >> But, I do not
    >> enjoy the inconvenience of running a licence manager and having my work
    >> come to a halt when my license expires every year. (Yes, some IT
    >> departments do not automatically update the licenses on every system,
    >> including my laptop). I also fear that sometime I may not have a
    >> matlab license and my work will be locked, despite my best attempts to
    >> create open-source tools. Most annoying is that I do not know how to
    >> or cannot create multi-threaded functions and I can't call matlab
    >> functions in shell scripts (matlab doesn't have a shebang for
    >> scripting).

    >
    > All valid points, except that you can call matlab functions on the UNIX
    > (and Windows?) command line (though probably not with a shebang, but I
    > don't see that as essential).
    >
    >> For a start, I've noticed these useful sites:
    >> http://narray.rubyforge.org/index.html.en
    >> http://rb-gsl.rubyforge.org/
    >> http://sciruby.codeforpeople.com/sr.cgi/FrontPage

    >
    > Thanks for the pointers.
    >
    > Chris


    i do a lot with narray, mmap, and gsl. let me know if you have questions.

    what are your current projects? if you don't mind me asking...

    cheers.

    -a

    --
    happiness is not something ready-made. it comes from your own actions.
    - h.h. the 14th dali lama
     
    , Feb 2, 2006
    #10
  11. Darren L. Weber

    Guest

    On Fri, 3 Feb 2006 wrote:

    > Phil
    >
    >> narray is implemented in C.

    >
    > I took a quick look at its API and it only seemed to offer the LU
    > matrix decomposition; which seems a bit minimal.
    >
    >> It would be great to have a completely free tool for research since
    >> computational methods are becoming central to so much research.

    >
    > Agreed.


    but you can use many gsl methods with narray - check out the docs.

    narrays are also stored in memory just like c arrays - so you may call a whole
    world of methods on the data via third party libs or even dl calls.

    cheers.

    -a

    --
    happiness is not something ready-made. it comes from your own actions.
    - h.h. the 14th dali lama
     
    , Feb 2, 2006
    #11
  12. Phil Tomson wrote:
    > In article <>,
    > anne001 <> wrote:

    ...
    >> I don't know how much stats ruby can do, certainly the matlab stat
    >> package is not complete, but I can't imagine that a programming
    >> language would be that developed in this area.
    >>
    >> Another reason to stay with matlab is readability. Matlab is a higher
    >> level language. It sounds like you are a programmer selling a product,
    >> so it might be different, but I work at a university, and students
    >> inherit the matlab programs from one another, and then typically they
    >> have to figure out what the program does to get the data in correctly
    >> for their data, get the figures they want for their conditions... the
    >> fact that it is written in a higher level language seems to help.

    >
    > YMMV. I don't find Matlab very readable. I much prefer Ruby's syntax.


    Unless MATLAB has changed in the last 3 or 4 years, it's a real pain to
    work with objects, as opposed to numbers. The problem I could not
    overcome was that all assignments have copy semantics rather than
    reference semantics. There was no way to take a reference to an object
    and assign that to a variable or to a field in some structure. So is was
    effectively impossible to build data structures whose reference graphs
    were not trees.

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    Joel VanderWerf, Feb 2, 2006
    #12
  13. Phil Tomson wrote:
    > Here's an tantalizing idea that I'll throw out: C is one thing and, yes, it's
    > fast.... but what if you could actually map some algorithm or math operation to
    > _hardware_ and thus get a large speed advantage even over C? I'm working on
    > something like this... details forthcoming. The idea is very similar to
    > Ruby::Inline. It will allow seamless communication between your Ruby program
    > and an FPGA board acting as hardware accelerator (these boards are available
    > for less than $200 now).


    That would be very cool. Eventually, we're going to be porting a
    wireless network stack to FPGA, and being able to still use Ruby would
    be very nice.

    What kind of code goes inside your "inline" blocks?

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    Joel VanderWerf, Feb 2, 2006
    #13
  14. Darren L. Weber

    Guest

    On Fri, 3 Feb 2006, Joel VanderWerf wrote:

    > Phil Tomson wrote:
    >> Here's an tantalizing idea that I'll throw out: C is one thing and, yes, it's
    >> fast.... but what if you could actually map some algorithm or math operation to
    >> _hardware_ and thus get a large speed advantage even over C? I'm working on
    >> something like this... details forthcoming. The idea is very similar to
    >> Ruby::Inline. It will allow seamless communication between your Ruby program
    >> and an FPGA board acting as hardware accelerator (these boards are available
    >> for less than $200 now).

    >
    > That would be very cool. Eventually, we're going to be porting a
    > wireless network stack to FPGA, and being able to still use Ruby would
    > be very nice.
    >
    > What kind of code goes inside your "inline" blocks?


    this sounds insanely cool. can you ping me offline phil?

    -a

    --
    happiness is not something ready-made. it comes from your own actions.
    - h.h. the 14th dali lama
     
    , Feb 2, 2006
    #14
  15. Acceleration of algorithms by moving the number-crunching parts to FPGAs
    has been well-utilized in the life sciences industry for a while now.
    Due to non-disclosure agreements, I cannot provide the details, but my
    company has been providing such solutions for the last couple of years.

    Normally, once the board design is finalized, a suitable C
    implementation of the algorithm is used as the basis for a Verilog
    implementation to be burned into the FPGA. Typically, these FPGAs are
    mounted on PCI cards or USB devices.

    A driver does the interface between the application in software and the
    FPGA, handling interrupt events, data handover, etc.

    I would be interested in your work, unless you are building a commercial
    closed-source solution.

    Best regards,

    JS

    wrote:
    > On Fri, 3 Feb 2006, Joel VanderWerf wrote:
    >
    >> Phil Tomson wrote:
    >>> Here's an tantalizing idea that I'll throw out: C is one thing and,
    >>> yes, it's
    >>> fast.... but what if you could actually map some algorithm or math
    >>> operation to
    >>> _hardware_ and thus get a large speed advantage even over C? I'm
    >>> working on
    >>> something like this... details forthcoming. The idea is very similar to
    >>> Ruby::Inline. It will allow seamless communication between your Ruby
    >>> program
    >>> and an FPGA board acting as hardware accelerator (these boards are
    >>> available
    >>> for less than $200 now).

    >>
    >> That would be very cool. Eventually, we're going to be porting a
    >> wireless network stack to FPGA, and being able to still use Ruby would
    >> be very nice.
    >>
    >> What kind of code goes inside your "inline" blocks?

    >
    > this sounds insanely cool. can you ping me offline phil?
    >
    > -a
    >
     
    Srinivas Jonnalagadda, Feb 3, 2006
    #15
  16. Darren L. Weber

    Phil Tomson Guest

    In article <>,
    Joel VanderWerf <> wrote:
    >Phil Tomson wrote:
    >> Here's an tantalizing idea that I'll throw out: C is one thing and, yes, it's
    >> fast.... but what if you could actually map some algorithm or math operation to
    >> _hardware_ and thus get a large speed advantage even over C? I'm working on
    >> something like this... details forthcoming. The idea is very similar to
    >> Ruby::Inline. It will allow seamless communication between your Ruby program
    >> and an FPGA board acting as hardware accelerator (these boards are available
    >> for less than $200 now).

    >
    >That would be very cool. Eventually, we're going to be porting a
    >wireless network stack to FPGA, and being able to still use Ruby would
    >be very nice.
    >
    >What kind of code goes inside your "inline" blocks?
    >


    Initially it's going to be VHDL, a hardware description language. There are
    lots of efforts afoot to use C and a hardware description language, but I'm
    just not sure yet that there's much advantage (YET) over using an HDL like
    VHDL. The semantics of an HDL are much different than the semantics of a
    programming language like C or Ruby. HDLs have to model the parallelism
    inherant in hardware.

    So the idea of RubyInline::HDL is evolutionary rather than revolutionary: use
    Ruby for what it's good for and use HDLs for what they're good for. Of course,
    from a practical standpoint, using VHDL also makes it easy to use the free
    (though not open source, unfortuneatly) tools from Xilinx to program their
    FPGAs.

    In the future, I'd like to build on RHDL (Ruby as an HDL) so that it can
    generate VHDL as an output. At that point then the inlined 'language' would
    be RHDL (really a Ruby-based DSL). Also, at some point it would be good to
    support FpgaC which is an open source C programming environment for FPGAs.

    In the short term, you'll either need to know VHDL to use Inline::HDL, or
    perhaps we could build a repository (using gems perhaps?) of useful VHDL
    modules that users could download from some website. Lots of common DSP
    functions could be made available. This could make it so that you wouldn't
    have to know much VHDL to use Inline::HDL.

    So, for example, say you want to use Ruby to do some image processing. Maybe
    you want that image processing to be in real time. Maybe C isn't even fast
    enough for whatever you're doing. You download some image processing packages
    which are written in VHDL and you inline them using Inline::HDL. Your system
    would look something like:

    camera -> FPGA (running image processing algorithms)->PCI bus->Ruby program

    In this scenario the dataflow is pretty much one way, but you can imagine other
    scenarios where your Ruby program sends some data to the FPGA to be processed
    and waits for an answer back. Maybe you're doing some sort of optimization
    that would take a long time in software, but would be much faster if it can be
    implemented in hardware. Your Ruby program would send a block of data over the
    PCI bus to the FPGA card, the FPGA card would then process that data and send
    back the processed data.

    RubyInline::HDL would take care of setting up all the interfaces (hardware and
    software) between the Ruby program and the FPGA board and it would take care
    of running the Xilinx tools to create the bitstream (if one doesn't already
    exist). That's the theory anyway; this is all very preliminary based on some
    work I did last summer using a
    VHDL simulator (ghdl) in place of an FPGA board. I need to finish that up to
    get testing going and then start working with actual hardware. Now
    that my Masters degree is about done (defense is tomorrow!) I'll have some time
    to put into this over the next couple of months.

    Phil
     
    Phil Tomson, Feb 3, 2006
    #16
  17. Darren L. Weber

    Phil Tomson Guest

    In article <>,
    Srinivas Jonnalagadda <> wrote:
    >Acceleration of algorithms by moving the number-crunching parts to FPGAs
    >has been well-utilized in the life sciences industry for a while now.
    >Due to non-disclosure agreements, I cannot provide the details, but my
    >company has been providing such solutions for the last couple of years.
    >


    Yes, this seems to be going on, though it's kind of been in the background.

    >Normally, once the board design is finalized, a suitable C
    >implementation of the algorithm is used as the basis for a Verilog
    >implementation to be burned into the FPGA. Typically, these FPGAs are
    >mounted on PCI cards or USB devices.


    Right, the idea here would be to either use the Raggedstone boards or one of
    the newer Xilinx boards which has USB. The architectures needs to be modular
    to support different boards, though initially I'll concentrate on Xilinx FPGAs.

    >
    >A driver does the interface between the application in software and the
    >FPGA, handling interrupt events, data handover, etc.
    >


    Right, that's what I'm thinking, though I want to make all of that interface
    stuff as transparent as possible to the user by using Ruby to generate code,
    etc.

    >I would be interested in your work, unless you are building a commercial
    >closed-source solution.


    The idea is for this to be open source. But if someone wants to pay me for
    support, such as developing the VHDL code, etc. that's cool. I'm actually
    hoping that as this sort of thing gets popular that there could be some way for
    me to derrive some income from this. I'm planning to spend the next three
    months or so on this putting in 20 to 40 hours/week, so I hope for some
    kind of payoff - it's kind of going to be my unpaid job unless I run out of
    money and have to look for more paying work ;-)

    I think that FPGA acceleration is
    the way we're going to move to the next level of performance as general purpose
    CPUs are not increasing in performance as much as they used to.

    Phil

    >
    >Best regards,
    >
    >JS
    >
    > wrote:
    >> On Fri, 3 Feb 2006, Joel VanderWerf wrote:
    >>
    >>> Phil Tomson wrote:
    >>>> Here's an tantalizing idea that I'll throw out: C is one thing and,
    >>>> yes, it's
    >>>> fast.... but what if you could actually map some algorithm or math
    >>>> operation to
    >>>> _hardware_ and thus get a large speed advantage even over C? I'm
    >>>> working on
    >>>> something like this... details forthcoming. The idea is very similar to
    >>>> Ruby::Inline. It will allow seamless communication between your Ruby
    >>>> program
    >>>> and an FPGA board acting as hardware accelerator (these boards are
    >>>> available
    >>>> for less than $200 now).
    >>>
    >>> That would be very cool. Eventually, we're going to be porting a
    >>> wireless network stack to FPGA, and being able to still use Ruby would
    >>> be very nice.
    >>>
    >>> What kind of code goes inside your "inline" blocks?

    >>
    >> this sounds insanely cool. can you ping me offline phil?
    >>
    >> -a
    >>

    >
    >
     
    Phil Tomson, Feb 3, 2006
    #17
  18. On 2/2/06, Phil Tomson <> wrote:
    >
    > In the future, I'd like to build on RHDL (Ruby as an HDL) so that it can
    > generate VHDL as an output. At that point then the inlined 'language' wo=

    uld
    > be RHDL (really a Ruby-based DSL). Also, at some point it would be good =

    to
    > support FpgaC which is an open source C programming environment for FPGAs=
     
    Wilson Bilkovich, Feb 3, 2006
    #18
  19. Phil Tomson wrote:
    > Yes, and part of the philosophical argument as I see it is that tools like
    > MatLab are becoming 'required' for doing research (many people think so
    > anyway), kind of like using LaTeX for writing papers. However, LaTeX is free.
    > It would be great to have a completely free tool for research since
    > computational methods are becoming central to so much research.
    >

    <Saturday morning rant>

    If you want to be part of a research community, you need a good reason
    *not* to use the language that community uses! In the finance community,
    APL, Matlab, Mathematica and to some extent R are the tools of choice.
    In digital signal processing, if you don't know Matlab, you might as
    well find some other line of work. In statistics, there are more
    options, but R is a heavyweight, as it is in bioinformatics.

    But let's talk briefly about money. :) One of the things that still
    amazes me, even though I've been part of a few open source communities
    for almost six years now, is the quality of free/open source software,
    essentially built by people who are doing it for love and not for money.
    There are *four* open source LISP implementations, Perl, Python, PHP,
    Ruby/Rails/PostgreSQL/MySQL/SQLite, R (and all the R library packages),
    Maxima, Axiom, Atlas, TeXmacs, LyX, and of course KDE, sitting on my
    workstation right now. And that workstation is running the free Gentoo
    Linux OS, the Linux 2.6 kernel and the entire GNU toolchain. And it all
    works -- most of it beta-level and sometimes even alpha-level!

    I haven't added up the lines of code -- which I could do easily since
    Gentoo is built from source :) -- but it must be in the millions! To
    collect the functional equivalents in commercial software ... well, for
    openers, the entry-level price for Matlab and Mathematica alone is about
    $1000 US each, IIRC. Add in SQL Server 2000, Windows XP, Visual Studio,
    a decent scientific typesetting package, a desktop publishing package,
    etc., and you'll probably spend $10000 US on software to get what I have

    One thing Ruby (and Perl and Python) is very good at is providing the
    "glue logic" to build upon existing software, whether free/open source
    or commercial. The PragProg folks even have a book on the subject, IIRC
    called "Enterprise Integration with Ruby". So ... there is a free tool
    for just about every research need, and there's no reason in the world a
    Ruby programmer can't have the best of all worlds.

    </Saturday morning rant>

    --
    M. Edward (Ed) Borasky

    http://linuxcapacityplanning.com
     
    M. Edward (Ed) Borasky, Feb 18, 2006
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Quick Function
    Replies:
    2
    Views:
    3,682
    Michael Borgwardt
    Nov 28, 2004
  2. eric jones
    Replies:
    0
    Views:
    689
    eric jones
    Jun 27, 2003
  3. Travis N. Vaught
    Replies:
    0
    Views:
    352
    Travis N. Vaught
    Jul 26, 2003
  4. Fie Pye
    Replies:
    11
    Views:
    678
    Fabian Braennstroem
    Oct 8, 2006
  5. Fie Pye
    Replies:
    0
    Views:
    296
    Fie Pye
    Sep 6, 2006
Loading...

Share This Page