[ANN] FastRI 0.2.0: full-text searching, smarter search strategies

Discussion in 'Ruby' started by Mauricio Fernandez, Nov 15, 2006.

  1. FastRI is an alternative to the ri command-line tool. It is *much* faster, and
    also allows you to offer RI lookup services over DRb. FastRI is smarter than
    ri, and can find classes anywhere in the hierarchy without specifying the
    "full path". FastRI can perform full-text searching. Its RubyGems support is
    better than ri's, and it knows which gem a method/class definition came from.

    Getting it
    ==========
    Additional information, tarballs... at
    http://eigenclass.org/hiki.rb?fastri

    FastRI can be installed with RubyGems:
    gem install fastri
    (if you get an old version/a 404 error, please allow some time after the
    release until the package propagates to the RubyForge mirrors). Please read
    below for an important note regarding the RubyGems packages.

    Changes since version 0.1.1 (2006-11-10)
    ========================================
    Features
    --------
    * fri can do full-text search (-S, --full-text); try fri -S byte order
    * fri can now determine where a method actually came from for core classes
    e.g. fri File.inject -> docs for Enumerable#inject
    * you can specify which ports the DRb services must bind to:
    fastri-server -s 192.168.1.2:54321
    fri -s 192.168.1.2:12345
    * new search methods: "anywhere" (a) and "anywhere, case-indep." (A)

    Acknowledgements
    ================
    Tomasz Wegrzanowski
    * contributed the code that allows fri to find a method in the ancestors
    for core classes

    Usage
    =====
    There are two parts to FastRI:
    * the server: fastri-server
    * the client: fri

    FastRI uses a Rinda Ring to allow servers to be discovered automatically
    without needing to indicate the DRb URIs manually. It can work across
    machines if you make sure the ring server is bound to the correct interface,
    and the ACL permissions are correct.

    Examples
    ========
    $ fastri-server (creates the index on the first run, blocks)

    Later, (times measured with a cold cache):
    $ time ruby bin/fri -f plain Array#fetch
    ------------------------------------------------------------ Array#fetch
    array.fetch(index) -> obj
    [...]
    real 0m0.287s (real 0m0.127s with a hot cache)
    user 0m0.048s
    sys 0m0.008s

    Compare to:
    $ time ri -T -f plain Array#fetch
    ------------------------------------------------------------ Array#fetch
    [...]
    real 0m10.136s (real ~ 1.5s with a hot cache)
    user 0m1.140s
    sys 0m0.464s

    This illustrates FastRI's ability to locate classes deep in the class
    hierarchy:

    $ fri Base
    ------------------------------------------------------ Multiple choices:

    ActionMailer::Base, ActionView::Base, ActionWebService::API::Base,
    ActionWebService::Base, ActionWebService::Client::Base,
    ActiveRecord::Base, MapReduce::ActiveRecord::Base,
    RSS::Maker::Base, Scruffy::Components::Base,
    Scruffy::Formatters::Base, Scruffy::Layers::Base,
    Scruffy::Renderers::Base, Scruffy::Themes::Base

    $ fri Themes::Base
    ------------------------------------------- Class: Scruffy::Themes::Base
    Scruffy::Themes::Base
    Author: Brasten Sager

    Date: August 14th, 2006

    Compare to
    $ ri Themes::Base .... several seconds later ...
    Nothing known about Themes::Base

    A small note about RubyGems + FastRI.
    =====================================
    RubyGems adds a noticeable overhead to fri, making it run slower than if you
    installed it directly from the tarball with setup.rb.

    Compare the execution time when installed with RubyGems:
    $ time fri -f plain String > /dev/null

    real 0m0.385s
    user 0m0.244s
    sys 0m0.036s

    to the time fri actually takes to run, without the overhead introduced by
    RubyGems:
    $ time ruby bin/fri -f plain String > /dev/null

    real 0m0.088s
    user 0m0.040s
    sys 0m0.008s

    If you care about those extra 300ms (and there are situations where they will
    matter, e.g. when using fri for method completion), get FastRI from the
    tarballs.

    License
    =======
    FastRI is licensed under the same terms as Ruby. See LICENSE.

    Feedback
    ========
    Bug reports, patches, comments... are appreciated.
    You can contact the author via <>. Please add "fastri" to the
    subject in order to bypass the spam filters.

    --
    Mauricio Fernandez - http://eigenclass.org - singular Ruby
    Mauricio Fernandez, Nov 15, 2006
    #1
    1. Advertising

  2. I like fastri :)

    Did you implement a way to tell fri about the default pager it should
    use?
    On the 0.1.1 announcement thread you said you probably would have
    done this in 0.2.0.
    I tried by setting a PAGER envvar but it didn't work (many unix tools
    check for that envvar, "PAGER=cat man man" is just an example).

    --
    Gabriele Marrone
    Gabriele Marrone, Nov 16, 2006
    #2
    1. Advertising

  3. On Fri, Nov 17, 2006 at 12:12:52AM +0900, Gabriele Marrone wrote:
    > I like fastri :)


    :)

    > Did you implement a way to tell fri about the default pager it should use?
    > On the 0.1.1 announcement thread you said you probably would have done this
    > in 0.2.0. I tried by setting a PAGER envvar but it didn't work (many unix
    > tools check for that envvar, "PAGER=cat man man" is just an example).


    Sorry, it's not in 0.2.0, but you can find it in HEAD; just get
    http://eigenclass.org/repos/fastri/head/bin/fri

    $ fri -h

    Usage: fri [options] <query>
    -s, --bind ADDR Bind to ADDR for incoming DRb connections.
    (default: 127.0.0.1)
    -O, --order ORDER Specify lookup order.
    (default: eEnNpPxX)
    Uppercase: case-indep.
    e:exact n:nested p:partial (completion)
    x:nested and partial
    a:match method name anywhere
    --show-matches Only show matching entries.
    -S, --full-text Perform full-text search.
    -F, --full-text-dir DIR Use full-text index in DIR
    (default: /home/batsman/.fastri-fulltext)
    -f, --format FMT Format to use when displaying output:
    ansi, plain (default: ansi)
    -P, --[no-]pager Use pager.
    (default: don't)
    --pager-cmd PAGER Use pager PAGER.
    (default: don't)
    -w, --width WIDTH Set the width of the output.
    -h, --help Show this help message


    fri will use a pager by default in full-text search mode, (either the one
    given to --pager-cmd, ENV["PAGER"], "less", "more" or "pager", in that order)
    but *not* in normal mode, unless you ask for it with -P.

    You can alias fri="fri -P" to always use a pager, and it will honor the PAGER
    env. variable.

    I might as well implement option parsing for ENV["FRI_OPTIONS"].
    --
    Mauricio Fernandez - http://eigenclass.org - singular Ruby
    Mauricio Fernandez, Nov 16, 2006
    #3
  4. Mauricio Fernandez

    Peña, Botp Guest

    :From =3D> "Mauricio Julio Fern=E1ndez Pradier"
    # On Fri, Nov 17, 2006 at 12:12:52AM +0900, Gabriele Marrone wrote:
    # > I like fastri :)
    # :)

    Hi Mauricio,

    1) it seems that fri does not implem multiple queries yet? eg

    C:\temp>fri upcase downcase
    ---------------------------------------------------------- String#upcase
    str.upcase =3D> new_str
    ------------------------------------------------------------------------
    Returns a copy of _str_ with all lowercase letters replaced with
    their uppercase counterparts. The operation is locale
    insensitive---only characters ``a'' to ``z'' are affected.

    "hEllO".upcase #=3D> "HELLO"


    C:\temp>

    yet

    C:\temp>ri -f plain -T upcase downcase
    ---------------------------------------------------------- String#upcase
    str.upcase =3D> new_str
    ------------------------------------------------------------------------
    Returns a copy of _str_ with all lowercase letters replaced with
    their uppercase counterparts. The operation is locale
    insensitive---only characters ``a'' to ``z'' are affected.

    "hEllO".upcase #=3D> "HELLO"

    -------------------------------------------------------- String#downcase
    str.downcase =3D> new_str
    ------------------------------------------------------------------------
    Returns a copy of _str_ with all uppercase letters replaced with
    their lowercase counterparts. The operation is locale
    insensitive---only characters ``A'' to ``Z'' are affected.

    "hEllO".downcase #=3D> "hello"


    2) also, can we allow a -T for fri (since some do like pager while =
    others like do not --sometimes. i prefer piping and teeing and all =
    output in one :)


    C:\temp>fri -f plain -T upcase downcase
    c:/ruby/lib/ruby/1.8/optparse.rb:1381:in `complete': invalid option: -T =
    (OptionParser::InvalidOption)


    3) it would be nice if queries (if possible) may return related topics. =
    eg, something like, "Related methods: capitalize, downcase, upcase, =
    trim, ltrim, etc...". This is very good for noobies like me. this may be =
    quite a hard work since we may need an index..=20


    thanks for fri. it really is fast even on mswindows.
    kind regards -botp
    Peña, Botp, Nov 17, 2006
    #4
  5. On Fri, Nov 17, 2006 at 11:08:56AM +0900, Peña, Botp wrote:
    > 1) it seems that fri does not implem multiple queries yet? eg


    oh, I'd never used that with ri (actually, I didn't use ri that much, for the
    reason you know)

    > 2) also, can we allow a -T for fri (since some do like pager while others
    > like do not --sometimes. i prefer piping and teeing and all output in one :)


    In HEAD, you have
    $ fri -h
    ...
    -P, --[no-]pager Use pager.
    (default: don't)
    --pager-cmd PAGER Use pager PAGER.
    (default: don't)

    By default, it doesn't use a pager, unless you are doing full-text searches.

    > 3) it would be nice if queries (if possible) may return related topics. eg,
    > something like, "Related methods: capitalize, downcase, upcase, trim, ltrim,
    > etc...".


    Yes, I'm thinking about the best way to achieve this; the obvious candidate is
    latent semantic indexing.

    > This is very good for noobies like me. this may be quite a hard work since
    > we may need an index..


    You cannot keep playing the newbie card for years ;) ... you've been around
    since Jan 2001 at least! (I found your 1st post, according to blade, in
    RubyConf's foundational thread [ruby-talk:9868]!)

    --
    Mauricio Fernandez - http://eigenclass.org - singular Ruby
    Mauricio Fernandez, Nov 18, 2006
    #5
  6. Re: FastRI 0.2.0: full-text searching, smarter search strate

    Hi,

    remote access doesn't work for me. Anyone got this up an running? I did
    something like:

    fastri-server -a 192.168.100.0/24 -s 192.168.100.163
    Looking for Ring server...
    No Ring server found, starting my own.
    $ fastri-server 0.0.1 (FastRI 0.2.0) listening on
    druby://192.168.100.163:35800
    ACL:
    deny all
    allow 127.0.0.1
    allow 192.168.100.0/24

    Local access works just fine, but from remote I got:
    $ fri -s 192.168.100.163 Array
    Couldn't initialize DRb and locate the Ring server.

    Any ideas?

    Regards

    Thomas

    --
    Posted via http://www.ruby-forum.com/.
    Thomas Nitsche, Nov 21, 2006
    #6
  7. Re: FastRI 0.2.0: full-text searching, smarter search strate

    On Wed, Nov 22, 2006 at 12:34:29AM +0900, Thomas Nitsche wrote:
    > remote access doesn't work for me. Anyone got this up an running? I did
    > something like:
    >
    > fastri-server -a 192.168.100.0/24 -s 192.168.100.163

    ====================
    > Looking for Ring server...
    > No Ring server found, starting my own.
    > $ fastri-server 0.0.1 (FastRI 0.2.0) listening on
    > druby://192.168.100.163:35800
    > ACL:
    > deny all
    > allow 127.0.0.1
    > allow 192.168.100.0/24
    >
    > Local access works just fine, but from remote I got:
    > $ fri -s 192.168.100.163 Array

    =================
    > Couldn't initialize DRb and locate the Ring server.
    >


    The address given to fri -s ADDR is not the address of the server (that one
    is autodiscovered using the Ring), but the address the local DRb service must
    bind to (in short, fri also exports a DRb service that will be used by the
    remote fastri-server to tell fri where it is).

    So, if your server is 192.168.100.163 and your client is running on
    192.168.100.164, you'd have to do

    (in 192.168.100.163)
    $ fastri-server -a 192.168.100.0/24 -s 192.168.100.163

    (in 192.168.100.164)
    $ fri -s 192.168.100.164 Array

    or

    $ export FASTRI_ADDR=192.168.100.164
    $ fri Array


    I'm revisiting that code to see if I can make 'fri Array' work without
    specifying the local address, but for the time being the above should work.

    --
    Mauricio Fernandez - http://eigenclass.org - singular Ruby
    Mauricio Fernandez, Nov 21, 2006
    #7
  8. Re: FastRI 0.2.0: full-text searching, smarter search strate

    Thx a lot. It works perfectly that way. It's almost sort of RTFM, but I
    misunderstood the sentence "Note that FASTRI_ADDR is the *local*
    address: the server will be discovered automatically."

    Regards,

    Thomas

    --
    Posted via http://www.ruby-forum.com/.
    Thomas Nitsche, Nov 21, 2006
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Samuel R. Neff
    Replies:
    2
    Views:
    579
    bradley
    Jun 10, 2005
  2. Mauricio Fernandez
    Replies:
    24
    Views:
    366
    Gabriele Marrone
    Nov 15, 2006
  3. Mauricio Fernandez
    Replies:
    3
    Views:
    161
    Mauricio Fernandez
    Nov 10, 2006
  4. Mauricio Fernandez
    Replies:
    5
    Views:
    104
    Robert Feldt
    Nov 23, 2006
  5. Daniel Schömer

    FastRI, Error on full-text search

    Daniel Schömer, May 4, 2008, in forum: Ruby
    Replies:
    0
    Views:
    100
    Daniel Schömer
    May 4, 2008
Loading...

Share This Page