[ANN] FastRI 0.2.0: full-text searching, smarter search strategies

[Mauricio Fernandez]

FastRI is an alternative to the ri command-line tool. It is *much* faster, and
also allows you to offer RI lookup services over DRb. FastRI is smarter than
ri, and can find classes anywhere in the hierarchy without specifying the
"full path". FastRI can perform full-text searching. Its RubyGems support is
better than ri's, and it knows which gem a method/class definition came from.

Getting it
==========
Additional information, tarballs... at
http://eigenclass.org/hiki.rb?fastri

FastRI can be installed with RubyGems:
gem install fastri
(if you get an old version/a 404 error, please allow some time after the
release until the package propagates to the RubyForge mirrors). Please read
below for an important note regarding the RubyGems packages.

Changes since version 0.1.1 (2006-11-10)
========================================
Features
--------
* fri can do full-text search (-S, --full-text); try fri -S byte order
* fri can now determine where a method actually came from for core classes
e.g. fri File.inject -> docs for Enumerable#inject
* you can specify which ports the DRb services must bind to:
fastri-server -s 192.168.1.2:54321
fri -s 192.168.1.2:12345
* new search methods: "anywhere" (a) and "anywhere, case-indep." (A)

Acknowledgements
================
Tomasz Wegrzanowski
* contributed the code that allows fri to find a method in the ancestors
for core classes

Usage
=====
There are two parts to FastRI:
* the server: fastri-server
* the client: fri

FastRI uses a Rinda Ring to allow servers to be discovered automatically
without needing to indicate the DRb URIs manually. It can work across
machines if you make sure the ring server is bound to the correct interface,
and the ACL permissions are correct.

Examples
========
$ fastri-server (creates the index on the first run, blocks)

Later, (times measured with a cold cache):
$ time ruby bin/fri -f plain Array#fetch
------------------------------------------------------------ Array#fetch
array.fetch(index) -> obj
[...]
real 0m0.287s (real 0m0.127s with a hot cache)
user 0m0.048s
sys 0m0.008s

Compare to:
$ time ri -T -f plain Array#fetch
------------------------------------------------------------ Array#fetch
[...]
real 0m10.136s (real ~ 1.5s with a hot cache)
user 0m1.140s
sys 0m0.464s

This illustrates FastRI's ability to locate classes deep in the class
hierarchy:

$ fri Base
------------------------------------------------------ Multiple choices:

ActionMailer::Base, ActionView::Base, ActionWebService::API::Base,
ActionWebService::Base, ActionWebService::Client::Base,
ActiveRecord::Base, MapReduce::ActiveRecord::Base,
RSS::Maker::Base, Scruffy::Components::Base,
Scruffy::Formatters::Base, Scruffy::Layers::Base,
Scruffy::Renderers::Base, Scruffy::Themes::Base

$ fri Themes::Base
------------------------------------------- Class: Scruffy::Themes::Base
Scruffy::Themes::Base
Author: Brasten Sager

Date: August 14th, 2006

Compare to
$ ri Themes::Base .... several seconds later ...
Nothing known about Themes::Base

A small note about RubyGems + FastRI.
=====================================
RubyGems adds a noticeable overhead to fri, making it run slower than if you
installed it directly from the tarball with setup.rb.

Compare the execution time when installed with RubyGems:
$ time fri -f plain String > /dev/null

real 0m0.385s
user 0m0.244s
sys 0m0.036s

to the time fri actually takes to run, without the overhead introduced by
RubyGems:
$ time ruby bin/fri -f plain String > /dev/null

real 0m0.088s
user 0m0.040s
sys 0m0.008s

If you care about those extra 300ms (and there are situations where they will
matter, e.g. when using fri for method completion), get FastRI from the
tarballs.

License
=======
FastRI is licensed under the same terms as Ruby. See LICENSE.

Feedback
========
Bug reports, patches, comments... are appreciated.
You can contact the author via <[email protected]>. Please add "fastri" to the
subject in order to bypass the spam filters.

 

[Gabriele Marrone]

I like fastri

Did you implement a way to tell fri about the default pager it should
use?
On the 0.1.1 announcement thread you said you probably would have
done this in 0.2.0.
I tried by setting a PAGER envvar but it didn't work (many unix tools
check for that envvar, "PAGER=cat man man" is just an example).

 

[Mauricio Fernandez]

I like fastri


Did you implement a way to tell fri about the default pager it should use?
On the 0.1.1 announcement thread you said you probably would have done this
in 0.2.0. I tried by setting a PAGER envvar but it didn't work (many unix
tools check for that envvar, "PAGER=cat man man" is just an example).

Sorry, it's not in 0.2.0, but you can find it in HEAD; just get
http://eigenclass.org/repos/fastri/head/bin/fri

$ fri -h

Usage: fri [options] <query>
-s, --bind ADDR Bind to ADDR for incoming DRb connections.
(default: 127.0.0.1)
-O, --order ORDER Specify lookup order.
(default: eEnNpPxX)
Uppercase: case-indep.
e:exact n:nested partial (completion)
x:nested and partial
a:match method name anywhere
--show-matches Only show matching entries.
-S, --full-text Perform full-text search.
-F, --full-text-dir DIR Use full-text index in DIR
(default: /home/batsman/.fastri-fulltext)
-f, --format FMT Format to use when displaying output:
ansi, plain (default: ansi)
-P, --[no-]pager Use pager.
(default: don't)
--pager-cmd PAGER Use pager PAGER.
(default: don't)
-w, --width WIDTH Set the width of the output.
-h, --help Show this help message


fri will use a pager by default in full-text search mode, (either the one
given to --pager-cmd, ENV["PAGER"], "less", "more" or "pager", in that order)
but *not* in normal mode, unless you ask for it with -P.

You can alias fri="fri -P" to always use a pager, and it will honor the PAGER
env. variable.

I might as well implement option parsing for ENV["FRI_OPTIONS"].

 

[Peña, Botp]

:From =3D> "Mauricio Julio Fern=E1ndez Pradier"
# On Fri, Nov 17, 2006 at 12:12:52AM +0900, Gabriele Marrone wrote:
# > I like fastri
#

Hi Mauricio,

1) it seems that fri does not implem multiple queries yet? eg

C:\temp>fri upcase downcase
---------------------------------------------------------- String#upcase
str.upcase =3D> new_str
------------------------------------------------------------------------
Returns a copy of _str_ with all lowercase letters replaced with
their uppercase counterparts. The operation is locale
insensitive---only characters ``a'' to ``z'' are affected.

"hEllO".upcase #=3D> "HELLO"


C:\temp>

yet

C:\temp>ri -f plain -T upcase downcase
---------------------------------------------------------- String#upcase
str.upcase =3D> new_str
------------------------------------------------------------------------
Returns a copy of _str_ with all lowercase letters replaced with
their uppercase counterparts. The operation is locale
insensitive---only characters ``a'' to ``z'' are affected.

"hEllO".upcase #=3D> "HELLO"

-------------------------------------------------------- String#downcase
str.downcase =3D> new_str
------------------------------------------------------------------------
Returns a copy of _str_ with all uppercase letters replaced with
their lowercase counterparts. The operation is locale
insensitive---only characters ``A'' to ``Z'' are affected.

"hEllO".downcase #=3D> "hello"


2) also, can we allow a -T for fri (since some do like pager while =
others like do not --sometimes. i prefer piping and teeing and all =
output in one


C:\temp>fri -f plain -T upcase downcase
c:/ruby/lib/ruby/1.8/optparse.rb:1381:in `complete': invalid option: -T =
(OptionParser::InvalidOption)


3) it would be nice if queries (if possible) may return related topics. =
eg, something like, "Related methods: capitalize, downcase, upcase, =
trim, ltrim, etc...". This is very good for noobies like me. this may be =
quite a hard work since we may need an index..=20


thanks for fri. it really is fast even on mswindows.
kind regards -botp

 

[Mauricio Fernandez]

1) it seems that fri does not implem multiple queries yet? eg

oh, I'd never used that with ri (actually, I didn't use ri that much, for the
reason you know)

2) also, can we allow a -T for fri (since some do like pager while others
like do not --sometimes. i prefer piping and teeing and all output in one

In HEAD, you have
$ fri -h
...
-P, --[no-]pager Use pager.
(default: don't)
--pager-cmd PAGER Use pager PAGER.
(default: don't)

By default, it doesn't use a pager, unless you are doing full-text searches.

3) it would be nice if queries (if possible) may return related topics. eg,
something like, "Related methods: capitalize, downcase, upcase, trim, ltrim,
etc...".

Yes, I'm thinking about the best way to achieve this; the obvious candidate is
latent semantic indexing.

This is very good for noobies like me. this may be quite a hard work since
we may need an index..

You cannot keep playing the newbie card for years ... you've been around
since Jan 2001 at least! (I found your 1st post, according to blade, in
RubyConf's foundational thread [ruby-talk:9868]!)

 

[Thomas Nitsche]

Hi,

remote access doesn't work for me. Anyone got this up an running? I did
something like:

fastri-server -a 192.168.100.0/24 -s 192.168.100.163
Looking for Ring server...
No Ring server found, starting my own.
$ fastri-server 0.0.1 (FastRI 0.2.0) listening on
druby://192.168.100.163:35800
ACL:
deny all
allow 127.0.0.1
allow 192.168.100.0/24

Local access works just fine, but from remote I got:
$ fri -s 192.168.100.163 Array
Couldn't initialize DRb and locate the Ring server.

Any ideas?

Regards

Thomas

 

[Mauricio Fernandez]

remote access doesn't work for me. Anyone got this up an running? I did
something like:

fastri-server -a 192.168.100.0/24 -s 192.168.100.163 ====================
Looking for Ring server...
No Ring server found, starting my own.
$ fastri-server 0.0.1 (FastRI 0.2.0) listening on
druby://192.168.100.163:35800
ACL:
deny all
allow 127.0.0.1
allow 192.168.100.0/24

Local access works just fine, but from remote I got:
$ fri -s 192.168.100.163 Array =================
Couldn't initialize DRb and locate the Ring server.

The address given to fri -s ADDR is not the address of the server (that one
is autodiscovered using the Ring), but the address the local DRb service must
bind to (in short, fri also exports a DRb service that will be used by the
remote fastri-server to tell fri where it is).

So, if your server is 192.168.100.163 and your client is running on
192.168.100.164, you'd have to do

(in 192.168.100.163)
$ fastri-server -a 192.168.100.0/24 -s 192.168.100.163

(in 192.168.100.164)
$ fri -s 192.168.100.164 Array

or

$ export FASTRI_ADDR=192.168.100.164
$ fri Array


I'm revisiting that code to see if I can make 'fri Array' work without
specifying the local address, but for the time being the above should work.

 

[Thomas Nitsche]

Thx a lot. It works perfectly that way. It's almost sort of RTFM, but I
misunderstood the sentence "Note that FASTRI_ADDR is the *local*
address: the server will be discovered automatically."

Regards,

Thomas