search source code project

[cmk128]

Hi
Here is my project:

1) Apache + tomcat + mysql (have to use JSP, no php)
2) 3 servers : Linux FC6 , 2GB ram , 2.8Ghz
3) Goal : to build a website to let people search the source code from
over 1 million files.

It is unable to use mysql to build the index from the source code
because people can search "int x=0" rather than english word, so all
the indexing server doesn't word. I guess every time the user submit a
search, i use grep command to search all the 1 million files. But it
is too slow, how can i improve it?

thanks
from Peter ([email protected])

 

[cmk128]

Forgot to say, in my xeon 3Ghz server, i need 10 seconds to grep 20000
files. If i need to grep 1 million files, then i need at least 500
seconds, it is too slow.
thanks
from Peter

 

[dnass]

Forgot to say, in my xeon 3Ghz server, i need 10 seconds to grep 20000
files. If i need to grep 1 million files, then i need at least 500
seconds, it is too slow.
thanks
from Peter

Hi,

Have you tried Lucene http://lucene.apache.org/java/docs/

you are welcome
Dnas

 

[Chris Uppal]

3) Goal : to build a website to let people search the source code from
over 1 million files.

Have you looked at OpenGrok ?

(It uses Lucene as its text search engine internally, but adds stuff for
specialising to searching and displaying code.)

-- chris