indexing web pages - in python?

[Dan Stromberg]

Are there any open source search engines written in python for indexing a
given collection of (internal only) html pages? Right now I'm talking
about dozens, but hopefully it'll be hundreds or thousands at some point.

I'm thinking some sort of CGI script, with perhaps a cron job that updates
the indexes.

I'm not particularly looking for something that has a full RDBMS behind
it - just a file that stores indexes. I'll go with an RDBMS-based
solution if I must, but I don't think that's really needed at this point.

TIA

 

[Kevin T. Ryan]

Are there any open source search engines written in python for indexing a
given collection of (internal only) html pages? Right now I'm talking
about dozens, but hopefully it'll be hundreds or thousands at some point.

I'm thinking some sort of CGI script, with perhaps a cron job that updates
the indexes.

I'm not particularly looking for something that has a full RDBMS behind
it - just a file that stores indexes. I'll go with an RDBMS-based
solution if I must, but I don't think that's really needed at this point.

TIA

You could try:

http://gnosis.cx/download/indexer.py

There is an extensive write-up by the author at:

http://gnosis.cx/publish/programming/charming_python_15.txt

Might be something you'd be interested in ...