Hi Rob,
There are many approaches that can be taken to this problem. I will discuss a few of them here and you can decide which of these will best suit you.
1. Google - Although this doesn't involve any programming (aside from a few simple html commands) I feel that this is worth mentioning. Google provide a search service that you can add to your site easily and allows users to search within your site or on Google itself. However, this being a ASP.Net forum I am guessing you would like to actually implement the searche engine yourself or have a greater level of customization options.
2. (Very) Simple Engine for Database driven content
If your site is primarily made up of content that is stored within a database you can develop a simple search engine by using SQL statements to search the content. As an example, assume you have a database that stores content in a table named "Articles". This table contains the following fields:
ArticleId - A unique index assigned to each article
Author - The name of the author
Submitted - A Date/Time value of when the article was submitted
Title - The title of the article
Content - The content of the article
To create a simple search engine in this scenario you would create a page that had a search box which accepted the search query (txtQuery), a drop down box that allowed you to choose which field you wanted to search within (ddSearchType) and a button (btnSearch) that when clicked searched the database. The "Click" event handler for the button would then create a connection to your database and perform a query such as the following:
Code:
SqlConnection cn = null;
SqlCommand cmd = null;
SqlDataReader rdr = null;
try
{
// open connection to the database using the connection string
// "csDatabase" stored in the web.config file
cn = new SqlConnection(ConfigurationManager.ConnectionStrings["csDatabase"].ConnectionString);
cn.Open();
// Build the simple sql query string
// Select * From Articles WHERE <fieldname> = '%<search terms>%'
string sqlQuery = "SELECT * FROM Articles WHERE " + ddSearchType.SelectedValue + " = '%" + txtQuery.Text + "%'";
cmd = new SqlCommand(sqlQuery, cn);
rdr = cmd.ExecuteReader();
while(rdr.Read())
{
// Insert code that handles the results here....
}
}
catch(Exception ex)
{
// Handle any exceptions that may have occurred
Response.Write(ex.Message);
Response.End();
}
finally
{
// Close any open connections and readers
if(cn != null)
{
cn.Close();
}
if(rdr != null)
{
rdr.Close();
}
}
This sql command will return every article that contains the search terms somewhere within the chosen field. This solution can obviously be amended to allow more advanced searches such as only finding articles that begin or end with the search terms, or only find articles that match the search terms exactly. This is all fairly easy to implement and is up to you to decide how to implement this.
Note: This is a
very simple concept for a
very simple search engine. If you want to add search relevency or search a site that doesn't store it's content entirely (or partially) in a database you need to look at the method below.
3. Crawler / Spider
The third option would be to develop a more advanced system that uses a search engine spider or crawler that will "visit" each of your pages and find out what content you have there. You can then store this into a catalog/database which can then be used to search against.
The crawler should be able to do several things:
- Find a list of files in your site
- Retrieve the content from each page
- Clean up the content (remove html code, script etc.)
- Store the content as a collection of words into a catalog/database pointing to the page(s) they were found on
- Move onto the next page in your site
Depending on how you want to approach the above you could either "crawl" all of the files in a paticular directory on your web server or you could develop a more complex crawler that follows links within your site.
Additionally, you could make your crawler more complex by providing with algorithms to decide what content should be indexed and what content should be left, as well as things like when an image is found indexing it's ALT tags if found rather than just completely ignoring it. Again, this all depends on the needs of your paticular site.
The complexity of your catalog/database is also entirely down to you but all that it really required for a simple search engine is keyword, page position, and the file it was found in.
You can now build a search page that checks the database to find the pages your search terms were found in and build a result set that can be displayed to your user however you want.
You can add any number of algorithms to allow multiple search terms, boolean logic, search relevency etc. but it is up to you to decide how you want to implement these and what ones you actually require.
4. As a final option you could buy a commercial product that you couold install onto your web server. There are lots of these about, a search on google will shine soem light on this.
This response is purposely not meant as a step by step guide on how to build a search engine, but as a few ideas that will get you going. If you are looking for a more detaild explanation or article a quick search on Google will bring up hundreds!
I would suggest that you read a few of these articles. You will then have a good understanding of what type of search engine is right for your site and what the best way to tackle the problem is.
As a quick note about the code snippet above. It is C# code and I haven't tried to build it.
Anyway, I hope this is of some help to you.
Gary Francis
Software Developer
SSI Computer Services Ltd.
Providing IT Support solutions in London