Web Crawling Spidering Question

[Rusty Hill]

My boss just gave me a project to create a spider that crawls a series of
web pages and harvests certain pieces of data they contain and store that
data in a database. Of course he needs all this done sooner than may be
realistic.

Can anyone point me to some samples, links or articles that might help me
significantly jump start my efforts on this? Thanks.

 

[Nicholas Paldino [.NET/C# MVP]]

Rusty,

Take a look at the HttpWebRequest and HttpWebResponse classes. These
will help you get the contents of the page.

As for placing the data in a database, if you are using Sql Server, take
a look at the System.Data.SqlClient namespace, as they will help you with
getting the data into the database.

 

[David]

Have a look for searcharoo on codeproject.com.

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available

 

[Hakan Fatih YILDIRIM]

u will need httpwebrequest and hhtpwebresponse class in system.net
namespace and also an html parser to parse what you want.i have an
project about this.