Java program that "surfers" the web

[yuvalste]

Hi
I'd like to write a JAVA program that "surfers" the web -
Today I have a site that I'm using to get some data. I'm using a user
& password to get in and retrieve the data. All of this I do manually
these days.
I'd like my program to access that web site and retrieve that data
automatically (meaning it will have to insert user & password, find
the write form and etc.)

Which classes should I be using ?

 

[Mich]

Hi
I'd like to write a JAVA program that "surfers" the web -
Today I have a site that I'm using to get some data. I'm using a user
& password to get in and retrieve the data. All of this I do manually
these days.
I'd like my program to access that web site and retrieve that data
automatically (meaning it will have to insert user & password, find
the write form and etc.)

Which classes should I be using ?

First, you need to be able to read the page and use something like
org.apache.commons.httpclient.HttpConnection

 

[Luc The Perverse]

Hi
I'd like to write a JAVA program that "surfers" the web -
Today I have a site that I'm using to get some data. I'm using a user
& password to get in and retrieve the data. All of this I do manually
these days.
I'd like my program to access that web site and retrieve that data
automatically (meaning it will have to insert user & password, find
the write form and etc.)

Which classes should I be using ?

Programs that surf the web automatically are called webcrawlers usually.

This is the top link searching google for Java Webcrawler

http://java.sun.com/developer/technicalArticles/ThirdParty/WebCrawler/

It includes example code.

Making a simple webcrawler first would make sense - worry about
authentication afterward.

Are you trying to retrieve all the information from the same site?