Java program that "surfers" the web

Discussion in 'Java' started by yuvalste@hotmail.com, Apr 8, 2007.

  1. Guest

    Hi
    I'd like to write a JAVA program that "surfers" the web -
    Today I have a site that I'm using to get some data. I'm using a user
    & password to get in and retrieve the data. All of this I do manually
    these days.
    I'd like my program to access that web site and retrieve that data
    automatically (meaning it will have to insert user & password, find
    the write form and etc.)

    Which classes should I be using ?
     
    , Apr 8, 2007
    #1
    1. Advertising

  2. Mich Guest

    <> wrote in message
    news:...
    > Hi
    > I'd like to write a JAVA program that "surfers" the web -
    > Today I have a site that I'm using to get some data. I'm using a user
    > & password to get in and retrieve the data. All of this I do manually
    > these days.
    > I'd like my program to access that web site and retrieve that data
    > automatically (meaning it will have to insert user & password, find
    > the write form and etc.)
    >
    > Which classes should I be using ?


    First, you need to be able to read the page and use something like
    org.apache.commons.httpclient.HttpConnection
     
    Mich, Apr 8, 2007
    #2
    1. Advertising

  3. <> wrote in message
    news:...
    > Hi
    > I'd like to write a JAVA program that "surfers" the web -
    > Today I have a site that I'm using to get some data. I'm using a user
    > & password to get in and retrieve the data. All of this I do manually
    > these days.
    > I'd like my program to access that web site and retrieve that data
    > automatically (meaning it will have to insert user & password, find
    > the write form and etc.)
    >
    > Which classes should I be using ?
    >


    Programs that surf the web automatically are called webcrawlers usually.

    This is the top link searching google for Java Webcrawler

    http://java.sun.com/developer/technicalArticles/ThirdParty/WebCrawler/

    It includes example code.

    Making a simple webcrawler first would make sense - worry about
    authentication afterward.

    Are you trying to retrieve all the information from the same site?

    --
    LTP

    :)
     
    Luc The Perverse, Apr 8, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rey
    Replies:
    4
    Views:
    11,074
    Roedy Green
    Dec 12, 2003
  2. Replies:
    1
    Views:
    545
    Andrew Thompson
    Oct 31, 2005
  3. joshivaibhav
    Replies:
    3
    Views:
    692
    joshivaibhav
    Nov 21, 2006
  4. Shadow123
    Replies:
    1
    Views:
    420
    Shadow123
    Jul 9, 2008
  5. Shadow123
    Replies:
    0
    Views:
    311
    Shadow123
    Aug 11, 2008
Loading...

Share This Page