web page retrieval

Discussion in 'Java' started by anonymous@coolgroups.com, May 27, 2004.

  1. Guest

    What is the simplest way to retrieve a web page in Java?
    Ideally, I want to pass in a URL and get back a String.
     
    , May 27, 2004
    #1
    1. Advertising

  2. Chris Smith Guest

    wrote:
    > What is the simplest way to retrieve a web page in Java?
    > Ideally, I want to pass in a URL and get back a String.


    Since a URL isn't guaranteed to return text data and you don't know the
    encoding head of time, it's not that simple. You can certainly create a
    URL and get an InputStream, but to convert to characters, you'll need to
    use the Content-Type response header to deduce the encoding.
    Unfortunately, that's rather complex so I'll wait for you to say you
    need it, and then maybe I or someone else will have time to write it for
    you. The basics are:

    URL url = new URL("http://something");
    InputStream stream = url.openStream();

    try
    {
    // read the data from the stream
    }
    finally
    {
    stream.close();
    }

    or, if you want to convert to text with a given character encoding:

    URL url = new URL("http://something");
    InputStream stream = url.openStream();

    try
    {
    InputStreamReader reader = new InputStreamReader(
    stream, "encoding-name");

    // read the data from the reader
    }
    finally
    {
    stream.close();
    }

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
     
    Chris Smith, May 27, 2004
    #2
    1. Advertising

  3. On Thu, 27 May 2004 18:03:23 GMT, wrote:

    > What is the simplest way to retrieve a web page in Java?
    > Ideally, I want to pass in a URL and get back a String.


    Isn't it about time you Bob, that you
    found the JavaDocs and tutorials, and
    headed over to c.l.j.help?

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
     
    Andrew Thompson, May 27, 2004
    #3
  4. kaeli Guest

    In article <>,
    enlightened us with...
    > What is the simplest way to retrieve a web page in Java?
    > Ideally, I want to pass in a URL and get back a String.
    >
    >


    A snippet from code I have that does this very thing...


    URL url=null;
    String line=null;
    InputStream in=null;
    BufferedReader input=null;
    URLConnection connection=null;
    String pageContent="";


    url = new URL("http://www.google.com");
    connection = url.openConnection();
    connection.setDoInput(true);
    in = connection.getInputStream();
    input = new BufferedReader(new InputStreamReader(in));

    while ((line = input.readLine()) != null)
    {
    // put the whole thing into a string
    pageContent += line;
    }


    --
    --
    ~kaeli~
    The man who fell into an upholstery machine is fully
    recovered.
    http://www.ipwebdesign.net/wildAtHeart
    http://www.ipwebdesign.net/kaelisSpace
     
    kaeli, May 27, 2004
    #4
  5. Roedy Green Guest

    On Thu, 27 May 2004 18:03:23 GMT, wrote or
    quoted :

    >What is the simplest way to retrieve a web page in Java?
    >Ideally, I want to pass in a URL and get back a String.


    see http://mindprod.com/fileio.html

    tell it you want to learn to do a CGI get. This what browsers do to
    get webpages.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, May 28, 2004
    #5
  6. BlackFoliage Guest

    wrote:
    > What is the simplest way to retrieve a web page in Java?
    > Ideally, I want to pass in a URL and get back a String.
    >


    The Jakarta commons HttpClient works great.

    http://jakarta.apache.org/commons/httpclient/

    -Patrick
     
    BlackFoliage, May 28, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. George Sew
    Replies:
    2
    Views:
    488
    bruce barker
    Feb 12, 2004
  2. =?Utf-8?B?U3dheQ==?=

    Form information retrieval

    =?Utf-8?B?U3dheQ==?=, Apr 18, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    347
    Pete Beech
    Apr 18, 2004
  3. Rick Spiewak

    Re: HTML Form value retrieval

    Rick Spiewak, May 7, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    447
    =?Utf-8?B?TWF0dA==?=
    May 7, 2004
  4. news.microsoft.com

    ASP.net Ques: Image Retrieval

    news.microsoft.com, Jan 16, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    450
  5. news.microsoft.com

    ASP.net ques: Webpage data retrieval and parsing

    news.microsoft.com, Jan 16, 2005, in forum: ASP .Net
    Replies:
    5
    Views:
    4,981
    Rajesh Kumar
    Feb 1, 2005
Loading...

Share This Page