How do you read the content of internal web pages?

Discussion in 'Java' started by Thierry Lam, Jul 25, 2007.

  1. Thierry Lam

    Thierry Lam Guest

    Thierry Lam, Jul 25, 2007
    #1
    1. Advertising

  2. On Jul 25, 11:22 am, Thierry Lam <> wrote:
    > I have the following jsp pages on different tomcat web servers:
    >
    > http://serverone/bugs/download.jsp?...ree/bugs/download.jsp?num=1234&file=Jul25.txt
    >
    > Is there a way in java to read the content of those urls? I can do it
    > easily in perl with the module LWP::UserAgent
    >
    > http://search.cpan.org/~gaas/libwww-perl-5.805/lib/LWP/UserAgent.pm
    >
    > Does anyone have the equivalent for java?
    >
    > Thierry


    You cannot read the "content" of the JSPs, but you can, however, read
    the "output" of those JSPs as if your code is a browser, which is what
    I think you meant by User Agent. You can either use Apache's
    HttpClient API (my preference), or roll your own using the
    java.net.URLConnection. You will find plenty of examples if you google
    these two APIs.

    It is not as straightforward as LWP :)

    -cheers,
    Manish
     
    Manish Pandit, Jul 25, 2007
    #2
    1. Advertising

  3. Thierry Lam

    Thierry Lam Guest

    Yes, it's the content of the jsp pages that I want to read. I usually
    post on the newsgroup when I can't find anything from googling. If
    anyone got any small sample working codes, let me know.
     
    Thierry Lam, Jul 25, 2007
    #3
  4. Thierry Lam

    Daniel Pitts Guest

    On Jul 25, 11:47 am, Thierry Lam <> wrote:
    > Yes, it's the content of the jsp pages that I want to read. I usually
    > post on the newsgroup when I can't find anything from googling. If
    > anyone got any small sample working codes, let me know.


    Something along the lines of:
    new URL("http://serverone/bugs/download.jsp?
    num=1234&file=Jul25.txt").getContent();

    <http://java.sun.com/j2se/1.4.2/docs/api/java/net/URL.html>

    At the very worst case, you will need to call getContentAsStream() and
    then read in the content. although getContent() might to what you
    need.
     
    Daniel Pitts, Jul 25, 2007
    #4
  5. Thierry Lam

    Daniel Pitts Guest

    On Jul 25, 11:32 am, Manish Pandit <> wrote:
    > On Jul 25, 11:22 am, Thierry Lam <> wrote:
    >
    > > I have the following jsp pages on different tomcat web servers:

    >
    > >http://serverone/bugs/download.jsp?num=1234&file=Jul25.txthttp://serv...

    >
    > > Is there a way in java to read the content of those urls? I can do it
    > > easily in perl with the module LWP::UserAgent

    >
    > >http://search.cpan.org/~gaas/libwww-perl-5.805/lib/LWP/UserAgent.pm

    >
    > > Does anyone have the equivalent for java?

    >
    > > Thierry

    >
    > You cannot read the "content" of the JSPs, but you can, however, read
    > the "output" of those JSPs as if your code is a browser, which is what
    > I think you meant by User Agent. You can either use Apache's
    > HttpClient API (my preference), or roll your own using the
    > java.net.URLConnection. You will find plenty of examples if you google
    > these two APIs.
    >
    > It is not as straightforward as LWP :)

    Actually, it is very straightforward. You don't need to mess with
    URLConnection for retreiving the content of specific URLs, new
    URL(urlString).getContent() should do the trick.

    >
    > -cheers,
    > Manish


    Daniel.
     
    Daniel Pitts, Jul 25, 2007
    #5
  6. Thierry Lam

    shakah Guest

    On Jul 25, 2:47 pm, Thierry Lam <> wrote:
    > Yes, it's the content of the jsp pages that I want to read. I usually
    > post on the newsgroup when I can't find anything from googling. If
    > anyone got any small sample working codes, let me know.


    Check out java.net.URLConnection, its getContent() method might be all
    you need.

    Below is a quick-and-dirty example of another way to use
    URLConnection, though you'll have to catch the Exceptions to get it to
    compile cleanly:

    public StringBuffer fetch(String sURL) {
    StringBuffer sbResponse = new StringBuffer(8192) ;

    java.net.URL url = new java.net.URL(sURL) ;
    java.net.URLConnection urlc = url.openConnection() ;
    urlc.setDoInput(true) ;
    urlc.setUseCaches(false) ;

    java.io.InputStream is = urlc.getInputStream() ;
    int nContentLength = urlc.getContentLength() ;
    byte [] ab = new byte[nContentLength] ;
    int nRead=0 ;
    while(nRead < nContentLength) {
    nRead += is.read(ab, nRead, nContentLength - nRead) ;
    }
    sbResponse.append(new String(ab, "utf-8")) ;
    is.close();
    is = null ;

    ((java.net.HttpURLConnection) urlc).disconnect() ;
    urlc = null ;

    return sbResponse ;
    }
     
    shakah, Jul 25, 2007
    #6
  7. Thierry Lam

    Roedy Green Guest

    Roedy Green, Jul 29, 2007
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. hazz
    Replies:
    6
    Views:
    49,646
    SkyUCHC
    Jun 9, 2010
  2. Replies:
    0
    Views:
    726
  3. Nathan Sokalski
    Replies:
    5
    Views:
    1,031
    Gaurav Vaish \(www.Edujini-Labs.com\)
    Jan 10, 2007
  4. Nathan Sokalski
    Replies:
    4
    Views:
    306
    Nathan Sokalski
    Dec 21, 2006
  5. Nathan Sokalski
    Replies:
    4
    Views:
    344
    Nathan Sokalski
    Dec 21, 2006
Loading...

Share This Page