How do you read the content of internal web pages?

T

Thierry Lam

M

Manish Pandit

I have the following jsp pages on different tomcat web servers:

http://serverone/bugs/download.jsp?...ree/bugs/download.jsp?num=1234&file=Jul25.txt

Is there a way in java to read the content of those urls? I can do it
easily in perl with the module LWP::UserAgent

http://search.cpan.org/~gaas/libwww-perl-5.805/lib/LWP/UserAgent.pm

Does anyone have the equivalent for java?

Thierry

You cannot read the "content" of the JSPs, but you can, however, read
the "output" of those JSPs as if your code is a browser, which is what
I think you meant by User Agent. You can either use Apache's
HttpClient API (my preference), or roll your own using the
java.net.URLConnection. You will find plenty of examples if you google
these two APIs.

It is not as straightforward as LWP :)

-cheers,
Manish
 
T

Thierry Lam

Yes, it's the content of the jsp pages that I want to read. I usually
post on the newsgroup when I can't find anything from googling. If
anyone got any small sample working codes, let me know.
 
D

Daniel Pitts

Yes, it's the content of the jsp pages that I want to read. I usually
post on the newsgroup when I can't find anything from googling. If
anyone got any small sample working codes, let me know.

Something along the lines of:
new URL("http://serverone/bugs/download.jsp?
num=1234&file=Jul25.txt").getContent();

<http://java.sun.com/j2se/1.4.2/docs/api/java/net/URL.html>

At the very worst case, you will need to call getContentAsStream() and
then read in the content. although getContent() might to what you
need.
 
D

Daniel Pitts

You cannot read the "content" of the JSPs, but you can, however, read
the "output" of those JSPs as if your code is a browser, which is what
I think you meant by User Agent. You can either use Apache's
HttpClient API (my preference), or roll your own using the
java.net.URLConnection. You will find plenty of examples if you google
these two APIs.

It is not as straightforward as LWP :)
Actually, it is very straightforward. You don't need to mess with
URLConnection for retreiving the content of specific URLs, new
URL(urlString).getContent() should do the trick.
-cheers,
Manish

Daniel.
 
S

shakah

Yes, it's the content of the jsp pages that I want to read. I usually
post on the newsgroup when I can't find anything from googling. If
anyone got any small sample working codes, let me know.

Check out java.net.URLConnection, its getContent() method might be all
you need.

Below is a quick-and-dirty example of another way to use
URLConnection, though you'll have to catch the Exceptions to get it to
compile cleanly:

public StringBuffer fetch(String sURL) {
StringBuffer sbResponse = new StringBuffer(8192) ;

java.net.URL url = new java.net.URL(sURL) ;
java.net.URLConnection urlc = url.openConnection() ;
urlc.setDoInput(true) ;
urlc.setUseCaches(false) ;

java.io.InputStream is = urlc.getInputStream() ;
int nContentLength = urlc.getContentLength() ;
byte [] ab = new byte[nContentLength] ;
int nRead=0 ;
while(nRead < nContentLength) {
nRead += is.read(ab, nRead, nContentLength - nRead) ;
}
sbResponse.append(new String(ab, "utf-8")) ;
is.close();
is = null ;

((java.net.HttpURLConnection) urlc).disconnect() ;
urlc = null ;

return sbResponse ;
}
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top