gettin all pages in a website

[kkrish]

hi all,
Is it possible to read all the html pages in website, given
the web site address?If so how to get the pages.Should we search for
all the "href" links , will it be sufficient.I am new to java jsp.
Thanks in advance.
Krishna.V.J

 

[Ingo R. Homann]

Hi,

hi all,
Is it possible to read all the html pages in website, given
the web site address?If so how to get the pages.Should we search for
all the "href" links , will it be sufficient.I am new to java jsp.
Thanks in advance.
Krishna.V.J

There are several (versions of a) program(s) called "wget" which do
exactly that. (Note that this has nothing to do with java...)

Ciao,
Ingo

 

[kkrish]

Hi,
Thanks.Is it impossible to do capturing website pages in java?If
possible how to proceed.

 

[Philipp Leitner]

Hi,
Thanks.Is it impossible to do capturing website pages in java?If
possible how to proceed.

Of course it is not /impossible/ to do, but I don't know of any
standard library to do it (what means that you would have to implement
the functionality yourself) - that may be a little annoying (depending
on your project size), but not a huge problem I guess.

Just one sidenote: searching for '<a href' will generally not be enough
- there are also plenty of other redirects out there (JavaScript for
example).

/philipp

 

[TechBookReport]

Hi,
Thanks.Is it impossible to do capturing website pages in java?If
possible how to proceed.

You might find the following article useful:
http://www.developer.com/java/other/article.php/1573761

 

[Oliver Wong]

Just one sidenote: searching for '<a href' will generally not be enough
- there are also plenty of other redirects out there (JavaScript for
example).

There may also be "secret" pages that aren't linked to anywhere else.
There may be password protected webpages. There may be dynamically generated
web pages which depend upon the IP address of the request (for example, if
the request comes from 127.0.0.1, the "super-administrator" page is shown,
otherwise the "normal user" page is shown). With dynamically generated web
pages, there could be infinitely many pages. So in general, no, it's not
possible to get all pages in a website.

- Oliver