Reading web pages from a standalone application

[moGio]

Hi all,
sorry for my english.

I don't have much experience on java so I don't know if what I want to
realize is possible or not.

I am trying to do a little standalone java application which reads web
pages, parse them and show some data.

Using URL.openConnection().connect() and some reading on it with an
InputStreamReader, I am able to read a standard web pages.

The problem is that the pages I need to read are protected: for
example, just think about a webmail sevice: first there is an
autentication form with the fields:
username
password

after submission you are free to navigate the pages (inbox, sent,
trash, options, ...).

I believe that when you login, username/password are stored in the
session active between your browser and the server.

In my application I am able to read the login page, find out the
user/password fields and the "action" of the form (the page following
the submit) but than I don't know what to do...

I tried to write on the connection those data and than I tried to load
the following page, but I'm always redirected to the login page: I
think I am not correctly using sessions or cookies in my application...

do anyone of you gurus have some suggestion? Maybe a link to some
tutorial? Is possible to realize my intentions with java?

I know I haven't been too much clear... my problem in summury is: I
can't read web pages from a standalone java application because I'm
always redirected to the login page.

tnx in advice

 

[Chris Smith]

Using URL.openConnection().connect() and some reading on it with an
InputStreamReader, I am able to read a standard web pages.

The problem is that the pages I need to read are protected: for
example, just think about a webmail sevice: first there is an
autentication form with the fields:
username
password

I tried to write on the connection those data and than I tried to load
the following page, but I'm always redirected to the login page: I
think I am not correctly using sessions or cookies in my application...

Yep. Cookies are actually a bit of a pain with URLConnection. Google
for Jakarta Commons HttpClient for a more complete HTTP client-side API
that includes easy cookie management.

 

[moGio]

Chris Smith ha scritto:

Yep. Cookies are actually a bit of a pain with URLConnection. Google
for Jakarta Commons HttpClient for a more complete HTTP client-side API
that includes easy cookie management.

Yes, I finally realized that the package java.net is not the best to
use to work with http,
I am studying the one you suggested (which seems to me so much
powerful) and also HttpUnit (on sourceforge).

thx