Post HTML forms and reading results

[C. Kindle]

is it possible for a java application to enter values in a html form, post
it, and reading the resulting html document, e.g. google.com? it seems that
the other side has to be some kind of cgi-script or it doesn't work. i found
an example at
http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html,
but trying it with google, or any other html form, it doesnt work.

source:
=====
public void doPost(){
String queryValue = URLEncoder.encode("java");
try {
URL url = new URL(http://www.google.com);
URLConnection connection = url.openConnection();
connection.setDoOutput(true);

PrintWriter out = new PrintWriter(
connection.getOutputStream());
out.println("q=" + queryValue);
out.close();

BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
String inputLine;

while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);

in.close();
}
catch(Exception e) {
//On exception output error message
System.out.println("url connection error");
}
}

 

[Andrew Thompson]

is it possible for a java application to enter values in a html form, post
it, and reading the resulting html document, e.g. google.com?

I hear that Google and such are very fussy
about who they serve..

(They have pre-designed Java API's
for accessing search results.)

catch(Exception e) {
//On exception output error message

// always advisable..
e.printStackTrace();

System.out.println("url connection error");
}
}

HTH

 

[Mike Smith]

but trying it with google, or any other html form, it doesnt work.

Try:
http://www.rgagnon.com/howto.html

Under Java choose Networking, at the bottom of the list of code snippits is
one called: "Fetch a page from Google". Basically, you just have to look
like a browser when you connect. It's not rocket science.

Mike

 

[Andy Fish]

is it possible for a java application to enter values in a html form, post
it, and reading the resulting html document, e.g. google.com? it seems that
the other side has to be some kind of cgi-script or it doesn't work. i found
an example at
http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html,
but trying it with google, or any other html form, it doesnt work.

you may want to take a look at the apache commons httpclient. this is a
reusable library containing much functionality related to HTTP.

 

[Andrew Thompson]

(Snip URL that leads..)

...Basically, you just have to look
like a browser when you connect. It's not rocket science.

Huh! Another great example..
Wish I had thought to hunt around the
"Real's How-To" at the time..

 

[Andrew Thompson]

On Mon, 17 May 2004 05:15:05 GMT, Mike Smith wrote:

C.K. wrote *actually* (but implied attribution to Mike)
(Mike Smith *replied*..)

(Snip URL that leads..)
<http://www.rgagnon.com/javadetails/java-0399.html>

Oops!

 

[Roedy Green]

is it possible for a java application to enter values in a html form, post
it, and reading the resulting html document, e.g. google.com? it seems that
the other side has to be some kind of cgi-script or it doesn't work. i found
an example at
http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html,
but trying it with google, or any other html form, it doesnt work.

see http://mindprod.com/jgloss/cgi.html
and http://mindprod.com/jgloss/fileio.html
for sample code.

 

[C. Kindle]

thank you for your answers. by querying a page with the url of the resulting
page, search parameters and their values it works

what i want is to simulate a html form submission from java, since some
query forms require the query to be executed by their 'original' query form.
what i found doesn't seem to work.

 

[Andrew Thompson]

On Mon, 17 May 2004 09:38:43 +0200, C. Kindle wrote:

Apparently re this thread..
<http://google.com/groups?th=c8303b0abcbc08fc>
Please do not change the subject line of a thread
or strat a new thread unless it is a different subject.

Even thenit is best to find the first at Google
groups and link to it.

Further comments to bottom..

<top-post corrected>
Please do not top-post either C., as
it breaks the thread of the ..thread.

thank you for your answers. by querying a page with the url of the resulting
page, search parameters and their values it works

what i want is to simulate a html form submission from java, since some
query forms require the query to be executed by their 'original' query form.
what i found doesn't seem to work.

Now.. you may have to put a little more
effort and thought into your statements..

I have never seen an..
"DoesNotSeemToWorkException" ;-)
<http://www.physci.org/codes/javafaq.jsp#exact>

Also, you geve a great little (the best kind)
example whrn you 1st posted, could you show us
_exactly_ what you have now?
<http://www.physci.org/codes/sscce.jsp>

Also, I am not sure how much experience you have
with debuggin, but it seems this may be a better
group for the moment..
<http://www.physci.org/codes/javafaq.jsp#cljh>

HTH

 

[Chris Smith]

thank you for your answers. by querying a page with the url of the resulting
page, search parameters and their values it works

what i want is to simulate a html form submission from java, since some
query forms require the query to be executed by their 'original' query form.
what i found doesn't seem to work.

Unfortunately, you haven't said much that could give us a clue what
isn't working.

One guess is that in your original code sample, you specified the URL as
"http://www.google.com". That's wrong; you need to specify the URL that
the form should be submitting to, not the URL that produces the form to
begin with. In Google's case, that's "http://www.google.com/search".

If you don't know that for your specific form (and don't want to collect
it ahead of time for some reason), then you're left with the regrettable
task of making *two* HTTP requests... the first will return the form,
and you need to parse it looking for the 'action' attribute of the form
tag. After you've got that, you can then submit the real form request.

In the latter case, you might actually want to look into some unit
testing tools like Canoo WebTest, which are decent at letting you script
interaction with web sites.

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation