Post HTML forms and reading results

Discussion in 'Java' started by C. Kindle, May 16, 2004.

  1. C. Kindle

    C. Kindle Guest

    is it possible for a java application to enter values in a html form, post
    it, and reading the resulting html document, e.g. google.com? it seems that
    the other side has to be some kind of cgi-script or it doesn't work. i found
    an example at
    http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html,
    but trying it with google, or any other html form, it doesnt work.

    source:
    =====
    public void doPost(){
    String queryValue = URLEncoder.encode("java");
    try {
    URL url = new URL(http://www.google.com);
    URLConnection connection = url.openConnection();
    connection.setDoOutput(true);

    PrintWriter out = new PrintWriter(
    connection.getOutputStream());
    out.println("q=" + queryValue);
    out.close();

    BufferedReader in = new BufferedReader(
    new InputStreamReader(
    connection.getInputStream()));
    String inputLine;

    while ((inputLine = in.readLine()) != null)
    System.out.println(inputLine);

    in.close();
    }
    catch(Exception e) {
    //On exception output error message
    System.out.println("url connection error");
    }
    }
    C. Kindle, May 16, 2004
    #1
    1. Advertising

  2. On Sun, 16 May 2004 23:47:24 +0200, C. Kindle wrote:

    > is it possible for a java application to enter values in a html form, post
    > it, and reading the resulting html document, e.g. google.com?


    I hear that Google and such are very fussy
    about who they serve..

    (They have pre-designed Java API's
    for accessing search results.)

    > catch(Exception e) {
    > //On exception output error message


    // always advisable..
    e.printStackTrace();

    > System.out.println("url connection error");
    > }
    > }


    HTH

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
    Andrew Thompson, May 16, 2004
    #2
    1. Advertising

  3. C. Kindle

    Mike Smith Guest

    > but trying it with google, or any other html form, it doesnt work.

    Try:
    http://www.rgagnon.com/howto.html

    Under Java choose Networking, at the bottom of the list of code snippits is
    one called: "Fetch a page from Google". Basically, you just have to look
    like a browser when you connect. It's not rocket science.

    Mike
    Mike Smith, May 17, 2004
    #3
  4. C. Kindle

    Andy Fish Guest

    "C. Kindle" <> wrote in message
    news:c88nou$as0$-plus.net...
    > is it possible for a java application to enter values in a html form, post
    > it, and reading the resulting html document, e.g. google.com? it seems

    that
    > the other side has to be some kind of cgi-script or it doesn't work. i

    found
    > an example at
    >

    http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html,
    > but trying it with google, or any other html form, it doesnt work.
    >


    you may want to take a look at the apache commons httpclient. this is a
    reusable library containing much functionality related to HTTP.
    Andy Fish, May 17, 2004
    #4
  5. On Mon, 17 May 2004 05:15:05 GMT, Mike Smith wrote:

    >> but trying it with google, or any other html form, it doesnt work.


    (Snip URL that leads..)
    <http://www.rgagnon.com/javadetails/java-0399.html>

    > ...Basically, you just have to look
    > like a browser when you connect. It's not rocket science.


    Huh! Another great example..
    Wish I had thought to hunt around the
    "Real's How-To" at the time.. :)

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
    Andrew Thompson, May 17, 2004
    #5
  6. > On Mon, 17 May 2004 05:15:05 GMT, Mike Smith wrote:

    C.K. wrote *actually* (but implied attribution to Mike)
    >>> but trying it with google, or any other html form, it doesnt work.


    (Mike Smith *replied*..)
    > (Snip URL that leads..)
    > <http://www.rgagnon.com/javadetails/java-0399.html>


    Oops!
    Andrew Thompson, May 17, 2004
    #6
  7. C. Kindle

    Roedy Green Guest

    On Sun, 16 May 2004 23:47:24 +0200, "C. Kindle" <> wrote
    or quoted :

    >is it possible for a java application to enter values in a html form, post
    >it, and reading the resulting html document, e.g. google.com? it seems that
    >the other side has to be some kind of cgi-script or it doesn't work. i found
    >an example at
    >http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html,
    >but trying it with google, or any other html form, it doesnt work.


    see http://mindprod.com/jgloss/cgi.html
    and http://mindprod.com/jgloss/fileio.html
    for sample code.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
    Roedy Green, May 17, 2004
    #7
  8. C. Kindle

    C. Kindle Guest

    Re: simulate a html form submission

    thank you for your answers. by querying a page with the url of the resulting
    page, search parameters and their values it works

    what i want is to simulate a html form submission from java, since some
    query forms require the query to be executed by their 'original' query form.
    what i found doesn't seem to work.


    "Mike Smith" <> schrieb im Newsbeitrag
    news:tTXpc.129$...
    > > but trying it with google, or any other html form, it doesnt work.

    >
    > Try:
    > http://www.rgagnon.com/howto.html
    >
    > Under Java choose Networking, at the bottom of the list of code snippits

    is
    > one called: "Fetch a page from Google". Basically, you just have to look
    > like a browser when you connect. It's not rocket science.
    >
    > Mike
    >
    >
    C. Kindle, May 17, 2004
    #8
  9. Re: simulate a html form submission

    On Mon, 17 May 2004 09:38:43 +0200, C. Kindle wrote:

    Apparently re this thread..
    <http://google.com/groups?th=c8303b0abcbc08fc>
    Please do not change the subject line of a thread
    or strat a new thread unless it is a different subject.

    Even thenit is best to find the first at Google
    groups and link to it.

    Further comments to bottom..

    > "Mike Smith" <> schrieb im Newsbeitrag

    (C.K.)
    >>> but trying it with google, or any other html form, it doesnt work.

    ...
    >> http://www.rgagnon.com/howto.html
    >>
    >> Under Java choose Networking, at the bottom of the list of code snippits is
    >> one called: "Fetch a page from Google". Basically, you just have to look
    >> like a browser when you connect. It's not rocket science.


    <top-post corrected>
    Please do not top-post either C., as
    it breaks the thread of the ..thread.
    <http://www.physci.org/codes/javafaq.jsp#netiquette>

    > thank you for your answers. by querying a page with the url of the resulting
    > page, search parameters and their values it works
    >
    > what i want is to simulate a html form submission from java, since some
    > query forms require the query to be executed by their 'original' query form.
    > what i found doesn't seem to work.


    Now.. you may have to put a little more
    effort and thought into your statements..

    I have never seen an..
    "DoesNotSeemToWorkException" ;-)
    <http://www.physci.org/codes/javafaq.jsp#exact>

    Also, you geve a great little (the best kind)
    example whrn you 1st posted, could you show us
    _exactly_ what you have now?
    <http://www.physci.org/codes/sscce.jsp>

    Also, I am not sure how much experience you have
    with debuggin, but it seems this may be a better
    group for the moment..
    <http://www.physci.org/codes/javafaq.jsp#cljh>

    HTH

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
    Andrew Thompson, May 17, 2004
    #9
  10. C. Kindle

    Chris Smith Guest

    Re: simulate a html form submission

    C. Kindle wrote:
    > thank you for your answers. by querying a page with the url of the resulting
    > page, search parameters and their values it works
    >
    > what i want is to simulate a html form submission from java, since some
    > query forms require the query to be executed by their 'original' query form.
    > what i found doesn't seem to work.


    Unfortunately, you haven't said much that could give us a clue what
    isn't working.

    One guess is that in your original code sample, you specified the URL as
    "http://www.google.com". That's wrong; you need to specify the URL that
    the form should be submitting to, not the URL that produces the form to
    begin with. In Google's case, that's "http://www.google.com/search".

    If you don't know that for your specific form (and don't want to collect
    it ahead of time for some reason), then you're left with the regrettable
    task of making *two* HTTP requests... the first will return the form,
    and you need to parse it looking for the 'action' attribute of the form
    tag. After you've got that, you can then submit the real form request.

    In the latter case, you might actually want to look into some unit
    testing tools like Canoo WebTest, which are decent at letting you script
    interaction with web sites.

    --
    www.designacourse.com
    The Easiest Way to Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
    Chris Smith, May 17, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Monique Y. Mudama
    Replies:
    1
    Views:
    440
    Monique Y. Mudama
    Jun 28, 2005
  2. wmburns
    Replies:
    5
    Views:
    681
    wayne
    Feb 12, 2006
  3. Hal Vaughan
    Replies:
    4
    Views:
    771
    Hal Vaughan
    Jan 14, 2008
  4. Eric
    Replies:
    2
    Views:
    451
  5. Don Glover the younger

    ASP, FORMS, POST METHOD And Post with out form(???)

    Don Glover the younger, Jul 13, 2003, in forum: ASP General
    Replies:
    0
    Views:
    429
    Don Glover the younger
    Jul 13, 2003
Loading...

Share This Page