trouble figuring out HttpURLConnection

[Flip]

I'm trying to post data to a webpage, then get the next page, but I can't
seem to get that to work. I'm trying to follow examples on a couple of
webpages, but I'm always getting back the same html as I started with. Can
anyone see what I'm doing wrong? Thanks.

Code snippet-------------------------------------------------------------
package com.pch.serverconnection;
import java.net.*;
import java.io.*;

public class GoodURLPost {
public GoodURLPost() {
try {
StringBuffer sb = new StringBuffer();
URL url = new URL( "http://www.domainnamehere.com/default.jsp" );
String body = "/servlet/DoSearch?aSearchString=searchforsomestring";
// find the newline character(s) on the current system
String newline = null;
try {
newline = System.getProperty("line.separator");
} catch (Exception e) {
newline = "\n";
}

// URL must use the http protocol!
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setAllowUserInteraction(false); // you may not ask the user
conn.setDoOutput(true); // we want to send things
// the Content-type should be default, but we set it anyway
//conn.setRequestProperty( "Content-type", "text/html" );
// the content-length should not be necessary, but we're cautious
conn.setRequestProperty( "Content-length",
Integer.toString(body.length()));

// get the output stream to POST our form data
OutputStream rawOutStream = conn.getOutputStream();
PrintWriter pw = new PrintWriter(rawOutStream);

pw.print(body); // here we "send" our body!
pw.flush();
pw.close();

// get the input stream for reading the reply
// IMPORTANT! Your body will not get transmitted if you get the
// InputStream before completely writing out your output first!
InputStream rawInStream = conn.getInputStream();

// get response
BufferedReader rdr = new BufferedReader(new
InputStreamReader(rawInStream));
String line;

while ((line = rdr.readLine()) != null) {
sb.append(line + newline );
}
System.out.println( sb.toString() );
} catch (Exception e) {
System.out.println("Exception "+e.toString());
e.printStackTrace();
}
}
public static void main(String[] args) {
GoodURLPost goodURLPost1 = new GoodURLPost();
}

}

 

[Flip]

you might wanna check the setInstanceFollowRedirects option, and the use

of
Thanks for the heads up on the setInstanceFollowRedirects.

I used the same code to make out this utility class, which you're welcome

to use if you want.

Doh. The attachment didn't come through. Would it be possible to attach
the code here?

 

[R@nsh!]

here's the code as text:
btw - I guess there are several "bad" getter/setters methods.
Your comments are appreciated
/*

* Created on Oct 30, 2003

*/

package com.qfish.web;

import java.io.*;

import java.net.*;

import java.util.ArrayList;

import java.util.Iterator;

/**

* @author R@nsh!

*

*/

public class Form implements WebConstants

{

private HttpURLConnection conn;

private ArrayList cookies = new ArrayList();

private String urlStr;

private String queryString;

private String redirectUrl;

private String method;

private StringBuffer responseHtml;


private static final boolean DEBUG = Boolean.getBoolean("debug");

/**

*

* @param url the full URL of the form. must include http/s protocl

* @param encodedQueryString

* @param method either POST or GET

*/

public Form(String url, String encodedQueryString, String method)

{

this(url, encodedQueryString, method, false);

}

/**

*

* @param url the full URL of the form. must include http/s protocl

* @param encodedQueryString

* @param method either POST or GET

* @param secure

*/

public Form(String url, String encodedQueryString, String method, boolean
secure)

{

this.urlStr = url;

queryString = encodedQueryString;

this.method = method;

}

public Form submit()

{

if (method.equalsIgnoreCase(post))

{

submitPost();

}

return this;

}

/**

* Submits the form using the POST method

*/

private void submitPost()

{

try

{

debug(queryString);

debug(urlStr);

URL url = new URL(urlStr);

conn = (HttpURLConnection) url.openConnection();

setCookiesToConnection();

conn.setInstanceFollowRedirects(false);

//allows to stop and retrieve the cookies

conn.setRequestMethod(post);

conn.setAllowUserInteraction(false); // you may not ask the user

conn.setDoOutput(true); // we want to send things

// the Content-type should be default, but we set it anyway

conn.setRequestProperty("Content-type",
"application/x-www-form-urlencoded");

// the content-length should not be necessary, but we're cautious

conn.setRequestProperty("Content-length",
Integer.toString(queryString.length()));

// get the output stream to POST our form data

// IMPORTANT! Your body will not get transmitted if you get the

// InputStream before completely writing out your output first!

OutputStream rawOutStream = conn.getOutputStream();

PrintWriter pw = new PrintWriter(rawOutStream);

pw.print(queryString); // here we "send" our body!

pw.flush();

pw.close();

parseHeaders();

readResponse();

}

catch (MalformedURLException e)

{

// TODO Auto-generated catch block

e.printStackTrace();

}

catch (ProtocolException e)

{

// TODO Auto-generated catch block

e.printStackTrace();

}

catch (IOException e)

{

// TODO Auto-generated catch block

e.printStackTrace();

}

}

private void parseHeaders()

{

int n = 1;

// n=0 has no key, and the HTTP return status in the value field

boolean done = false;

while (!done)

{

String headerKey = conn.getHeaderFieldKey(n);

String headerVal = conn.getHeaderField(n);

if (headerKey != null || headerVal != null)

{

System.out.println(headerKey + "=" + headerVal);

if (headerKey.equalsIgnoreCase("set-cookie"))

{

cookies.add(headerVal);

}

else if (headerKey.equalsIgnoreCase("Location"))

{

redirectUrl = headerVal;

}

}

else

{

done = true;

}

n++;

}

System.out.println(conn.getHeaderField(0));

}

/**

* Sets existing cookies (if any) to the current
<code>HttpURLConnection</code> object

*/

private void setCookiesToConnection()

{

Iterator itr = cookies.iterator();

while (itr.hasNext())

{

conn.addRequestProperty("Cookie", (String) itr.next());

}

}

/**

* Reads the server's response to a <code>StringBuffer</code>, available

* via getResponseHtml()

* @see getResponseHtml()

*/

private void readResponse()

{

String newline = System.getProperty("line.separator");

try

{

responseHtml = new StringBuffer();

// find the newline character(s) on the current system

// get the input stream for reading the reply & print the response

InputStream rawInStream = conn.getInputStream();

// get response

BufferedReader rdr = new BufferedReader(new InputStreamReader(rawInStream));

String line;

while ((line = rdr.readLine()) != null)

{

responseHtml.append(line);

responseHtml.append(newline);

}

}

catch (IOException e)

{

responseHtml.append("An error has occured reading the server's response.");

responseHtml.append(newline);

responseHtml.append(e.getStackTrace().toString());

e.printStackTrace();

}

}

/**

* @return the connection object

*/

public HttpURLConnection getConn()

{

return conn;

}

/**

* @return

*/

public ArrayList getCookies()

{

return cookies;

}

/**

* @return

*/

public String getMethod()

{

return method;

}

/**

* @return

*/

public String getQueryString()

{

return queryString;

}

/**

* @return

*/

public String getRedirectUrl()

{

return redirectUrl;

}

/**

* @return

*/

public StringBuffer getResponseHtml()

{

return responseHtml;

}

/**

* @return

*/

public String getUrlStr()

{

return urlStr;

}

/**

* @param connection

*/

public void setConn(HttpURLConnection connection)

{

conn = connection;

}

/**

* @param list

*/

public void setCookies(ArrayList list)

{

cookies = list;

}

/**

* @param string

*/

public void setMethod(String string)

{

method = string;

}

/**

* @param string

*/

public void setQueryString(String string)

{

try

{

queryString = URLEncoder.encode(string, utf8);

}

catch (UnsupportedEncodingException e)

{

//utf-8 is supported by default, so this shouldn't really happen

e.printStackTrace();

}

}

/**

* @param string

*/

public void setRedirectUrl(String string)

{

redirectUrl = string;

}

/**

* @param string

*/

public void setUrlStr(String string)

{

urlStr = string;

}


private void debug(String msg)

{

if (DEBUG)

{

System.err.println(msg);

}

}

}

 

[Flip]

I'm trying this at home tonight, but unfortunately am getting some problems
(can't find WebConstants, there's a utf8 that's causing me some issues).
Maybe you could email it to me at (e-mail address removed)? Much appreciated.
Thanks.

 

[Flip]

Oh, don't mind me. I just commented out the WebConstants and put the post
and utf8 to strings and things are good to compile. I'll see what happens
when I try to run it. Thanks for your help with this. I'll let you know how
it goes! :>

 

[Flip]

I noticed you set the setInstanceFollowRedirects to false, why is that?
Could I change that to true?

 

[Flip]

I noticed you set the setInstanceFollowRedirects to false, why is that?
Could I change that to true?

 

[Flip]

I noticed you set the setInstanceFollowRedirects to false, why is that?
Could I change that to true?

 

[Flip]

I keep getting a null pointer exception. :< After the constructor what
method do you run? Maybe it's late for me to be doing this, I'm heading to
bed and will try again tomorrow night. Thanks.