URL Processing

[Mike Barnes]

I am trying to build a search tool that will go against several
different Job posting sites. I started with Monster.com and immediately
ran into a problem in which I am unable to retrieve the results of my
query. It appears that the site is generating the data using javascript.
Besides the URLConnection what do I need to do to get the data that I am
looking for. Below is the code that I am using currently.

Thanks in advance for any help or tips that may be offered.

/*
* Created on Apr 19, 2004
*
* TODO To change the template for this generated file go to
* Window - Preferences - Java - Code Generation - Code and Comments
*/
/**
* @author Shelton Johnson
*
* TODO To change the template for this generated type comment go to
* Window - Preferences - Java - Code Generation - Code and Comments
*/
//------------------------------------------------------------//
//DnldURL.java: //
//------------------------------------------------------------//
//A Java program that demonstrates a procedure that can be //
//used to download the contents of a specified URL. //
//------------------------------------------------------------//
//Code created by Developer's Daily //
//http://www.DevDaily.com //
//------------------------------------------------------------//
import java.io.BufferedInputStream;
import java.io.DataInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;

public class DnldURL {
public static void main(String[] args) {
//-----------------------------------------------------//
// Step 1: Start creating a few objects we'll need.
//-----------------------------------------------------//
URL u;
InputStream is = null;
DataInputStream dis;
String s;
try {
//------------------------------------------------------------//
// Step 2: Create the URL. //
//------------------------------------------------------------//
// Note: Put your real URL here, or better yet, read it as a //
// command-line arg, or read it from a file. //
//------------------------------------------------------------//
u = new URL(

"http://jobsearch.monster.com/jobsearch.asp?q=java&lid=950&brd=1.1862,1863&cy=US&re=14&rv=true");
//----------------------------------------------//
// Step 3: Open an input stream from the url. //
//----------------------------------------------//
is = u.openStream(); // throws an IOException
//-------------------------------------------------------------//
// Step 4: //
//-------------------------------------------------------------//
// Convert the InputStream to a buffered DataInputStream. //
// Buffering the stream makes the reading faster; the //
// readLine() method of the DataInputStream makes the reading //
// easier. //
//-------------------------------------------------------------//
dis = new DataInputStream(new BufferedInputStream(is));
//------------------------------------------------------------//
// Step 5: //
//------------------------------------------------------------//
// Now just read each record of the input stream, and print //
// it out. Note that it's assumed that this problem is run //
// from a command-line, not from an application or applet. //
//------------------------------------------------------------//
while ((s = dis.readLine()) != null) {
System.out.println(s);
}
} catch (MalformedURLException mue) {
System.out.println("Ouch - a MalformedURLException happened.");
mue.printStackTrace();
System.exit(1);
} catch (IOException ioe) {
System.out.println("Oops- an IOException happened.");
ioe.printStackTrace();
System.exit(1);
} finally {
//---------------------------------//
// Step 6: Close the InputStream //
//---------------------------------//
try {
is.close();
} catch (IOException ioe) {
// just going to ignore this one
}
} // end of 'finally' clause
} // end of main
} // end of class definition