post request + read resulting data

[Angus Parvis]

Hi,

I want to write a class that sends a HTTP post request to
http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
the file returned as a result.

I know how to send a post request, but fail to read the .DAT-file
returend as the result. Only thing i read is the HTML page I'm posting
the data to.

Here's my code:

--

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.iutputStreamWriter;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLEncoder;

public class Dummy {
public static void main(String args[]) throws Exception {
String data = URLEncoder.encode("txtTicker", "UTF-8") + "="
+ URLEncoder.encode("AA", "UTF-8");
data += URLEncoder.encode("cmdSubmit", "UTF-8") + "="
+ URLEncoder.encode("Download", "UTF-8");
// Send data
URL url = new URL(

"http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new
OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();

BufferedReader rd = new BufferedReader(new InputStreamReader(conn
.getInputStream()));

String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}

rd.close();
}
}

--

Any idea how to solve the problem? What do I have to change to read the
..DAT-file returned?

Thanks for your time and help,

Angus

 

[Chris Smith]

I want to write a class that sends a HTTP post request to
http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
the file returned as a result.

I know how to send a post request, but fail to read the .DAT-file
returend as the result. Only thing i read is the HTML page I'm posting
the data to.

There's some weird JavaScript in the page at that URL. One thing I
noticed is that the JavaScript actually changes the "action" attribute
of the form at some point. You probably need to decode that and figure
out the right place to submit the form.

Incidentally, it appears that someone did this specifically to prevent
people from doing what you're doing. By continuing, you are probably
picking a fight with the maintainer of this web site. Are you sure you
are on solid legal ground?

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Chris]

Hi,

I want to write a class that sends a HTTP post request to
http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
the file returned as a result.

I know how to send a post request, but fail to read the .DAT-file
returend as the result. Only thing i read is the HTML page I'm posting
the data to.

Your post request is probably in the wrong format. What if you encode your
parameters as a GET? Just append them to the URL and see what happens.

 

[Angus Parvis]

There's some weird JavaScript in the page at that URL. One thing I
noticed is that the JavaScript actually changes the "action" attribute
of the form at some point. You probably need to decode that and figure
out the right place to submit the form.

When I turn off JavaScript in my Mozilla Navigator, this page still
works as before. I think the JavaScript doesn't affect the form.

Incidentally, it appears that someone did this specifically to prevent
people from doing what you're doing. By continuing, you are probably
picking a fight with the maintainer of this web site. Are you sure you
are on solid legal ground?

A friend of mine asked me to do this for him, I'm just curious how to
solve the problem.

 

[Andrea Desole]

Hi,

I want to write a class that sends a HTTP post request to
http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
the file returned as a result.

I know how to send a post request, but fail to read the .DAT-file
returend as the result. Only thing i read is the HTML page I'm posting
the data to.

Are you sure the dat file is really returned? Maybe you are missing some
parameters in your request, and the application just sends you back to
the page

 

[Chris Uppal]

Incidentally, it appears that someone did this specifically to prevent
people from doing what you're doing. By continuing, you are probably
picking a fight with the maintainer of this web site.

As in, (reformatted quote from that page):

<strong>
PLEASE NOTE: IT IS STRICTLY PROHIBITED TO DOWNLOAD DELAYED QUOTE TABLE DATA
FROM THIS WEB SITE BY USING AUTO-EXTRACTION PROGRAMS/QUERIES AND/OR SOFTWARE.
CBOE WILL BLOCK IP ADDRESSES OF ALL PARTIES WHO ATTEMPT TO DO SO. THIS DATA IS
PROPERTY OF THOMSON FINANCIAL/ILX. DOWNLOADING THIS DATA IN ANY OTHER WAY THAN
BY MANUAL TICKER SYMBOL ENTRY IS STRICTLY PROHIBITED.
</strong>

You could hardly ask for a clearer statement.

-- chris