post request + read resulting data

Discussion in 'Java' started by Angus Parvis, Mar 23, 2005.

  1. Angus Parvis

    Angus Parvis Guest

    Hi,

    I want to write a class that sends a HTTP post request to
    http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
    the file returned as a result.

    I know how to send a post request, but fail to read the .DAT-file
    returend as the result. Only thing i read is the HTML page I'm posting
    the data to.

    Here's my code:

    --

    import java.io.BufferedReader;
    import java.io.InputStreamReader;
    import java.io_OutputStreamWriter;
    import java.net.URL;
    import java.net.URLConnection;
    import java.net.URLEncoder;

    public class Dummy {
    public static void main(String args[]) throws Exception {
    String data = URLEncoder.encode("txtTicker", "UTF-8") + "="
    + URLEncoder.encode("AA", "UTF-8");
    data += URLEncoder.encode("cmdSubmit", "UTF-8") + "="
    + URLEncoder.encode("Download", "UTF-8");
    // Send data
    URL url = new URL(

    "http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx");
    URLConnection conn = url.openConnection();
    conn.setDoOutput(true);
    OutputStreamWriter wr = new
    OutputStreamWriter(conn.getOutputStream());
    wr.write(data);
    wr.flush();
    wr.close();

    BufferedReader rd = new BufferedReader(new InputStreamReader(conn
    .getInputStream()));

    String line;
    while ((line = rd.readLine()) != null) {
    System.out.println(line);
    }

    rd.close();
    }
    }

    --

    Any idea how to solve the problem? What do I have to change to read the
    ..DAT-file returned?

    Thanks for your time and help,

    Angus
     
    Angus Parvis, Mar 23, 2005
    #1
    1. Advertising

  2. Angus Parvis

    Chris Smith Guest

    Angus Parvis <> wrote:
    > I want to write a class that sends a HTTP post request to
    > http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
    > the file returned as a result.
    >
    > I know how to send a post request, but fail to read the .DAT-file
    > returend as the result. Only thing i read is the HTML page I'm posting
    > the data to.


    There's some weird JavaScript in the page at that URL. One thing I
    noticed is that the JavaScript actually changes the "action" attribute
    of the form at some point. You probably need to decode that and figure
    out the right place to submit the form.

    Incidentally, it appears that someone did this specifically to prevent
    people from doing what you're doing. By continuing, you are probably
    picking a fight with the maintainer of this web site. Are you sure you
    are on solid legal ground?

    --
    www.designacourse.com
    The Easiest Way To Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
     
    Chris Smith, Mar 23, 2005
    #2
    1. Advertising

  3. Angus Parvis

    Chris Guest

    "Angus Parvis" <> wrote in message
    news:M530e.9778$...
    > Hi,
    >
    > I want to write a class that sends a HTTP post request to
    > http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
    > the file returned as a result.
    >
    > I know how to send a post request, but fail to read the .DAT-file
    > returend as the result. Only thing i read is the HTML page I'm posting
    > the data to.


    Your post request is probably in the wrong format. What if you encode your
    parameters as a GET? Just append them to the URL and see what happens.
     
    Chris, Mar 23, 2005
    #3
  4. Angus Parvis

    Angus Parvis Guest

    Chris Smith wrote:

    > There's some weird JavaScript in the page at that URL. One thing I
    > noticed is that the JavaScript actually changes the "action" attribute
    > of the form at some point. You probably need to decode that and figure
    > out the right place to submit the form.


    When I turn off JavaScript in my Mozilla Navigator, this page still
    works as before. I think the JavaScript doesn't affect the form.

    > Incidentally, it appears that someone did this specifically to prevent
    > people from doing what you're doing. By continuing, you are probably
    > picking a fight with the maintainer of this web site. Are you sure you
    > are on solid legal ground?


    A friend of mine asked me to do this for him, I'm just curious how to
    solve the problem.
     
    Angus Parvis, Mar 23, 2005
    #4
  5. Angus Parvis wrote:
    > Hi,
    >
    > I want to write a class that sends a HTTP post request to
    > http://www.cboe.com/DelayedQuote/QuoteTableDownload.aspx and downloads
    > the file returned as a result.
    >
    > I know how to send a post request, but fail to read the .DAT-file
    > returend as the result. Only thing i read is the HTML page I'm posting
    > the data to.


    Are you sure the dat file is really returned? Maybe you are missing some
    parameters in your request, and the application just sends you back to
    the page
     
    Andrea Desole, Mar 23, 2005
    #5
  6. Angus Parvis

    Chris Uppal Guest

    Chris Smith wrote:

    > Incidentally, it appears that someone did this specifically to prevent
    > people from doing what you're doing. By continuing, you are probably
    > picking a fight with the maintainer of this web site.


    As in, (reformatted quote from that page):

    <strong>
    PLEASE NOTE: IT IS STRICTLY PROHIBITED TO DOWNLOAD DELAYED QUOTE TABLE DATA
    FROM THIS WEB SITE BY USING AUTO-EXTRACTION PROGRAMS/QUERIES AND/OR SOFTWARE.
    CBOE WILL BLOCK IP ADDRESSES OF ALL PARTIES WHO ATTEMPT TO DO SO. THIS DATA IS
    PROPERTY OF THOMSON FINANCIAL/ILX. DOWNLOADING THIS DATA IN ANY OTHER WAY THAN
    BY MANUAL TICKER SYMBOL ENTRY IS STRICTLY PROHIBITED.
    </strong>

    You could hardly ask for a clearer statement.

    -- chris
     
    Chris Uppal, Mar 23, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bjorn Jensen
    Replies:
    2
    Views:
    829
    Bjorn Jensen
    Mar 22, 2005
  2. James Wallace
    Replies:
    0
    Views:
    1,060
    James Wallace
    Oct 17, 2003
  3. Anatoli Trifonov

    ASP Control names in resulting HTML

    Anatoli Trifonov, Jun 6, 2004, in forum: ASP .Net
    Replies:
    5
    Views:
    347
    John Saunders
    Jun 7, 2004
  4. Dil via .NET 247

    SHA1 length of resulting hash

    Dil via .NET 247, Aug 4, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    17,961
    Dil via .NET 247
    Aug 4, 2004
  5. Bart Vandewoestyne
    Replies:
    1
    Views:
    293
    Ian Collins
    Oct 23, 2012
Loading...

Share This Page