Connecting to dynamically generated data

Discussion in 'Java' started by Derek, Aug 19, 2005.

  1. Derek

    Derek Guest

    Hello all, this is my first time posting to a Usenet group, so please
    forgive me for my "green-ness." Anyway, my problem is this: I need to
    connect to a website that provides current and historical weather data
    and does so through a web interface. To get the information you pass it
    the type of data you want and a few constraints and the server
    retrieves the data in plain text.

    Here's an example:
    http://nomads.ncdc.noaa.gov:9091/dods/NCEP_GFS/200506/20050616/gfs_3_20050616_0600_fff.asc?tmp[0:0][0:6:25][0:180][0:359]

    However, if I try to connect to the page with the following,

    URL url = new
    URL("http://nomads.ncdc.noaa.gov:9091/dods/NCEP_GFS/200506/20050616/gfs_3_20050616_0600_fff.ascii?tmp[0:0][0:6:25][0:180][0:359]");
    InputStream stream = url.openStream();

    I get the following Exception:
    java.net.SocketException: Unexpected end of file from server
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown
    Source)
    at java.net.URL.openStream(Unknown Source)

    Now, I've already set the proxy so that it can connect to outside web
    pages (I've already tried it out on more mundane pages with no
    problems), so I'm not sure what else I need to do to be able to connect
    to it to retrieve the data. Can anyone help me out? Many thanks!

    -Derek
    Derek, Aug 19, 2005
    #1
    1. Advertising

  2. Derek

    Derek Guest

    Ugh, links got all screwed up. Try copying this:
    nomads.ncdc.noaa.gov:9091/dods/NCEP_GFS/200506/20050616/gfs_3_20050616_0600_fff.asc?tmp[0:0][0:6:25][0:180][0:359]
    Derek, Aug 19, 2005
    #2
    1. Advertising

  3. On 19 Aug 2005 11:34:38 -0700, Derek wrote:

    > Hello all, this is my first time posting to a Usenet group, so please
    > forgive me for my "green-ness."


    Cool. First tip. Please repeat the key words of the subject
    in the body of the post, as some people ignore
    subject lines..

    Sub: Connecting to dynamically generated data

    'Dynamically generated', eh..? That could be the problem
    (though I'm not sure).

    >..Anyway, my problem is this: I need to
    > connect to a website


    Which one?

    > http://nomads.ncdc.noaa.gov:9091/dods/NCEP_GFS/200506/20050616/gfs_3_20050616_0600_fff.asc?tmp[0:0][0:6:25][0:180][0:359]


    Oh right.. <http://www.noaa.gov/>
    The weather folks, they have some interesting data.
    They also seem relatively Java friendly..
    <http://shovel.hpcc.noaa.gov/cgi-bin/htsearch?config=htdig&restrict=&exclude=&method=and&format=builtin-long&sort=score&words=java>

    Are you sure they do not already provide an API for providing
    this data to Java processes?

    > However, if I try to connect to the page with the following,
    >
    > URL url = new
    > URL("http://nomads.ncdc.noaa.gov:9091/dods/NCEP_GFS/200506/20050616/gfs_3_20050616_0600_fff.ascii?tmp[0:0][0:6:25][0:180][0:359]");
    > InputStream stream = url.openStream();
    >
    > I get the following Exception:
    > java.net.SocketException: Unexpected end of file from server


    I did a quick check here to the point of opening an input stream,
    and got no errors. Are you sure the error indicated that *exact* line?

    > ...so I'm not sure what else I need to do to be able to connect
    > to it to retrieve the data.


    I strongly recommend getting in touch with the noaa support, I
    suspect that
    a) the 'dynamic' nature of the information feed is the problem.
    b) if they are OK with direct programmatic access, they will
    either point you to the API designed to access it, or maybe point
    you to the little FAQ that shows you the trick.

    HTH

    --
    Andrew Thompson
    physci.org 1point1c.org javasaver.com lensescapes.com athompson.info
    "..I pick up all the pieces and make an island. Might even raise just a
    little sand"
    Jimi Hendrix 'Voodoo Chile (Slight Return)'
    Andrew Thompson, Aug 19, 2005
    #3
  4. Derek

    Derek Guest

    Thanks a bunch for the reply! I was hoping that because the data just
    generates and presents itself if you try to access it through a web
    browser that it would do the same when trying to connect to it through
    Java. I'll try your suggestions, hopefully there is an API I can use.
    Thanks again!
    Derek, Aug 19, 2005
    #4
  5. Derek

    Roedy Green Guest

    On 19 Aug 2005 11:34:38 -0700, "Derek" <> wrote
    or quoted :

    >
    >Now, I've already set the proxy so that it can connect to outside web
    >pages (I've already tried it out on more mundane pages with no
    >problems), so I'm not sure what else I need to do to be able to connect
    >to it to retrieve the data. Can anyone help me out? Many thanks!


    Tackle a simpler problem first, e.g. read a perfectly vanilla web
    page.

    Then get a packet sniffer. See
    http://mindprod.com/jgloss/ethereal.html

    Then get your browser to poke at the weather site, and watch the
    interactions. That will help you understand what you have to do.

    If you are stuck, I could write you a class to handle this for a fee.
    Roedy Green, Aug 20, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris Jones
    Replies:
    0
    Views:
    1,271
    Chris Jones
    Jun 25, 2003
  2. Chris Jones

    Dynamically generated png images

    Chris Jones, Jun 27, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    516
    Kevin Spencer
    Jun 27, 2003
  3. joshblair

    Format data in dynamically generated columns from crosstab query

    joshblair, Jan 12, 2006, in forum: ASP .Net Datagrid Control
    Replies:
    4
    Views:
    260
    joshblair
    Jan 16, 2006
  4. ted benedict
    Replies:
    24
    Views:
    275
    ted benedict
    Jan 19, 2006
  5. Sethos
    Replies:
    2
    Views:
    99
    Sethos
    Mar 7, 2006
Loading...

Share This Page