URL connection -- browser headers

Discussion in 'Java' started by Manivannan Palanichamy, Aug 10, 2007.

  1. Hi
    I am writing a java url connection code to read a web page. The
    intelligent web server denies the request (returns 500 error). Yes, I
    know the reason -- the web server identifies my java client as a robot/
    spam program. Is there any way to pass the browser headers like ,
    Accept-Language: en ?
    I know I can pass these using java socket, but is there anyway to
    put these values in properties for URL connection?

    --
    Manivannan.Palanichamy (@) Oracle.com
    http://mani.gw.googlepages.com/index.html
    Manivannan Palanichamy, Aug 10, 2007
    #1
    1. Advertising

  2. Manivannan Palanichamy wrote:
    ...
    >I am writing a java url connection code to read a web page.


    URL?

    >...The
    >intelligent web server denies the request (returns 500 error). Yes, I
    >know the reason -- the web server identifies my java client as a robot/
    >spam program. Is there any way to pass the browser headers like ,
    >Accept-Language: en ?


    Sure, but why not ..
    a) convince the site administrators to allow connections
    from UA's that identify themselves as Java, or..
    b) respect their wishes *not* to connect?

    --
    Andrew Thompson
    http://www.athompson.info/andrew/

    Message posted via http://www.javakb.com
    Andrew Thompson, Aug 11, 2007
    #2
    1. Advertising

  3. Manivannan Palanichamy wrote:
    > I am writing a java url connection code to read a web page. The
    > intelligent web server denies the request (returns 500 error). Yes, I
    > know the reason -- the web server identifies my java client as a robot/
    > spam program. Is there any way to pass the browser headers like ,
    > Accept-Language: en ?
    > I know I can pass these using java socket, but is there anyway to
    > put these values in properties for URL connection?


    URL url = new URL("http://www.foobar.com/");
    HttpURLConnection con = (HttpURLConnection)url.openConnection();
    con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE
    5.01; Windows NT 5.0)");

    (this is emulating Windows 2000 and IE 5.0 - you may want something
    more recent)

    Arne
    =?ISO-8859-1?Q?Arne_Vajh=F8j?=, Aug 11, 2007
    #3
  4. On Aug 11, 7:10 am, Arne Vajhøj <> wrote:
    > Manivannan Palanichamy wrote:
    > > I am writing a java url connection code to read a web page. The
    > > intelligent web server denies the request (returns 500 error). Yes, I
    > > know the reason -- the web server identifies my java client as a robot/
    > > spam program. Is there any way to pass the browser headers like ,
    > > Accept-Language: en ?
    > > I know I can pass these using java socket, but is there anyway to
    > > put these values in properties for URL connection?

    >
    > URL url = new URL("http://www.foobar.com/");
    > HttpURLConnection con = (HttpURLConnection)url.openConnection();
    > con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE
    > 5.01; Windows NT 5.0)");
    >
    > (this is emulating Windows 2000 and IE 5.0 - you may want something
    > more recent)
    >
    > Arne


    Cool and quick idea. Haven't tried it. but, thanks in advance.

    --
    Manivannan.Palanichamy (@) Oracle.com
    http://mani.gw.googlepages.com/index.html
    Manivannan Palanichamy, Aug 11, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jon paugh
    Replies:
    1
    Views:
    708
  2. dont bother
    Replies:
    0
    Views:
    796
    dont bother
    Mar 3, 2004
  3. Msj121
    Replies:
    2
    Views:
    396
    Roedy Green
    Sep 19, 2007
  4. Phil
    Replies:
    4
    Views:
    671
    Gabriel Genellina
    Jan 17, 2010
  5. Ian
    Replies:
    2
    Views:
    1,944
Loading...

Share This Page