URL connection -- browser headers

[Manivannan Palanichamy]

Hi
I am writing a java url connection code to read a web page. The
intelligent web server denies the request (returns 500 error). Yes, I
know the reason -- the web server identifies my java client as a robot/
spam program. Is there any way to pass the browser headers like ,
Accept-Language: en ?
I know I can pass these using java socket, but is there anyway to
put these values in properties for URL connection?

 

[Andrew Thompson]

Manivannan Palanichamy wrote:
...

I am writing a java url connection code to read a web page.
URL?

...The
intelligent web server denies the request (returns 500 error). Yes, I
know the reason -- the web server identifies my java client as a robot/
spam program. Is there any way to pass the browser headers like ,
Accept-Language: en ?

Sure, but why not ..
a) convince the site administrators to allow connections
from UA's that identify themselves as Java, or..
b) respect their wishes *not* to connect?

 

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

I am writing a java url connection code to read a web page. The
intelligent web server denies the request (returns 500 error). Yes, I
know the reason -- the web server identifies my java client as a robot/
spam program. Is there any way to pass the browser headers like ,
Accept-Language: en ?
I know I can pass these using java socket, but is there anyway to
put these values in properties for URL connection?

URL url = new URL("http://www.foobar.com/");
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE
5.01; Windows NT 5.0)");

(this is emulating Windows 2000 and IE 5.0 - you may want something
more recent)

Arne

 

[Manivannan Palanichamy]

URL url = new URL("http://www.foobar.com/");
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE
5.01; Windows NT 5.0)");

(this is emulating Windows 2000 and IE 5.0 - you may want something
more recent)

Arne

Cool and quick idea. Haven't tried it. but, thanks in advance.