IOException for URL

[Alan]

When I try the code below with "http://www.google.com/search?
source=ig&hl=en&rlz=&q=something" in the URLs.txt file, I get an
IOException error. The server is returning an HTTP response code
403. However, when I open the same URL in my browser, it works
fine.

A different but similarly formated URL to another server worked
fine. Maybe this server is expection something additional?

I am new at this, and I wonder if it is obvious what the
problems is.

Thanks, Alan


// download text content of URL

import java.net.*;
import java.io.*;

public class URLget
{
public static void main ( String[] args ) throws IOException
{
try
{
BufferedReader infile = new BufferedReader(new
FileReader("URLs.txt"));

PrintWriter out = new PrintWriter(new
FileOutputStream("downloaded.html"));
String aURL, str;

while ((aURL = infile.readLine()) != null)
{
URL url = new URL(aURL);
System.out.println(aURL);

BufferedReader in = new BufferedReader(new
InputStreamReader(url.openStream()));

while ((str = in.readLine()) != null)
{
out.println(str);
}

in.close();
}
infile.close();

out.close();
}
catch (MalformedURLException e) {e.printStackTrace();}
catch (IOException e) {e.printStackTrace();}
}
}

 

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

When I try the code below with "http://www.google.com/search?
source=ig&hl=en&rlz=&q=something" in the URLs.txt file, I get an
IOException error. The server is returning an HTTP response code
403. However, when I open the same URL in my browser, it works
fine.

A different but similarly formated URL to another server worked
fine. Maybe this server is expection something additional?

I think Google tests on browser type.

In 2004 the following worked:

URL url = new URL("http://www.google.dk/search?q=hej");
HttpURLConnection con = (HttpURLConnection)url.openConnection();
con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible;
MSIE 5.01; Windows NT 5.0)");
con.setRequestProperty("Referrer", "http://www.google.dk/");
con.connect();
if(con.getResponseCode() == HttpURLConnection.HTTP_OK) {
InputStream is = con.getInputStream();
byte[] b = new byte[1000];
int n;
while((n = is.read(b)) >= 0) {
System.out.println(new String(b,0,n));
}
is.close();
} else {
System.out.println(con.getResponseCode() + " " +
con.getResponseMessage());
}
con.disconnect();


Arne