A
Andrea Francia
I need write a program which download many files from different web
sites. The web sites requires basic authentication. And I want spread
many threads, each one downloading a list of files that need different
authentication credentials.
Anyone knows what is the simplest way to achieve this? I found many http
libraries but all seem very complex.
I studied how to use the standard java library to achieve this, but it
seems that is not feasible with it.
For example the next code download a file that not require authentication.
URL url = new URL("http://www.example.org/file.txt");
URLConnection con = url.openConnection();
BufferedInputStream in;
in = new BufferedInputStream(con.getInputStream());
OutputStream out = new FileOutputStream("C:\\file.txt");
int i = 0;
byte[] bytesIn = new byte[8096];
while ((i = in.read(bytesIn)) >= 0) {
out.write(bytesIn, 0, i);
}
out.close();
in.close();
But the problems arise when you try to download a file that need
authentication in a threaded enviroment.
To provide the authentication credentials you should use the
Authenticator.setDefault() method which is a static method and therefore
not usable in a threaded enviroment.
I tried also embedding the username and password in the URL but these
where ignored.
url = new URL("http://username:[email protected]/file.txt");
Thanks
sites. The web sites requires basic authentication. And I want spread
many threads, each one downloading a list of files that need different
authentication credentials.
Anyone knows what is the simplest way to achieve this? I found many http
libraries but all seem very complex.
I studied how to use the standard java library to achieve this, but it
seems that is not feasible with it.
For example the next code download a file that not require authentication.
URL url = new URL("http://www.example.org/file.txt");
URLConnection con = url.openConnection();
BufferedInputStream in;
in = new BufferedInputStream(con.getInputStream());
OutputStream out = new FileOutputStream("C:\\file.txt");
int i = 0;
byte[] bytesIn = new byte[8096];
while ((i = in.read(bytesIn)) >= 0) {
out.write(bytesIn, 0, i);
}
out.close();
in.close();
But the problems arise when you try to download a file that need
authentication in a threaded enviroment.
To provide the authentication credentials you should use the
Authenticator.setDefault() method which is a static method and therefore
not usable in a threaded enviroment.
I tried also embedding the username and password in the URL but these
where ignored.
url = new URL("http://username:[email protected]/file.txt");
Thanks