Need to download 20000 pdf files

[Hemant]

I am working on a project that requires me to have access to more than
20000 pdf files. Any suggestions on how to go about searching over the
internet and be able to download the files?

 

[Tad McClellan]

be able to download the files?

use LWP::Simple;
my $pdf = get 'http://some.domain/directory/file.pdf';

 

[Peter Wyzl]

:I am working on a project that requires me to have access to more than
: 20000 pdf files. Any suggestions on how to go about searching over the
: internet and be able to download the files?


Here is something I ran to download a bunch of .swf files into the swf
directory on my c: drive
chdir 'c:/swf';
for ('001' .. '340'){
next if (-e "c:/swf/a_0${_}.swf");
system "lwp-download http://www.swfsite.com/a_0${_}.swf";
}
exit;



lwp-download comes with Activeperl

The files I need were all named a_0***.swf where *** represents the numbers
from 001 to 340

Modify it to suit your needs (assuming you have Windows and lwp-download. I
don't know if Activestate's perl for Unix includes lwp-download.



P

 

[Gregory Toomey]

I am working on a project that requires me to have access to more than
20000 pdf files. Any suggestions on how to go about searching over the
internet and be able to download the files?

Assuming one directory tree:
http://www.gnu.org/software/wget/wget.html

gtoomey

 

[Shane (aka froggy)]

I am working on a project that requires me to have access to more than
20000 pdf files. Any suggestions on how to go about searching over the
internet and be able to download the files?

er.. from all the other posts it seems an easy task
but I dont get it.. are you asking how to just download the pdf
documents.. or search them out over the net.. or search out random pdf
documents for downloading?