Get file list over http

[Rohit]

Hi, I am trying to write a script, which downloads files over http. I
am using curl command to download the file, but using curl I have to
pass filename. Filename on server would be changing frequently, so is
there any way to get list of files over http? Once I get filename,
then I can pass that filename to curl and download the file.

I would also be thankful, if you can suggest any other way to achieve
this task.

If I have posted this message in wrong group, please rout me to
correct group.

I would appreciate your reply. Thanks in advance.

Thanks,
Rohit

 

[Rohit]

Rohitwrote:


To 'get a list of files over http' usually means to retrieve
the index of a directory. Depending on the configuration of
the server, this may be allowed/provided or not. If the browser
displays an explorer-like view when accessing a URL ending with
a directory name, it serves back a html page with a list of
links to each file.

To retrieve this index page, you can use the LWP::Simple module.
To extract links from the page, HTML::LinkExtor or HTML::TreeBuilder
can aid you. If you have successfully extracted the links (all
modules come with a number of examples) you can once again
use LWP::Simple and retrieve the files directly to disk with its
getstore() method.

HTH
-Chris

Thanks Chris, It works great.