Iterating and downloading a URL that lists files?

Discussion in 'ASP .Net' started by Jim Bancroft, Aug 2, 2007.

  1. Jim Bancroft

    Jim Bancroft Guest

    Is there a way to open a URL, one that shows nothing other than a directory
    listing, and iteratively go through the list to download the files?
     
    Jim Bancroft, Aug 2, 2007
    #1
    1. Advertising

  2. On Aug 2, 5:15 am, "Jim Bancroft" <> wrote:
    > Is there a way to open a URL, one that shows nothing other than a directory
    > listing, and iteratively go through the list to download the files?


    Hi Jim,

    you can use a HttpWebRequest with a StreamReader to get the page from
    the URL, then the content of the page can be parsed with a regular
    expression to get the list of all links and then again use
    HttpWebRequest to get the files...

    Hope it helps
     
    Alexey Smirnov, Aug 2, 2007
    #2
    1. Advertising

  3. Hi,

    Alexey Smirnov wrote:
    > On Aug 2, 5:15 am, "Jim Bancroft" <> wrote:
    >> Is there a way to open a URL, one that shows nothing other than a directory
    >> listing, and iteratively go through the list to download the files?

    >
    > Hi Jim,
    >
    > you can use a HttpWebRequest with a StreamReader to get the page from
    > the URL, then the content of the page can be parsed with a regular
    > expression to get the list of all links and then again use
    > HttpWebRequest to get the files...
    >
    > Hope it helps


    I did that a long time ago with a Java applet. I then discovered that
    the page format was different in Netscape (back then) and IE. I hope
    things are more standard now!

    HTH,
    Laurent
    --
    Laurent Bugnion [MVP ASP.NET]
    Software engineering, Blog: http://www.galasoft.ch
    PhotoAlbum: http://www.galasoft.ch/pictures
    Support children in Calcutta: http://www.calcutta-espoir.ch
     
    Laurent Bugnion, MVP, Aug 2, 2007
    #3
  4. On Aug 2, 10:50 am, "Laurent Bugnion, MVP" <>
    wrote:

    > the page format was different in Netscape (back then) and IE. I hope
    > things are more standard now!


    Laurent, do you mean that the default directory listing depends on the
    client browser? It depends on a web server and not on a browser. I
    have no example for IIS, but here's the one for Apache

    Default settings:
    http://tomcat.apache.org/tomcat-6.0-doc/default-servlet.html

    Example how it looks on the website:
    http://cohred.org/cohred/content/

    When directory browsing in enabled on a web site and there isn't a
    default document available, IIS will display the contents of the
    directory as a similar page and it looks the same in IE, and other
    browsers
     
    Alexey Smirnov, Aug 2, 2007
    #4
  5. Hi,

    Alexey Smirnov wrote:
    > On Aug 2, 10:50 am, "Laurent Bugnion, MVP" <>
    > wrote:
    >
    >> the page format was different in Netscape (back then) and IE. I hope
    >> things are more standard now!

    >
    > Laurent, do you mean that the default directory listing depends on the
    > client browser? It depends on a web server and not on a browser. I
    > have no example for IIS, but here's the one for Apache


    Back then, the client browser used to add content to the directory
    listing page. I tested again right now in Firefox and IE, and apparently
    this is not the case anymore, so you're lucky!

    I am talking about Netspace 4 and IE4, that was back in... not sure...
    97 maybe?

    Anyway, good luck with your project!
    Laurent
    --
    Laurent Bugnion [MVP ASP.NET]
    Software engineering, Blog: http://www.galasoft.ch
    PhotoAlbum: http://www.galasoft.ch/pictures
    Support children in Calcutta: http://www.calcutta-espoir.ch
     
    Laurent Bugnion, MVP, Aug 2, 2007
    #5
  6. On Aug 2, 1:22 pm, "Laurent Bugnion, MVP" <>
    wrote:
    >
    > I am talking about Netspace 4 and IE4, that was back in... not sure...
    > 97 maybe?


    Huh? We are talking about ASP.NET :)
     
    Alexey Smirnov, Aug 2, 2007
    #6
  7. Hi,

    Alexey Smirnov wrote:
    > On Aug 2, 1:22 pm, "Laurent Bugnion, MVP" <>
    > wrote:
    >> I am talking about Netspace 4 and IE4, that was back in... not sure...
    >> 97 maybe?

    >
    > Huh? We are talking about ASP.NET :)


    Yeah I was rambling. Your post awoke old nostalgies, sorry :) I'll shut
    up now. :)

    Laurent
    --
    Laurent Bugnion [MVP ASP.NET]
    Software engineering, Blog: http://www.galasoft.ch
    PhotoAlbum: http://www.galasoft.ch/pictures
    Support children in Calcutta: http://www.calcutta-espoir.ch
     
    Laurent Bugnion, MVP, Aug 2, 2007
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?UTF-8?B?w4FuZ2VsIEd1dGnDqXJyZXogUm9kcsOtZ3Vleg==

    List of lists of lists of lists...

    =?UTF-8?B?w4FuZ2VsIEd1dGnDqXJyZXogUm9kcsOtZ3Vleg==, May 8, 2006, in forum: Python
    Replies:
    5
    Views:
    425
    =?UTF-8?B?w4FuZ2VsIEd1dGnDqXJyZXogUm9kcsOtZ3Vleg==
    May 15, 2006
  2. Gal Diskin

    Iterating over several lists at once

    Gal Diskin, Dec 13, 2006, in forum: Python
    Replies:
    17
    Views:
    462
    Roy Smith
    Dec 27, 2006
  3. JB
    Replies:
    2
    Views:
    391
  4. carl
    Replies:
    5
    Views:
    2,427
    James Kanze
    Nov 25, 2009
  5. iterating lists

    , Jan 23, 2010, in forum: Python
    Replies:
    7
    Views:
    309
    Stefan Behnel
    Jan 24, 2010
Loading...

Share This Page