How to batch download files from web page?

Discussion in 'Python' started by sj, May 12, 2004.

  1. sj

    sj Guest

    I wish to download hundreds of files from the University of Iowa sound
    archive. Doing it manually would be a daunting task especially since the
    files are each a few mega bytes long. Is there a standard way of using
    Python for such a task? I have a fair amount of programming experiance but
    very little of it relates to networks.

    For those who are intrested the University of Iowa's sound archive may be
    found at http://theremin.music.uiowa.edu/MIS.html

    Thanks in advance
    Steven Jones
     
    sj, May 12, 2004
    #1
    1. Advertisements

  2. sj

    Paul Rubin Guest

    sj <> writes:
    > I wish to download hundreds of files from the University of Iowa sound
    > archive. Doing it manually would be a daunting task especially since the
    > files are each a few mega bytes long. Is there a standard way of using
    > Python for such a task? I have a fair amount of programming experiance but
    > very little of it relates to networks.


    You could use the urllib module, or maybe you could use wget:

    http://www.gnu.org/software/wget/wget.html
     
    Paul Rubin, May 12, 2004
    #2
    1. Advertisements

  3. sj

    N.K Guest

    Consider http://harvestman.freezope.org
    ( Python)

    sj <> wrote in message news:<wyioc.26539$>...
    > I wish to download hundreds of files from the University of Iowa sound
    > archive. Doing it manually would be a daunting task especially since the
    > files are each a few mega bytes long. Is there a standard way of using
    > Python for such a task? I have a fair amount of programming experiance but
    > very little of it relates to networks.
    >
    > For those who are intrested the University of Iowa's sound archive may be
    > found at http://theremin.music.uiowa.edu/MIS.html
    >
    > Thanks in advance
    > Steven Jones
     
    N.K, May 12, 2004
    #3
  4. > I wish to download hundreds of files from the University of Iowa
    > sound archive. Doing it manually would be a daunting task
    > especially since the files are each a few mega bytes long. Is there
    > a standard way of using Python for such a task? I have a fair
    > amount of programming experiance but very little of it relates
    > to networks.
    >
    > For those who are intrested the University of Iowa's sound archive
    > may be found at http://theremin.music.uiowa.edu/MIS.html


    The easiest way to download the individual files is with
    urllib.urlretrieve().

    You can parse the HTML files using either htmllib.HTMLParser or the
    HTMLParser module, combined with urllib.urlopen().

    If you do this Google search:

    urlopen htmlparser

    The first several matches have some nice code samples showing how to find
    the links in a web page. Add urlretrieve() to download the files and you'll
    be off and running.

    -Mike
     
    Michael Geary, May 12, 2004
    #4
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. r dubey
    Replies:
    2
    Views:
    3,502
    ʹÃûÑï
    Dec 15, 2003
  2. Ryan Taylor
    Replies:
    2
    Views:
    1,326
    Ryan Taylor
    Nov 10, 2004
  3. Jon Maz
    Replies:
    2
    Views:
    512
    Jon Maz
    Dec 14, 2004
  4. Brett  Kelly
    Replies:
    1
    Views:
    934
    Steve C. Orr [MVP, MCSD]
    Jun 16, 2006
  5. Matias Surdi

    Re: Converting .py files to batch files.

    Matias Surdi, Sep 15, 2008, in forum: Python
    Replies:
    4
    Views:
    640
  6. powah
    Replies:
    6
    Views:
    722
    powah
    May 28, 2009
  7. Bruce Horrocks
    Replies:
    3
    Views:
    476
    Bruce Horrocks
    Jan 1, 2004
  8. Replies:
    3
    Views:
    596
Loading...