How to batch download files from web page?

Discussion in 'Python' started by sj, May 12, 2004.

  1. sj

    sj Guest

    I wish to download hundreds of files from the University of Iowa sound
    archive. Doing it manually would be a daunting task especially since the
    files are each a few mega bytes long. Is there a standard way of using
    Python for such a task? I have a fair amount of programming experiance but
    very little of it relates to networks.

    For those who are intrested the University of Iowa's sound archive may be
    found at http://theremin.music.uiowa.edu/MIS.html

    Thanks in advance
    Steven Jones
     
    sj, May 12, 2004
    #1
    1. Advertising

  2. sj

    Paul Rubin Guest

    sj <> writes:
    > I wish to download hundreds of files from the University of Iowa sound
    > archive. Doing it manually would be a daunting task especially since the
    > files are each a few mega bytes long. Is there a standard way of using
    > Python for such a task? I have a fair amount of programming experiance but
    > very little of it relates to networks.


    You could use the urllib module, or maybe you could use wget:

    http://www.gnu.org/software/wget/wget.html
     
    Paul Rubin, May 12, 2004
    #2
    1. Advertising

  3. sj

    N.K Guest

    Consider http://harvestman.freezope.org
    ( Python)

    sj <> wrote in message news:<wyioc.26539$>...
    > I wish to download hundreds of files from the University of Iowa sound
    > archive. Doing it manually would be a daunting task especially since the
    > files are each a few mega bytes long. Is there a standard way of using
    > Python for such a task? I have a fair amount of programming experiance but
    > very little of it relates to networks.
    >
    > For those who are intrested the University of Iowa's sound archive may be
    > found at http://theremin.music.uiowa.edu/MIS.html
    >
    > Thanks in advance
    > Steven Jones
     
    N.K, May 12, 2004
    #3
  4. > I wish to download hundreds of files from the University of Iowa
    > sound archive. Doing it manually would be a daunting task
    > especially since the files are each a few mega bytes long. Is there
    > a standard way of using Python for such a task? I have a fair
    > amount of programming experiance but very little of it relates
    > to networks.
    >
    > For those who are intrested the University of Iowa's sound archive
    > may be found at http://theremin.music.uiowa.edu/MIS.html


    The easiest way to download the individual files is with
    urllib.urlretrieve().

    You can parse the HTML files using either htmllib.HTMLParser or the
    HTMLParser module, combined with urllib.urlopen().

    If you do this Google search:

    urlopen htmlparser

    The first several matches have some nice code samples showing how to find
    the links in a web page. Add urlretrieve() to download the files and you'll
    be off and running.

    -Mike
     
    Michael Geary, May 12, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. r dubey
    Replies:
    2
    Views:
    3,317
    ʹÃûÑï
    Dec 15, 2003
  2. Hornberger, Chris

    RE: How to batch download files from web page?

    Hornberger, Chris, May 12, 2004, in forum: Python
    Replies:
    0
    Views:
    650
    Hornberger, Chris
    May 12, 2004
  3. Chris Rebert

    Re: Converting .py files to batch files.

    Chris Rebert, Sep 15, 2008, in forum: Python
    Replies:
    0
    Views:
    393
    Chris Rebert
    Sep 15, 2008
  4. Matias Surdi

    Re: Converting .py files to batch files.

    Matias Surdi, Sep 15, 2008, in forum: Python
    Replies:
    4
    Views:
    533
  5. Bruce Horrocks
    Replies:
    3
    Views:
    290
    Bruce Horrocks
    Jan 1, 2004
Loading...

Share This Page