Automatically save a webpage as a text file

Discussion in 'Perl Misc' started by Chris Petersen, Aug 21, 2003.

  1. Chris Petersen, Aug 21, 2003
    #1
    1. Advertising

  2. > I would like to write a program to automate this, I was wondering
    > which language would be best, and maybe get a couple of quick and
    > dirty examples.

    as you describe i think you got shell access to that machine?
    if so - just make a cronjob (google be your friend) and simply copy those
    files with bash 'cp' command.

    yours, dreamguard.
    Wolfgang 'Dreamguard' Nagele, Aug 21, 2003
    #2
    1. Advertising

  3. Chris Petersen

    Ted Davis Guest

    On 20 Aug 2003 16:44:24 -0700, (Chris
    Petersen) wrote:

    >OS: XP Pro with IE 6.0
    >
    >Every morning I come in, open 2 websites, and save their content as
    >TXT files, one is a TAB file the other is a CSV file.
    >
    >Example addresses:
    >http://somesite/reports/date.tab
    >http://somesite/reports/date.csv
    >
    >I then save them as:
    >http://somesite/reports/date.tab.txt
    >http://somesite/reports/date.csv.txt
    >
    >I would like to write a program to automate this, I was wondering
    >which language would be best, and maybe get a couple of quick and
    >dirty examples.
    >
    >Thanks in advance



    Either wget or Lynx will do this as a simple command

    wget http://somesite/reports/date.tab
    wget http://somesite/reports/date.csv

    lynx -dump http://somesite/reports/date.tab > date.tab
    lynx -dump http://somesite/reports/date.csv > date.csv


    <http://unxutils.sourceforge.net/> and <http://lynx.isc.org/release/>

    You can put the commands in a batch file and do the whole thing with
    one click. Of you log in each morning, you can put the batch file in
    your startup folder, though you might want to check their dates before
    downloading, even though double downloads (in case of reboot) would
    not likely be a problem.


    T.E.D. ( - e-mail must contain "T.E.D." or my .sig in the body)
    Ted Davis, Aug 21, 2003
    #3
  4. Chris Petersen <> wrote:

    > open 2 websites, and save their content as
    > TXT files



    > I would like to write a program to automate this, I was wondering
    > which language would be best, and maybe get a couple of quick and
    > dirty examples.



    Here it is in Perl (untested):

    use LWP::Simple;
    getstore( 'http://somesite/reports/date.tab',
    'http://somesite/reports/date.tab.txt' );
    getstore( 'http://somesite/reports/date.csv',
    'http://somesite/reports/date.csv.txt' );


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Aug 21, 2003
    #4
  5. Chris Petersen

    Bob Walton Guest

    Chris Petersen wrote:

    > OS: XP Pro with IE 6.0
    >
    > Every morning I come in, open 2 websites, and save their content as
    > TXT files, one is a TAB file the other is a CSV file.
    >
    > Example addresses:
    > http://somesite/reports/date.tab
    > http://somesite/reports/date.csv
    >
    > I then save them as:
    > http://somesite/reports/date.tab.txt
    > http://somesite/reports/date.csv.txt
    >
    > I would like to write a program to automate this, I was wondering
    > which language would be best, and maybe get a couple of quick and
    > dirty examples.

    ....


    Assuming you want to do it in Perl since you asked here:

    #!usr/bin/perl -w
    use strict;
    use LWP::Simple;
    getstore($ARGV[0],$ARGV[1]);

    Invoke as something like:

    perl file.pl http://xyz.com/path/etc file_to_store_it.in

    Fancy it up with checks to see if the arguments were supplied, and if
    the web page was successfully retrieved and stored.

    --
    Bob Walton
    Bob Walton, Aug 21, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. sincethe2003
    Replies:
    2
    Views:
    539
    Craig Deelsnyder
    Jul 14, 2004
  2. Kylin
    Replies:
    1
    Views:
    5,703
    SCDeveloper
    May 16, 2005
  3. subhadip
    Replies:
    0
    Views:
    616
    subhadip
    Mar 28, 2007
  4. Tommy

    Save File automatically

    Tommy, May 11, 2005, in forum: ASP General
    Replies:
    1
    Views:
    119
    Patrice
    May 11, 2005
  5. Les Juby
    Replies:
    1
    Views:
    83
Loading...

Share This Page