Automatically save a webpage as a text file

[Chris Petersen]

OS: XP Pro with IE 6.0

Every morning I come in, open 2 websites, and save their content as
TXT files, one is a TAB file the other is a CSV file.

Example addresses:
http://somesite/reports/date.tab
http://somesite/reports/date.csv

I then save them as:
http://somesite/reports/date.tab.txt
http://somesite/reports/date.csv.txt

I would like to write a program to automate this, I was wondering
which language would be best, and maybe get a couple of quick and
dirty examples.

Thanks in advance

 

[Wolfgang 'Dreamguard' Nagele]

I would like to write a program to automate this, I was wondering

which language would be best, and maybe get a couple of quick and
dirty examples.

as you describe i think you got shell access to that machine?
if so - just make a cronjob (google be your friend) and simply copy those
files with bash 'cp' command.

yours, dreamguard.

 

[Ted Davis]

OS: XP Pro with IE 6.0

Every morning I come in, open 2 websites, and save their content as
TXT files, one is a TAB file the other is a CSV file.

Example addresses:
http://somesite/reports/date.tab
http://somesite/reports/date.csv

I then save them as:
http://somesite/reports/date.tab.txt
http://somesite/reports/date.csv.txt

I would like to write a program to automate this, I was wondering
which language would be best, and maybe get a couple of quick and
dirty examples.

Thanks in advance

Either wget or Lynx will do this as a simple command

wget http://somesite/reports/date.tab
wget http://somesite/reports/date.csv

lynx -dump http://somesite/reports/date.tab > date.tab
lynx -dump http://somesite/reports/date.csv > date.csv


<http://unxutils.sourceforge.net/> and <http://lynx.isc.org/release/>

You can put the commands in a batch file and do the whole thing with
one click. Of you log in each morning, you can put the batch file in
your startup folder, though you might want to check their dates before
downloading, even though double downloads (in case of reboot) would
not likely be a problem.


T.E.D. ([email protected] - e-mail must contain "T.E.D." or my .sig in the body)

 

[Tad McClellan]

open 2 websites, and save their content as
TXT files

I would like to write a program to automate this, I was wondering
which language would be best, and maybe get a couple of quick and
dirty examples.

Here it is in Perl (untested):

use LWP::Simple;
getstore( 'http://somesite/reports/date.tab',
'http://somesite/reports/date.tab.txt' );
getstore( 'http://somesite/reports/date.csv',
'http://somesite/reports/date.csv.txt' );

 

[Bob Walton]

OS: XP Pro with IE 6.0

Every morning I come in, open 2 websites, and save their content as
TXT files, one is a TAB file the other is a CSV file.

Example addresses:
http://somesite/reports/date.tab
http://somesite/reports/date.csv

I then save them as:
http://somesite/reports/date.tab.txt
http://somesite/reports/date.csv.txt

I would like to write a program to automate this, I was wondering
which language would be best, and maybe get a couple of quick and
dirty examples.

....


Assuming you want to do it in Perl since you asked here:

#!usr/bin/perl -w
use strict;
use LWP::Simple;
getstore($ARGV[0],$ARGV[1]);

Invoke as something like:

perl file.pl http://xyz.com/path/etc file_to_store_it.in

Fancy it up with checks to see if the arguments were supplied, and if
the web page was successfully retrieved and stored.