Downloading form based web page

[Takeshi]

Hi,

I'm a Perl virgin (programmed in many languages though C/C++, Java). I
want to write a little script to download dynamic data generated by a
form based web page. By that, I mean, one has to select options
(actually one combo box item on the page) to have the data generated.

Currently, I do this manually, by having a timer app pop up a reminder
to save the data. I thought "there must be a better way to do this". I
googled and came accros wget but I don't think that can handle form
based web pages - it is simply good at sucking URLs. OK, the to get to
the point - anyone knows how I can write a little script that will allow
me to programatically select a combo box displayed on the page - and
then save the page ?

A little code snippet would be most useful as I am new to Perl
(alternatively, if some one has already written such a script or can
point me to the location of such a script, I'll be most grateful).

MTIA

 

[John Bokma]

Hi,

I'm a Perl virgin (programmed in many languages though C/C++, Java). I
want to write a little script to download dynamic data generated by a
form based web page. By that, I mean, one has to select options
(actually one combo box item on the page) to have the data generated.

Currently, I do this manually, by having a timer app pop up a reminder
to save the data. I thought "there must be a better way to do this". I
googled and came accros wget but I don't think that can handle form
based web pages - it is simply good at sucking URLs. OK, the to get to
the point - anyone knows how I can write a little script that will allow
me to programatically select a combo box displayed on the page - and
then save the page ?

If you know what is selected, you can call the CGI script directly using
a POST or GET, and parse the result. See http://johnbokma.com/perl/
for some small examples.

Have a peek at
<http://search.cpan.org/~petdance/WWW-Mechanize-1.02/lib/WWW/Mechanize.pm>
too.

 

[Takeshi]

May tkx John - very helpful links

If you know what is selected, you can call the CGI script directly using
a POST or GET, and parse the result. See http://johnbokma.com/perl/
for some small examples.

Have a peek at
<http://search.cpan.org/~petdance/WWW-Mechanize-1.02/lib/WWW/Mechanize.pm>
too.