Downloading form based web page

Discussion in 'Perl Misc' started by Takeshi, Jun 30, 2004.

  1. Takeshi

    Takeshi Guest

    Hi,

    I'm a Perl virgin (programmed in many languages though C/C++, Java). I
    want to write a little script to download dynamic data generated by a
    form based web page. By that, I mean, one has to select options
    (actually one combo box item on the page) to have the data generated.

    Currently, I do this manually, by having a timer app pop up a reminder
    to save the data. I thought "there must be a better way to do this". I
    googled and came accros wget but I don't think that can handle form
    based web pages - it is simply good at sucking URLs. OK, the to get to
    the point - anyone knows how I can write a little script that will allow
    me to programatically select a combo box displayed on the page - and
    then save the page ?

    A little code snippet would be most useful as I am new to Perl
    (alternatively, if some one has already written such a script or can
    point me to the location of such a script, I'll be most grateful).

    MTIA
     
    Takeshi, Jun 30, 2004
    #1
    1. Advertising

  2. Takeshi

    John Bokma Guest

    Takeshi wrote:

    > Hi,
    >
    > I'm a Perl virgin (programmed in many languages though C/C++, Java). I
    > want to write a little script to download dynamic data generated by a
    > form based web page. By that, I mean, one has to select options
    > (actually one combo box item on the page) to have the data generated.
    >
    > Currently, I do this manually, by having a timer app pop up a reminder
    > to save the data. I thought "there must be a better way to do this". I
    > googled and came accros wget but I don't think that can handle form
    > based web pages - it is simply good at sucking URLs. OK, the to get to
    > the point - anyone knows how I can write a little script that will allow
    > me to programatically select a combo box displayed on the page - and
    > then save the page ?


    If you know what is selected, you can call the CGI script directly using
    a POST or GET, and parse the result. See http://johnbokma.com/perl/
    for some small examples.

    Have a peek at
    <http://search.cpan.org/~petdance/WWW-Mechanize-1.02/lib/WWW/Mechanize.pm>
    too.

    --
    John MexIT: http://johnbokma.com/mexit/
    personal page: http://johnbokma.com/
    Experienced Perl programmer available: http://castleamber.com/
    Happy Customers: http://castleamber.com/testimonials.html
     
    John Bokma, Jun 30, 2004
    #2
    1. Advertising

  3. Takeshi

    Takeshi Guest

    May tkx John - very helpful links

    John Bokma wrote:
    > Takeshi wrote:
    >
    >> Hi,
    >>
    >> I'm a Perl virgin (programmed in many languages though C/C++, Java). I
    >> want to write a little script to download dynamic data generated by a
    >> form based web page. By that, I mean, one has to select options
    >> (actually one combo box item on the page) to have the data generated.
    >>
    >> Currently, I do this manually, by having a timer app pop up a reminder
    >> to save the data. I thought "there must be a better way to do this". I
    >> googled and came accros wget but I don't think that can handle form
    >> based web pages - it is simply good at sucking URLs. OK, the to get to
    >> the point - anyone knows how I can write a little script that will
    >> allow me to programatically select a combo box displayed on the page -
    >> and then save the page ?

    >
    >
    > If you know what is selected, you can call the CGI script directly using
    > a POST or GET, and parse the result. See http://johnbokma.com/perl/
    > for some small examples.
    >
    > Have a peek at
    > <http://search.cpan.org/~petdance/WWW-Mechanize-1.02/lib/WWW/Mechanize.pm>
    > too.
    >
     
    Takeshi, Jul 2, 2004
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Matt MC
    Replies:
    2
    Views:
    532
    Thomas 'PointedEars' Lahn
    May 25, 2005
  2. Replies:
    2
    Views:
    339
    =?ISO-8859-1?Q?Erik_Wikstr=F6m?=
    Aug 29, 2007
  3. Maciej
    Replies:
    0
    Views:
    350
    Maciej
    Mar 13, 2008
  4. kitchai yong via .NET 247

    authenticate win32 form client with form based authentication web services

    kitchai yong via .NET 247, May 13, 2004, in forum: ASP .Net Web Services
    Replies:
    1
    Views:
    239
    Jan Tielens
    May 13, 2004
  5. Matt MC
    Replies:
    2
    Views:
    175
    Thomas 'PointedEars' Lahn
    May 25, 2005
Loading...

Share This Page