Retrieving dynamic web content

Discussion in 'Perl Misc' started by Charlie, Nov 20, 2003.

  1. Charlie

    Charlie Guest

    I would like to write a script that retrieves dynamically generated web
    content. Are there any modules out there that can do this?

    Thanks in advance,
    Charlie
     
    Charlie, Nov 20, 2003
    #1
    1. Advertising

  2. Charlie <> wrote:
    >
    > I would like to write a script that retrieves dynamically generated web
    > content.



    Whether it is static or dynamic does not matter, a resource
    is a resource, it is up to the server to provide the requested
    resource in any way that it wants to.

    If what you want to do is fetch the resource, then your Perl
    program is acting as the _client_ rather than as the server.


    > Are there any modules out there that can do this?



    Yes.


    perldoc -q HTML

    How do I fetch an HTML file?

    How do I automate an HTML form submission?


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Nov 20, 2003
    #2
    1. Advertising

  3. Charlie

    Charlie Guest

    Thanks for your replies. I made a mistake in describing my original goal:
    the pages are in fact dynamically generated, but what I was actually
    referring to is data that is pulled up from a db using javascript. The
    HTML code doesn't reflect the data itself; is it possible to somehow
    retrieve the data that is displayed in a browser rather than the source?

    Charlie

    On 20 Nov 2003, Abigail wrote:

    > Charlie () wrote on MMMDCCXXXIII September MCMXCIII in
    > <URL:news:p>:
    > @@
    > @@ I would like to write a script that retrieves dynamically generated web
    > @@ content. Are there any modules out there that can do this?
    >
    >
    > For a client, it doesn't matter whether web content is generated
    > dynamically or not. In fact, he doesn't even know.
    >
    > The most commonly used modules for this sort of thing are:
    > LWP, LWP::Simple and WWW::Mechanize. Calling 'wget' or 'lynx'
    > is a solution too.
    >
    > If all you want to do is to fetch a document, I'd go for LWP::Simple.
    >
    >
    >
    > Abigail
    > --
    > srand 123456;$-=rand$_--=>@[[$-,$_]=@[[$_,$-]for(reverse+1..(@[=split
    > //=>"IGrACVGQ\x02GJCWVhP\x02PL\x02jNMP"));print+(map{$_^q^"^}@[),"\n"
    >
     
    Charlie, Nov 20, 2003
    #3
  4. Charlie

    Sam Holden Guest

    On Thu, 20 Nov 2003 14:46:21 -0800, Charlie <> wrote:
    >
    > Thanks for your replies. I made a mistake in describing my original goal:
    > the pages are in fact dynamically generated, but what I was actually
    > referring to is data that is pulled up from a db using javascript. The
    > HTML code doesn't reflect the data itself; is it possible to somehow
    > retrieve the data that is displayed in a browser rather than the source?


    Run it through a Javascript interpreter.

    --
    Sam Holden
     
    Sam Holden, Nov 20, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ben Amada
    Replies:
    3
    Views:
    2,878
    chetanwarade
    Aug 23, 2010
  2. hazz
    Replies:
    6
    Views:
    49,637
    SkyUCHC
    Jun 9, 2010
  3. aesquivel
    Replies:
    0
    Views:
    147
    aesquivel
    Jun 10, 2005
  4. Kurt
    Replies:
    0
    Views:
    187
  5. Replies:
    1
    Views:
    138
    Randy Webb
    Dec 19, 2005
Loading...

Share This Page