Taking snapshot of a webpage

Discussion in 'Perl Misc' started by sujay.tukai@gmail.com, Aug 9, 2006.

  1. Guest

    Can anyone help me how to take a snapshot of a webpage
     
    , Aug 9, 2006
    #1
    1. Advertising

  2. darkmoo Guest

    On Tue, 08 Aug 2006 22:23:57 -0700, sujay.tukai wrote:

    > Can anyone help me how to take a snapshot of a webpage


    Use wget & grab the input from that.

    But even more easier just lookup on cpan.org for a module that will do it
    for you. HTTP::Lite seems good

    http://search.cpan.org/~rhooper/HTTP-Lite-2.1.6/Lite.pm
     
    darkmoo, Aug 9, 2006
    #2
    1. Advertising

  3. Guest

    darkmoo wrote:

    > On Tue, 08 Aug 2006 22:23:57 -0700, sujay.tukai wrote:
    >
    > > Can anyone help me how to take a snapshot of a webpage

    >
    > Use wget & grab the input from that.
    >
    > But even more easier just lookup on cpan.org for a module that will do it
    > for you. HTTP::Lite seems good
    >
    > http://search.cpan.org/~rhooper/HTTP-Lite-2.1.6/Lite.pm


    I got it but my mai motive is to get the snapshot of the webpage by
    using different proxies. So that i can detect the how the page is
    viewed in different regions
     
    , Aug 9, 2006
    #3
  4. Guest

    darkmoo wrote:

    > On Tue, 08 Aug 2006 22:23:57 -0700, sujay.tukai wrote:
    >
    > > Can anyone help me how to take a snapshot of a webpage

    >
    > Use wget & grab the input from that.
    >
    > But even more easier just lookup on cpan.org for a module that will do it
    > for you. HTTP::Lite seems good
    >
    > http://search.cpan.org/~rhooper/HTTP-Lite-2.1.6/Lite.pm


    I got it. But my main motive is to get the snapshot of the webpage
    using different proxies, using which i can detect how the page is
    viewed in different regions
     
    , Aug 9, 2006
    #4
  5. wrote:

    > Can anyone help me how to take a snapshot of a webpage


    #!/usr/bin/perl
    use strict;
    use warnings;
    use LWP::UserAgent;

    my $webpage = 'http://www.google.com';

    my $ua = new LWP::UserAgent;
    $ua->agent("AgentName/0.1 " . $ua->agent);
    my $req = new HTTP::Request GET => $webpage;
    $req->content_type('text/html');
    my $res = $ua->request($req);
    unless ($res->is_success) {
    die "Error: couldn't get $webpage: $!\n";
    }

    my $snapshot = $res->content; # $snapshot holds the html of $webpage

    Hope this helps,

    --
    Bart
     
    Bart Van der Donck, Aug 9, 2006
    #5
  6. wrote:

    > [...]
    > I got it but my mai motive is to get the snapshot of the webpage by
    > using different proxies. So that i can detect the how the page is
    > viewed in different regions


    In my other article, just add

    $ua->proxy(['http', 'ftp'], 'http://my.proxy.com:8001/');

    or

    $ua->proxy('https', 'http://an.other.proxy.com:8002/');

    just after

    my $ua = new LWP::UserAgent;

    More info:

    http://search.cpan.org/~gaas/libwww-perl-5.803/lib/LWP/UserAgent.pm#Proxy_attributes

    --
    Bart
     
    Bart Van der Donck, Aug 9, 2006
    #6
  7. Guest

    I have already tried the LWP method but the webpage, whose snapshot i
    want to get contains lots of images, this takes a lot of toll on the
    program which does the download. For eg., the webpage takes nearly a
    minute to come..

    Can u give me some other ideas by which i can convert the webpage to a
    image and then use all the proxy and lwp thing to get it...
     
    , Aug 9, 2006
    #7
  8. -berlin.de Guest

    <> wrote in comp.lang.perl.misc:
    > I have already tried the LWP method but the webpage, whose snapshot i
    > want to get contains lots of images, this takes a lot of toll on the
    > program which does the download. For eg., the webpage takes nearly a
    > minute to come..
    >
    > Can u give me some other ideas by which i can convert the webpage to a
    > image and then use all the proxy and lwp thing to get it...


    Please don't use baby talk. Type "you", not "u". Saving two letters
    may have been a good idea when connections ran at 300 Baud. Today
    it's a childish affectation.

    In another post you said:

    > I got it but my mai motive is to get the snapshot of the webpage by
    > using different proxies. So that i can detect the how the page is
    > viewed in different regions


    If that is your motive you *must* download every bit of the page.
    A consolidated image like you propose would have to be generated
    on the server. It would only convey the server's idea of how the
    page should look like. It would look the same everywhere.

    If those are your web pages, take a hint and simplify them. If they
    aren't, you'll have to download them.

    Anno
     
    -berlin.de, Aug 9, 2006
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    1
    Views:
    438
  2. Paul
    Replies:
    14
    Views:
    861
    Alexey Smirnov
    Jun 19, 2008
  3. Jim Cain
    Replies:
    1
    Views:
    211
    Yukihiro Matsumoto
    Jul 18, 2003
  4. sifar
    Replies:
    5
    Views:
    435
  5. soren625
    Replies:
    2
    Views:
    382
    soren625
    Dec 12, 2006
Loading...

Share This Page