LWP::Simple get() problem

Discussion in 'Perl Misc' started by Hon Guin Lee - Web Producer - SMI Marketing, Sep 29, 2003.

  1. Hi All,

    I am having a problem that the LWP get() function cannot retrieve unlocalised content that begins with www from the specified URL entered on the web form I created. Using Mozilla 1.1, it just cannot retrieve the web document (hence it fails and returns undef - within the subroutine get_url), but for localised web content on the local web server, it can retrieve most web documents with no problem.

    Looking at the problem at a different perspective, I used other functions such as getstore(url,file) and mirror(url,file) where url is replaced with shift and a specified filename, the LWP::Debug just throws up some internal server errors displayed on the web browser requiring some a proxy: -

    --------------------------------------------------------------------------

    LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://sunweb.central.sun.com LWP::UserAgent::_need_proxy: Not proxied LWP::protocol::http::request: () LWP::UserAgent::request: Simple response: Found LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://sunweb.central.sun.com/redirect.jsp LWP::UserAgent::_need_proxy: Not proxied LWP::protocol::http::request: () LWP::protocol::collect: read 57 bytes LWP::UserAgent::request: Simple response:
    Found LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://sunweb.central.sun.com/location.jsp LWP::UserAgent::_need_proxy: Not proxied LWP::protocol::http::request: () LWP::protocol::collect: read 19 bytes LWP::UserAgent::request: Simple response: Found LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://sunweb.central.sun.com/redirect.jsp?location=Non-US LWP::UserAgent::_need_proxy: Not proxied LWP::protocol::http::request: () LWP::protocol::collect: read 57
    bytes LWP::UserAgent::request: Simple response: Found LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://sunweb.central.sun.com/cachedir/cachedtab_Non-US_NEWS.html LWP::UserAgent::_need_proxy: Not proxied LWP::protocol::http::request: () LWP::UserAgent::request: Simple response: Internal Server Error 500

    -- This is for a localised URL.

    --------------------------------------------------------------------------

    LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://www.sun.com LWP::UserAgent::_need_proxy: Not proxied LWP::protocol::http::request: () LWP::UserAgent::request: Simple response: Internal Server Error 500

    -- This is for a URL that begins with www.

    --------------------------------------------------------------------------

    Here is the script: -

    #!/usr/local/perl5.6/bin/perl -wT

    # perl script to get remote
    # urls and strip them and
    # upload them to teamsite

    use LWP::Simple qw(!head);
    use LWP::Debug '+';
    use CGI qw:)standard); # then only CGI.pm defines a head()
    use strict;

    print "Content-type: text/html\n\n";

    my $old_handle;

    $|++; #sets $| for STDOUT
    $old_handle = select( STDERR ); #change to STDERR
    $|++; #sets $| for STDERR
    select( $old_handle ); #change back to STDOUT

    my ($url) = @_;
    my $lang;

    process_form();
    get_url($url);

    # Passes the data from the server,
    # and takes them onto the PERL script.

    sub process_form {

    $url = param('url');
    $url = "http://$url";
    $lang = param('lang');

    }

    # Retrieves the contents of the
    # specified URL.

    sub get_url {

    my $page = getstore(shift,'hon.html');

    unless (defined $page) {
    print "Couldn't retrieve $url";
    }
    else {
    print "$page\n";
    }

    }
    ----------------------------------------------------------------------

    Are there any problems/reasons why the get() function does not retrieve unlocalised web content that doesn't start with www?

    Thanks.
    Hon Guin Lee - Web Producer - SMI Marketing, Sep 29, 2003
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Hon Guin Lee - Web Producer - SMI Marketing

    LWP::Simple get() problem

    Hon Guin Lee - Web Producer - SMI Marketing, Sep 26, 2003, in forum: Perl Misc
    Replies:
    1
    Views:
    90
    Michael Budash
    Sep 26, 2003
  2. Hon Guin Lee - Web Producer - SMI Marketing

    LWP::Simple get() refined problem

    Hon Guin Lee - Web Producer - SMI Marketing, Sep 29, 2003, in forum: Perl Misc
    Replies:
    6
    Views:
    128
    Bart Lateur
    Sep 30, 2003
  3. Paul E Collins

    Strange timeout with LWP::Simple::get

    Paul E Collins, Sep 22, 2004, in forum: Perl Misc
    Replies:
    0
    Views:
    212
    Paul E Collins
    Sep 22, 2004
  4. Replies:
    2
    Views:
    122
    Peter J. Holzer
    Jul 21, 2007
  5. Replies:
    0
    Views:
    92
Loading...

Share This Page