Re: LWP::Simple header information problems

Discussion in 'Perl' started by cp, Jun 24, 2003.

  1. cp

    cp Guest

    In article <kLLJa.10038$>,
    Mitchua <> wrote:

    > I have apache running on 2 linux boxes, one running Redhat 7.3 and the other
    > Mandrake 9.1. I have a simple Perl script that uses "head $url" from
    > LWP::Simple to access the header of a web page. It runs fine on the
    > Mandrake machine but returns 'false' on the Redhat machine with no other
    > errors. They both have the LWP::Simple module installed but are using
    > different versions of perl (if that matters).


    I might. But you haven't provided any code that illustrates the
    problem, so it's hard to tell.
    >
    > If I run another perl script that uses "get $url", they both return the
    > webpage. So why can't the Mandrake machine access the header information?


    Dunno. Show some code?
    >
    > Thanks for any insight you could give me.


    Help us to help you. Please be more specific. Instead of "it runs
    fines..." and "it returns false" it is more helpful to provides some
    specifics. What did you expect it to return? What did it return
    instead? Were there messages in the error log? Can you run it from the
    command line? Is the output different when you do?

    Shooting in the dark, perhpas this bit from perldoc LWP::Simple
    provides insight:

    CAVEAT
    Note that if you are using both LWP::Simple and the very
    popular CGI.pm module, you may be importing a `head' func-
    tion from each module, producing a warning like "Prototype
    mismatch: sub main::head ($) vs none". Get around this
    problem by just not importing LWP::Simple's `head' func-
    tion, like so:

    use LWP::Simple qw(!head);
    use CGI qw:)standard); # then only CGI.pm defines a
    head()

    Then if you do need LWP::Simple's `head' function, you can
    just call it as `LWP::Simple::head($url)'.

    SEE ALSO
    the LWP manpage, the lwpcook manpage, the LWP::UserAgent
    manpage, the HTTP::Status manpage, the lwp-request man-
    page, the lwp-mirror manpage

    --
    cp
    cp, Jun 24, 2003
    #1
    1. Advertising

  2. cp

    Mitchua Guest

    "cp" <> wrote in message
    news:240620031117080773%...
    <SNIP>
    > I might. But you haven't provided any code that illustrates the
    > problem, so it's hard to tell.


    I wrote a quick test script for you:
    Mitchua, Jun 24, 2003
    #2
    1. Advertising

  3. cp

    Mitchua Guest

    "cp" <> wrote in message
    news:240620031117080773%...
    <SNIP>
    > I might. But you haven't provided any code that illustrates the
    > problem, so it's hard to tell.


    I wrote a test program to demonstrate the problem. Here's the code:
    #!/usr/bin/perl -w

    use LWP::Simple;

    print "Content-type: text/html\n\n";

    my @headinfo = head($ARGV[0]);

    if (@headinfo) {

    print @headinfo . "Got header information\n";

    print "Modified date: " . localtime($headinfo[2]) . "\n";

    } else {

    print @headinfo . "Problem with head\n";

    }

    my $content = get($ARGV[0]);

    if ($content) {

    print "Got content too!\n";

    }


    When I run it on the Mandrake machine with
    http://mandrakemachine_address/cgi-bin/getheadtest.cgi?http://www.netkee.com
    I get a webpage: "5Got header information Modified date: Sun May 18 11:01:19
    2003 Got content too! "

    When I run it on the Redhat machine with
    http://redhatmachine_address/cgi-bin/getheadtest.cgi?http://www.netkee.com I
    get a webpage: "0Problem with head Got content too! "

    So like I said, both machine are getting the page, but the redhat machine is
    getting nothing from it's 'head' call for some reason.

    Thanks for your help,
    Mitchua
    Mitchua, Jun 24, 2003
    #3
  4. cp

    cp Guest

    In article <ZK%Ja.8933$>,
    Mitchua <> wrote:

    > When I run it on the Mandrake machine with
    > http://mandrakemachine_address/cgi-bin/getheadtest.cgi?http://www.netkee.com
    > I get a webpage: "5Got header information Modified date: Sun May 18 11:01:19
    > 2003 Got content too! "
    >
    > When I run it on the Redhat machine with
    > http://redhatmachine_address/cgi-bin/getheadtest.cgi?http://www.netkee.com I
    > get a webpage: "0Problem with head Got content too! "
    >
    > So like I said, both machine are getting the page, but the redhat machine is
    > getting nothing from it's 'head' call for some reason.
    >


    okay, my last thought on the subject, before I move onto something that
    pays the rent. perldoc LWP::Simple gives the head method as a way to
    see that the file exists, and mentions that, if you want more control
    of the headers, you need to use LWP::UserAgent. Example are provided in
    the docs, and in perldoc lwpcook. One pertains, which I've modified to
    be:

    #!/usr/bin/perl
    use strict;
    use warnings;

    use CGI qw(header param);

    my $url = param('url') || 'http://www.yahoo.com/';

    use LWP::UserAgent;
    my $ua = LWP::UserAgent->new();
    $ua->agent("Mozilla/7.0");
    $ua->proxy('http', 'http://10.1.0.1:8080/' );

    my $req = HTTP::Request->new( HEAD => $url );
    $req->header('Accept' => '*/*');

    # send request
    my $res = $ua->request($req);

    # check the outcome
    if ( $res->is_success ) {
    print header('text/plain'),
    $res->headers_as_string, "\n";
    }
    else {
    print header('text/plain'), "Error: " . $res->status_line . "\n";
    }

    Outputs:

    Cache-Control: private
    Date: Wed, 25 Jun 2003 18:23:07 GMT
    Age: 0
    Content-Length: 31427
    Content-Type: text/html
    Client-Date: Wed, 25 Jun 2003 18:23:08 GMT
    Client-Peer: 10.1.0.1:8080
    Client-Response-Num: 1
    P3p: policyref="http://p3p.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR
    ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi
    PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV"
    Proxy-Connection: close

    Of course, then I realized that what I have a chaching Firewall, and
    what I have here is the page that was retrieved and cahced. So that is
    something else to investigate.

    --
    cp
    cp, Jun 25, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Thomas =?ISO-8859-15?Q?G=F6tz?=

    LWP::Simple and utf8 problem

    Thomas =?ISO-8859-15?Q?G=F6tz?=, Apr 19, 2004, in forum: Perl
    Replies:
    0
    Views:
    713
    Thomas =?ISO-8859-15?Q?G=F6tz?=
    Apr 19, 2004
  2. Replies:
    0
    Views:
    336
  3. mlt
    Replies:
    2
    Views:
    831
    Jean-Marc Bourguet
    Jan 31, 2009
  4. Robert

    LWP Extra Header Lines

    Robert, Jun 24, 2003, in forum: Perl Misc
    Replies:
    0
    Views:
    121
    Robert
    Jun 24, 2003
  5. reggie

    LWP::Simple and authentication

    reggie, Jul 30, 2003, in forum: Perl Misc
    Replies:
    6
    Views:
    142
    Gregory Toomey
    Aug 3, 2003
Loading...

Share This Page