In loop wait for function return

[mUs]

I would like to loop through a few URL's utilizing the LWP::Simple get function.
Using the existing 'logic' ( or lack thereof ) is there a way to have the
script wait before executing the next iteration of the for loop. At the moment
the 'get' function does not have enough time to complete its' request.
I looked into fork, wait, exec, etc but those don't seem to be what i'm looking
for. I though 'eval' would do it but it doesn't. thanks


#!/usr/bin/perl -w

use strict;
use diagnostics;
use LWP::Simple;
require HTML::TokeParser;

my $url = 'http://localhost/page.html';
my $url2 = 'http://localhost/page2.html';
my $url3 = 'http://localhost/page3.html';
my @urls = qq($url $url2 $url3);

# yes, of course this works
#die "LWP::Simple is empty: $! " unless ( parse_page(my $webPage = get($url)));

# loop does not give LWP::Simple function get enough time to
# return before looping, just dies

for my $page (@urls) {
die "cant get web page: $!\n" unless ( my $webPage = get($page));
# eval does not die, exits with success but nothing sent to
# function
#eval {($webPage = get($page));}; die $@ if $@; # returns ok
#eval {&parse_page($webPage = get($page));}; die $@ if $@; #return ok but
# nothing send to function, script dies
}


sub parse_page {
shift ,etc
}

 

[Gregory Toomey]

I would like to loop through a few URL's utilizing the LWP::Simple get function.
Using the existing 'logic' ( or lack thereof ) is there a way to have the
script wait before executing the next iteration of the for loop. At the moment
the 'get' function does not have enough time to complete its' request.
I looked into fork, wait, exec, etc but those don't seem to be what i'm looking
for. I though 'eval' would do it but it doesn't. thanks

The get function returns a string, usually containing HTML. It generally
takes 1-2 seconds to run for me.

If you are saying that the web server at the other end takes a long time to
return you request, or does not return at all, then maybe he URL is wrong or
the web server is overoaded? Have you tried typing the URLs in a browser?

Forking of subprocesses does not solve the underlying problem.

gtoomey

 

[Eric J. Roode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I would like to loop through a few URL's utilizing the LWP::Simple get
function. Using the existing 'logic' ( or lack thereof ) is there a
way to have the script wait before executing the next iteration of the
for loop. At the moment the 'get' function does not have enough time
to complete its' request. I looked into fork, wait, exec, etc but
those don't seem to be what i'm looking for. I though 'eval' would do
it but it doesn't. thanks

The script *is* waiting for the 'get' to return before going on to the next
iteration of the loop. What are you seeing (versus what you're expecting)
that is causing you to think otherwise?

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBPzWdRGPeouIeTNHoEQJ+kACeK1PREMpNGus4MkEAcJc0rR7pi6sAoONf
MUsJJ+LnlbeQNSOZSZDbqpuQ
=Ila7
-----END PGP SIGNATURE-----

 

[Bart Lateur]

The script *is* waiting for the 'get' to return before going on to the next
iteration of the loop. What are you seeing (versus what you're expecting)
that is causing you to think otherwise?

To the OP: check the return status of the request. You won't be able to
do that using get(), but getstore(), which will save the retrieved data
as a file, will at least return the status code. If it's not 200,
something went wrong. More elaborate scripts using LWP::UserAgent can
return you the complete headers.