LWP User Agent/HTTP Request help needed!

[Bumble]

Hey all, basically I've got the LWP User Agent all working, it retrieves a
HTML page at the specified HTTP location... Now I only need a small part of
the HTML, the bit with the actual data in, what's the best way of doing
this? I was originally outputting the retrieved HTML to a HTML file and then
reading the file in, line by line until I found the required data and then
extracting that... However this requires creating a file on the server, of
which I don't seem to have permission to do, and seems a bit of a long
winded way of doing it. Is there a better way? Can I do a search of the my
$response variable somehow and then just store the bit I need? I am working
on getting file writing access on server, as I need to store the data
somehow...

Here is a snippet of the code...

$ua = LWP::UserAgent->new;
$ua->proxy('http', 'cache address here');

$request = HTTP::Request->new('GET', http://address here);

my $response = $ua->request($request);

 

[gnari]

Hey all, basically I've got the LWP User Agent all working, it retrieves a
HTML page at the specified HTTP location... Now I only need a small part of
the HTML, the bit with the actual data in, what's the best way of doing
this? I was originally outputting the retrieved HTML to a HTML file and then
reading the file in, line by line until I found the required data and then
extracting that... However this requires creating a file on the server, of
which I don't seem to have permission to do, and seems a bit of a long
winded way of doing it. Is there a better way? Can I do a search of the my
$response variable somehow and then just store the bit I need?

yes. what exactly is the problem with that?

if you need to process the data line by line, you can split it into
an array
my @arr=split /\n/,$response;
for my $line (@arr) {
# do stuff
}
or:
for my $line (split /\n/,$response) {...}

but with HTML does not really have a line structure, so it is
doubtful you gain anything by that.

if the data you are looking for is easily recognisable, you
can use a regular expression, otherwise you can use a
HTML parser from CPAN

gnari

 

[Tad McClellan]

Can I do a search of the my
$response variable somehow and then just store the bit I need?

$response->content() =~ /something/;