Retrieving web pages

Dackle · Dec 9, 2004

Newbie question: I'm trying to retrieve general HTML web pages but
running into a few problems, which in part are probably due to the
fact that I've never used Perl modules before. I downloaded
libwww-perl and copied LWP::Simple into the Perl lib directory, but
I'm still having trouble grabbing web pages. I tried something like
this:

use LWP::Simple;
$url="www.torontostar.com";
$content= get($url);

But I keep getting back a type 2 error saying File "internet" does
not exist. I'm using the ActiveState port of Perl on Windows XP
through the Crimson Editor. Any advice? Is there any other way to
retrieve web pages using existing Perl modules?

Henry Law · Dec 9, 2004

Newbie question: I'm trying to retrieve general HTML web pages but
running into a few problems, which in part are probably due to the
fact that I've never used Perl modules before. I downloaded
libwww-perl and copied LWP::Simple into the Perl lib directory, but
I'm still having trouble grabbing web pages. I tried something like
this:

use strict;
use warnings; # Always - no help in this group otherwise!!

use LWP::Simple;
$url="www.torontostar.com";
$content= get($url);

But I keep getting back a type 2 error saying File "internet" does
not exist. I'm using the ActiveState port of Perl on Windows XP
through the Crimson Editor. Any advice? Is there any other way to
retrieve web pages using existing Perl modules?

I don't know what is going wrong on your system (file called
"internet" doesn't sound right, somehow). But your problem is that
you've left out the "http:". This works on my system (Activestate
like yours); try it.

# ------------START-------------
use strict;
use warnings;

use LWP::Simple;
my $url="http://www.torontostar.com";
my $content= get($url);

print substr($content,0,50),"\n";
# --------------END-------------

Output is:
F:\$WIP>problem.pl
<HTML>
<HEAD>
<script language="Javascript" src="h

F:\$WIP>

Dackle · Dec 9, 2004

OK, I'll give it a try. I think I used http:// in the URL and it didn't
work, but I'll try use strict; and use warnings;.

Henry Law · Dec 9, 2004

OK, I'll give it a try. I think I used http:// in the URL and it didn't
work, but I'll try use strict; and use warnings;.

It's not a matter of "trying" ... putting them in makes it easier to
find some kinds of problem and since nobody writes perfect code first
time you should always use them.

Sorry to sound like a school teacher here but I'm trying to help ...
Saying that something "doesn't work" doesn't help people to help you.
Much better to say what you expected it to do and what it actually
did. Anyway, unless there's a bug in perl itself then the code you
write always "works", in the sense that it does what it does. The
problem comes when what it does is not what you thought it would do.

I meant to advise you earlier: it's much better to copy and paste
error messages, rather than giving a general description of them as
you have done above. The exact text is sometimes important.

Lastly, please don't top post; it's against the agreed guidelines for
this group. In addition many people hate it with a passion and most
of them are the ones you need to help you.

Sherm Pendley · Dec 9, 2004

Dackle said:
fact that I've never used Perl modules before. I downloaded
libwww-perl and copied LWP::Simple into the Perl lib directory

That's the first problem. Read and follow the instructions in 'perldoc
perlmodinstall' - don't just copy files around.

I'm still having trouble grabbing web pages. I tried something like
this:

Post the code you tried, not "something like" it. Copy and paste it,
don't try to re-type it.

Have you read the posting guidelines that appear here frequently?

sherm--

Tintin · Dec 10, 2004

Dackle said:
Newbie question: I'm trying to retrieve general HTML web pages but
running into a few problems, which in part are probably due to the
fact that I've never used Perl modules before. I downloaded
libwww-perl and copied LWP::Simple into the Perl lib directory.

You can't just randomly copy parts of Perl modules and hope they will work.

ActiveState Perl uses PPM to manage Perl modules. Either type 'ppm' at the
command line, or follow:

Start->Programs->ActiveState Perl->Perl Package Manager

Anyway, you don't need to install libwww as it comes standard with
ActiveState Perl.

Anno Siegel · Dec 10, 2004

Tintin said:
You can't just randomly copy parts of Perl modules and hope they will work.

You can always hope!

Anno

Dackle · Dec 10, 2004

Thanks, I've got it running now. The error was in fact caused by
the filename in the text editor and not Perl itself. Sorry for the
confusion and the sloppiness.

VBSome · Dec 10, 2004

If all you are trying to do is navigate some web pages and look at the
results then I would suggest you use WWW::Mechanize or if on a Win32
platform Win32::IE::Mechanize (which is the same module as the first one but
uses IE as a browser). These 2 modules are much easier to use then LWP.

Company web-site (all perl CGI) re-design, suggestions?	2	Dec 17, 2012
Cyrillic web pages with Perl and MySQL	1	Nov 27, 2005
How do you read the content of internal web pages?	6	Jul 25, 2007
Looking for modules to help downlaod web-pages...	3	Jul 20, 2007
Web programming: issues with large amounts og data	9	Dec 3, 2008
Using IO::Socket::INET for grabbing web pages	1	Mar 7, 2004
Strange timeout with LWP::Simple::get	0	Sep 22, 2004
using perl to submit username/password and access subsequent pages	2	Feb 9, 2005

Retrieving web pages

Dackle

Henry Law

Dackle

Henry Law

Sherm Pendley

Tintin

Anno Siegel

Dackle

VBSome

Ask a Question

Similar Threads

Members online

Forum statistics

Latest Threads