Need spider help to create my own wireless data

[Monty]

I just gained access to the Internet through my wireless phone. Most of
the sites I need data from contain no data because they're not setup for
wireless access. Quite frankly, everything I've seen so far is junk.

I'm thinking I need to learn how to "spider" a few important sites at an
indicated time, parse the data and print it to my own web server where I
can view it in a more "wireless friendly" mode.

Is this possible? Is it the right way to go? The sites I'm looking at
always print their data with the same HTML format so it would be easy to
strip out all the useless data and just keep what I need. I can do the
Perl coding for that but have no idea how to create the spider side of
it. How do I access data from another site using Perl in a CGI script?
That's the only coding I'm familiar with.

Monty

 

[Alan J. Flavell]

I'm thinking I need to learn how to "spider" a few important sites at an
indicated time, parse the data and print it to my own web server where I
can view it in a more "wireless friendly" mode.

I would suggest googling for Nick Kew's mod_accessibility. Might not
be precisely what you want, but the approach would seem to be a good
framework for it.

OK, you can skip the google - I have the URL here
http://www.webthing.com/software/mod_accessibility/

 

[Alan J. Flavell]

Though sometimes a few regexes will do the job - especially for
non-critical personal use stuff.

Even "personal use stuff" has a habit of adding features, bit by bit,
until it turns into an unmanageable mess. (And I say that from
well-earned personal experience.) I'd recommend starting with a sound
foundation, upon which, later enhancements could be added without
digging oneself into a deeper and deeper pit.

And, as I thought the example of Nick Kew's mod_accessibility could
illustrate (as the basis of an idea that could be implemented in other
languages, natch) - by doing the basic thing right you can have a
structure that's sufficiently versatile to do a range of jobs, which -
with the rapid developments of mobile/wireless kit - would seem
valuable even for a "non-critical personal use" project. As long as
the components (CPAN modules etc.) are there, all you should need to
code would be some Perl glue to hang them together.

IMHO, anyway.

 

[Monty]

Sam Holden wrote:

Yes, that will work perfectly. Thank you very much.

Monty