Need spider help to create my own wireless data

Discussion in 'Perl Misc' started by Monty, Aug 23, 2003.

  1. Monty

    Monty Guest

    I just gained access to the Internet through my wireless phone. Most of
    the sites I need data from contain no data because they're not setup for
    wireless access. Quite frankly, everything I've seen so far is junk.

    I'm thinking I need to learn how to "spider" a few important sites at an
    indicated time, parse the data and print it to my own web server where I
    can view it in a more "wireless friendly" mode.

    Is this possible? Is it the right way to go? The sites I'm looking at
    always print their data with the same HTML format so it would be easy to
    strip out all the useless data and just keep what I need. I can do the
    Perl coding for that but have no idea how to create the spider side of
    it. How do I access data from another site using Perl in a CGI script?
    That's the only coding I'm familiar with.

    Monty
    Monty, Aug 23, 2003
    #1
    1. Advertising

  2. On Sat, Aug 23, Monty inscribed on the eternal scroll:

    > I'm thinking I need to learn how to "spider" a few important sites at an
    > indicated time, parse the data and print it to my own web server where I
    > can view it in a more "wireless friendly" mode.


    I would suggest googling for Nick Kew's mod_accessibility. Might not
    be precisely what you want, but the approach would seem to be a good
    framework for it.

    OK, you can skip the google - I have the URL here
    http://www.webthing.com/software/mod_accessibility/
    Alan J. Flavell, Aug 23, 2003
    #2
    1. Advertising

  3. On Sat, Aug 23, Sam Holden inscribed on the eternal scroll:

    > Though sometimes a few regexes will do the job - especially for
    > non-critical personal use stuff.


    Even "personal use stuff" has a habit of adding features, bit by bit,
    until it turns into an unmanageable mess. (And I say that from
    well-earned personal experience.) I'd recommend starting with a sound
    foundation, upon which, later enhancements could be added without
    digging oneself into a deeper and deeper pit.

    And, as I thought the example of Nick Kew's mod_accessibility could
    illustrate (as the basis of an idea that could be implemented in other
    languages, natch) - by doing the basic thing right you can have a
    structure that's sufficiently versatile to do a range of jobs, which -
    with the rapid developments of mobile/wireless kit - would seem
    valuable even for a "non-critical personal use" project. As long as
    the components (CPAN modules etc.) are there, all you should need to
    code would be some Perl glue to hang them together.

    IMHO, anyway.
    Alan J. Flavell, Aug 23, 2003
    #3
  4. Monty

    Monty Guest

    Sam Holden wrote:

    Yes, that will work perfectly. Thank you very much.

    Monty

    >
    >
    >The LWP modules make it very easy to grab the content from a URL.
    >For simple stuff it can be as easy as:
    >
    >use LWP::Simple;
    >my $content = get('http://www.perl.com/');
    >
    >Then you just munge it to suit the format you want.
    >
    >The various HTML parsing modules (such as HTML::parser) make doing
    >that munging reasonably straight forward. Though sometimes a few
    >regexes will do the job - especially for non-critical personal use
    >stuff.
    >
    >
    >
    Monty, Aug 30, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Vasanth
    Replies:
    0
    Views:
    520
    Vasanth
    Jun 28, 2004
  2. Auction software
    Replies:
    0
    Views:
    398
    Auction software
    Jun 27, 2004
  3. Laszlo Zsolt Nagy

    Need a spider library

    Laszlo Zsolt Nagy, Oct 12, 2005, in forum: Python
    Replies:
    0
    Views:
    299
    Laszlo Zsolt Nagy
    Oct 12, 2005
  4. defn noob

    Writing my own wireless-driver, hard?

    defn noob, Aug 25, 2008, in forum: C Programming
    Replies:
    6
    Views:
    506
  5. Replies:
    0
    Views:
    373
Loading...

Share This Page