[ANN] Rails demo application for web page scraping


David Tran

Hi all

I wrote an application in Ruby on Rails which breaks up web pages and
allow you to weave these pieces together into your own personal page,
similar to a tapestry. I would like to invite interested programmers and
web users alike to take a look and, if possible, give me some
feedbacks/criticisms so I can improve the site. The goal here is to
stretch Ruby on Rails to see what it can really do, maybe growing the
site into a showcase for this new technology. There are good examples of
AJAX, drag and drop, and other DHTML techniques too.

I have had various success in scraping web pages, some are easier to
scrape than others. Some do not look right at all, but generally, sites
that have lots of tables of dynamic contents are the best to scrape
(like Yahoo Financial pages, for instance). Site with a lot of CSS
positioning are more difficult to do.

I plan to do a few more features like automatic data feeds, database
population, code generation (Java/Ruby/SQL) etc. of the scraped data,
but all depends on if there is enough interests, as I have a day job to
go to, so can only do this in my spare time.

Anyway, the URL for the site is http://www.boxaweb.com/.

Thank you for your time.


Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Latest member

Latest Threads