[ANN] scRUBYt! 0.2.3 - Hpricot and Mechanize on steroids

Discussion in 'Ruby' started by Peter Szinek, Feb 21, 2007.

  1. Peter Szinek

    Peter Szinek Guest

    Hello,

    I am pleased to announce that the new release of scRUBYt!, 0.2.3 is
    available for download.

    scRUBYt! is a very easy to learn and use, yet powerful Web scraping
    framework based on Hpricot and mechanize. It's purpose is to free you
    from the drudgery of web page crawling, looking up HTML tags,
    attributes, XPaths, form names and other typical low-level web scraping
    woes by figuring these out from your examples copy'n'pasted from the Web
    page.

    The current release has a lot of new features, tons of bugfixes and
    some shiny new examples - scraping reddit, del.icio.us, rubyforge login,
    wordpress automatic commenting for example.

    Thanks everybody for the great feedback!

    Cheers,
    Peter
    __
    http://www.rubyrailways.com :: Ruby and Web2.0 blog
    http://scrubyt.org :: Ruby web scraping framework
    http://rubykitchensink.ca/ :: The indexed archive of all things Ruby.
    Peter Szinek, Feb 21, 2007
    #1
    1. Advertising

  2. Peter Szinek

    Guest

    Re: scRUBYt! 0.2.3 - Hpricot and Mechanize on steroids

    Peter,

    I really, really like scRUBYt! so far, especially after so much
    scraping using PHP, however the lack of documentation kills me. Are
    there going to be more tutorials and examples soon? When I was first
    testing it I could not believe it was possible to scrape google with
    only a few lines, but what about more complicated pages? For example,
    it isn't possible to scrape Ask.com the same way you do Google because
    of it's markup, what are you supposed to do on those cases?

    I know how scraping works but I'm not very experienced with XPath so
    it would be really good to have more examples (in their final form,
    not the learner only because they stop working after a while for most
    of the time), plus a more detailed explanation of everything that can
    be done.

    Either way, this seems to be shaping up very well and I wish you good
    luck with it!

    Cheers

    On Feb 21, 7:46 am, Peter Szinek <> wrote:
    > Hello,
    >
    > I am pleased to announce that the new release of scRUBYt!, 0.2.3 is
    > available for download.
    >
    > scRUBYt! is a very easy to learn and use, yet powerful Web scraping
    > framework based on Hpricot and mechanize. It's purpose is to free you
    > from the drudgery of web page crawling, looking up HTML tags,
    > attributes, XPaths, form names and other typical low-level web scraping
    > woes by figuring these out from your examples copy'n'pasted from the Web
    > page.
    >
    > The current release has a lot of new features, tons of bugfixes and
    > some shiny new examples - scraping reddit, del.icio.us, rubyforge login,
    > wordpress automatic commenting for example.
    >
    > Thanks everybody for the great feedback!
    >
    > Cheers,
    > Peter
    > __http://www.rubyrailways.com:: Ruby and Web2.0 bloghttp://scrubyt.org:: Ruby web scraping frameworkhttp://rubykitchensink.ca/:: The indexed archive of all things Ruby.
    , Feb 21, 2007
    #2
    1. Advertising

  3. Peter Szinek

    Peter Szinek Guest

    Re: scRUBYt! 0.2.3 - Hpricot and Mechanize on steroids

    wrote:
    > Peter,
    >
    > I really, really like scRUBYt! so far, especially after so much
    > scraping using PHP, however the lack of documentation kills me. Are
    > there going to be more tutorials and examples soon? When I was first
    > testing it I could not believe it was possible to scrape google with
    > only a few lines, but what about more complicated pages? For example,
    > it isn't possible to scrape Ask.com the same way you do Google because
    > of it's markup, what are you supposed to do on those cases?


    There are tons of examples here:

    http://rubyforge.org/frs/download.php/17684/scrubyt-examples-0.2.3.zip

    I am also planning to finish the tutorials and add even more docs.
    What's up with ask.com? Send me what would you like to accomplish and I
    will help you with it.


    >
    > I know how scraping works but I'm not very experienced with XPath so
    > it would be really good to have more examples (in their final form,
    > not the learner only because they stop working after a while for most
    > of the time),

    OK - however, you can always replace the examples with the current ones
    and export the extractor to get a production one.

    plus a more detailed explanation of everything that can
    > be done.

    Yeah, that's my goal too, just I am flooded with everything else... But
    I am
    working on it all the time. Please subscribe to the feed at
    http://scrubyt.org
    if you would like to be notified if new stuff arrives... I am announcing
    everything there,

    > Either way, this seems to be shaping up very well and I wish you good
    > luck with it!

    Great! Please send as much feedback as possible so I can improve the
    whole stuff and add what's missing.

    Cheers,
    Peter

    __
    http://www.rubyrailways.com :: Ruby and Web2.0 blog
    http://scrubyt.org :: Ruby web scraping framework
    http://rubykitchensink.ca/ :: The indexed archive of all things Ruby.
    Peter Szinek, Feb 21, 2007
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dema
    Replies:
    0
    Views:
    101
  2. Peter Szinek
    Replies:
    0
    Views:
    127
    Peter Szinek
    Feb 5, 2007
  3. Replies:
    6
    Views:
    315
    Stefan Mahlitz
    Aug 16, 2007
  4. Ehud Rosenberg
    Replies:
    2
    Views:
    141
    Ehud Rosenberg
    Nov 14, 2007
  5. Cy Gar
    Replies:
    6
    Views:
    226
    Cy Gar
    May 19, 2008
Loading...

Share This Page