Mechanize Download File w/o Loading it All in Memory

Discussion in 'Ruby' started by Raymond O'Connor, Aug 28, 2009.

  1. Is there a way to get Mechanize to download a csv file, without loading
    it all into memory first? As I understand even if I used a pluggable
    parser, it will still download the whole file into memory before saving
    it to a file?

    Thanks!
    --
    Posted via http://www.ruby-forum.com/.
     
    Raymond O'Connor, Aug 28, 2009
    #1
    1. Advertising

  2. Raymond O'Connor

    Ryan Davis Guest

    On Aug 27, 2009, at 20:07 , Raymond O'Connor wrote:

    > Is there a way to get Mechanize to download a csv file, without
    > loading
    > it all into memory first? As I understand even if I used a pluggable
    > parser, it will still download the whole file into memory before
    > saving
    > it to a file?


    I don't actually know if this will solve your problem as I've not used
    it, but this looks like it is on the right track:

    > require 'rubygems'
    > require 'mechanize'
    >
    > agent = WWW::Mechanize.new
    > agent.pluggable_parser.pdf = WWW::Mechanize::FileSaver
    > agent.get('http://example.com/foo.pdf')


    then again, it might not. mechanize doesn't sound like a very good
    tool to be using for csv's in the first place as it can't actually DO
    much with them. Why not use a more straightforward tool like curl?
     
    Ryan Davis, Aug 28, 2009
    #2
    1. Advertising

  3. Ryan Davis wrote:
    > On Aug 27, 2009, at 20:07 , Raymond O'Connor wrote:
    >
    >> Is there a way to get Mechanize to download a csv file, without
    >> loading
    >> it all into memory first? As I understand even if I used a pluggable
    >> parser, it will still download the whole file into memory before
    >> saving
    >> it to a file?

    >
    > I don't actually know if this will solve your problem as I've not used
    > it, but this looks like it is on the right track:
    >
    >> require 'rubygems'
    >> require 'mechanize'
    >>
    >> agent = WWW::Mechanize.new
    >> agent.pluggable_parser.pdf = WWW::Mechanize::FileSaver
    >> agent.get('http://example.com/foo.pdf')

    >
    > then again, it might not. mechanize doesn't sound like a very good
    > tool to be using for csv's in the first place as it can't actually DO
    > much with them. Why not use a more straightforward tool like curl?


    It's a whole process just to get to the page where you can download the
    csv file. I have to log in to the site, request the file, etc. which
    is why I was hoping I could do it in mechanize
    --
    Posted via http://www.ruby-forum.com/.
     
    Raymond O'Connor, Aug 28, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Aaron Watters

    Re: Pdf download using mechanize

    Aaron Watters, Jul 6, 2010, in forum: Python
    Replies:
    0
    Views:
    839
    Aaron Watters
    Jul 6, 2010
  2. Jonathan Dobbie
    Replies:
    7
    Views:
    176
    7stud --
    Nov 10, 2007
  3. Ruby Newbie
    Replies:
    2
    Views:
    98
    Ruby Newbie
    Oct 15, 2008
  4. Adam Akhtar
    Replies:
    0
    Views:
    97
    Adam Akhtar
    Mar 23, 2009
  5. zak
    Replies:
    0
    Views:
    138
Loading...

Share This Page