Feeding data to be PUT with Net::HTTP?

Discussion in 'Ruby' started by Chris Markle, Feb 23, 2009.

  1. Chris Markle

    Chris Markle Guest

    Since I am HTTP PUT'ing large amounts of data (ideally using Net::HTTP)
    I would like to feed data in some way to an HTTP PUT so that I don't
    have to have the entire amount of data in memory before using methods
    like request or send_request to initiate the PUT. I saw one post that
    suggests it can't be done but I thought I'd check again.

    If it can't be done with Net::HTTP, are there other HTTP client
    libraries that I might be able to do this with, and maybe even some
    examples of how to do it?

    Thanks in advance...

    Chris
     
    Chris Markle, Feb 23, 2009
    #1
    1. Advertisements

  2. [Note: parts of this message were removed to make it a legal post.]

    have you tried 'httpclient'?there seems to be a streaming post method
    available:
    http://dev.ctor.org/doc/httpclient/

    L-P
     
    Louis-Philippe, Feb 24, 2009
    #2
    1. Advertisements

  3. Best way to understand this is to go to the source code - probably
    somewhere like /usr/lib/ruby/1.8/net/http.rb on your system.

    def put(path, data, initheader = nil) #:nodoc:
    res = request(Put.new(path, initheader), data)
    res.value unless @newimpl
    res
    end

    Trace it through, this calls req.exec to send the request.

    def exec(sock, ver, path) #:nodoc: internal use only
    if @body
    send_request_with_body sock, ver, path, @body
    elsif @body_stream
    send_request_with_body_stream sock, ver, path, @body_stream
    else
    write_header sock, ver, path
    end
    end

    Looking at send_request_with_body_stream, you can pass any object which
    responds to read(n), such as an open IO stream. If you set
    "Tranfer-Encoding: chunked" then you'll get it posted in chunks too,
    which simplifies buffering at the receiving side (if it also supports
    chunking)

    But it looks like you may need to create your Request object by hand and
    call #body_stream=, because set_body_internal doesn't understand about
    streaming objects. Perhaps something like this (untested):

    req = Net::HTTP::put.new(path)
    req.body_stream = File.open("/path/to/large/file")
    res = Net::HTTP.start(url.host, url.port) {|http|
    http.request(req)
    }

    HTH,

    Brian.
     
    Brian Candler, Feb 25, 2009
    #3
  4. Chris Markle

    Chris Markle Guest

    Brian,
    Thank you very much for such a detailed answer, and for including some
    code snippets. I was able to stream using this suggestion. And your
    comments about "you can pass any object that responds to read(n)" gives
    me other food for thought about this.

    Thanks again...

    Chris
     
    Chris Markle, Feb 25, 2009
    #4
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.