Net::HTTPResponse#read_body usable for timing?

D

Denis Haskin

[Note: parts of this message were removed to make it a legal post.]

I have some simple network performance analysis I need to do; easiest
and most flexible thing to do would be to use Net::HTTP in Ruby to do
this, but I want to make sure I'm getting somewhat realistic results.
(Note since I'm mainly using this for comparisons, not specific timings,
the numbers don't need to be *exact*, but would be nice if they were close.)

In any event, I'm using Net::HTTPResponse#read_body since I want to be
able to get intra-download timings, so I've got something like:

Net::HTTP.start(url.host, url.port) do |http|
http.request_get('/test.txt') do |res|
# start timer
res.read_body do |fragment|
# stop timer, calculate stats for this interval & fragment size
# restart timer
end
end
end

The numbers I'm getting seem *somewhat* right but vary a lot,
particularly because each fragment is pretty small (doc says it's as it
comes from the socket).

Does anyone know if read_body is synchronous wrt to the network reads?
E.g. is it realistic to use this for timing this way? I'll do some
comparisons with wget, but any help would be appreciated.

Thanks,

dwh
 
B

Brian Candler

Denis said:
Does anyone know if read_body is synchronous wrt to the network reads?

The source will be installed somewhere like
/usr/lib/ruby/1.8/net/http.rb - it's all in Ruby and fairly easy to
understand. Look for read_body and read_body_0.

However, you can see it relies on some Ruby internals, such as
Socket#read calling << on the dest buffer object; see ReadAdapter in
net/protocol.rb (*)

Basically, you'll get a separate Socket#read call for each chunk if
you're getting HTTP/1.1 chunked encoding for your result body, and a
single Socket#read call for non-chunked with Content-Length. tcpdump
should show what you're actually getting.

What I don't know is, if you do Socket#read(1_000_000, dest) is whether
Ruby breaks the read up into smaller ones and calls << onto dest for
each chunk, and if so, what buffer size it's using. That would involve
looking at the C source code.

HTH,

Brian.

(*) In fact Net::HTTP violates Ruby's own documentation, which says
explicitly that the result buffer may only be a String object.

---------------------------------------------------------------- IO#read
ios.read([length [, buffer]]) => string, buffer, or nil
 
D

Denis Haskin

[Note: parts of this message were removed to make it a legal post.]

Thanks, Brian -- I was figuring I would probably have to dive into the
source to look at this, thanks for the roadmap. I'll probably look at
it over the holidays for fun ;-)

dwh


Brian said:
Denis said:
Does anyone know if read_body is synchronous wrt to the network reads?

The source will be installed somewhere like
/usr/lib/ruby/1.8/net/http.rb - it's all in Ruby and fairly easy to
understand. Look for read_body and read_body_0.

However, you can see it relies on some Ruby internals, such as
Socket#read calling << on the dest buffer object; see ReadAdapter in
net/protocol.rb (*)

[...]
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,743
Messages
2,569,478
Members
44,898
Latest member
BlairH7607

Latest Threads

Top