Help interpreting benchmark results

[Juan Alvarez]

I ran this simple benchmark:

Benchmark.bm do |x|
x.report('Curl::Easy.http_get') do
Curl::Easy.http_get(text)
end

x.report('RestClient.get') do
RestClient.get(text)
end
end

RestClient uses the standard Net:HTTP library so the benchmark is really
trying to compare Net::HTTP vs curb (libcurl bindings).

These are the results I get:

user system total real
Curl::Easy.http_get 0.070000 0.130000 0.200000 ( 26.319104)
RestClient.get 0.970000 0.580000 1.550000 ( 23.557923)

I get these results consistently where curb's user, system and total
times are dramatically lower than Net::HTTP's. However, Net:HTTP is
reported to run faster by the real column. Shouldn't the reported real
time be consistent with the other columns?

I'm running this benchmark on a 2.4Ghz Core 2 Duo with 2GB RAM MacBook
Pro, if that helps.

Thanks,
Juan

 

[Sandor Szücs]

These are the results I get:

user system total =20
real
Curl::Easy.http_get 0.070000 0.130000 0.200000 ( 26.319104)
RestClient.get 0.970000 0.580000 1.550000 ( 23.557923)

I get these results consistently where curb's user, system and total
times are dramatically lower than Net::HTTP's. However, Net:HTTP is
reported to run faster by the real column. Shouldn't the reported real
time be consistent with the other columns?

No. "real" measures the time that is passed from start to finish.

Pseudocode
real_start =3D Time.now
yield # run test
real_end =3D Time.now
real_start.to_f - real_end.to_f


If the Kernel of your OS schedules another processes for, say 20 =20
seconds,
then the difference between total and real is >=3D 20 seconds.
I think the time called "real" should not be trusted in an evaluation.

regards, Sandor Sz=FCcs
--

 

[Robert Klemme]

No. "real" measures the time that is passed from start to finish.

Pseudocode
=A0real_start =3D Time.now
=A0yield # run test
=A0real_end =3D Time.now
=A0real_start.to_f - real_end.to_f


If the Kernel of your OS schedules another processes for, say 20 seconds,
then the difference between total and real is >=3D 20 seconds.
I think the time called "real" should not be trusted in an evaluation.

Not so fast. If another process has been running that may well be a
process spawned by our process.

13:19:09 Temp$ ruby19 bb.rb
user system total real
other process 0.000000 0.000000 0.139000 ( 10.375000)
in process 4.594000 0.000000 4.594000 ( 4.765000)
13:19:30 Temp$ cat bb.rb
require 'benchmark'

Benchmark.bm 20 do |b|
b.report 'other process' do
system *%w{sleep 10}
end

b.report 'in process' do
10_000_000.times {|i| i + 1}
end
end
13:19:35 Temp$

I'd check the curl lib - I would assume that it uses the command line
utility of the same name - which then is another process.

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end

 

[Sandor Szücs]

Not so fast. If another process has been running that may well be a
process spawned by our process.

Full ack that's right, if a process forks, like system() do you can =20
measure
that one too.

Benchmark.bm(20) do |b|
b.report 'fork' do
pid =3D Process.fork { sleep 3 }
Process.waitpid pid
end
end
user system total real
fork 0.000000 0.010000 0.020000 ( 3.008273)


regards, Sandor Sz=FCcs
--