Benchmarks Project and example for `||=`

[Intransition]

Has anyone ever put together a suite of benchmarks that simply measure
interesting Ruby practices? For example, here is a comparison of using
||= vs not using it.

n = 1000000

puts "#{n} Times"

t = Time.now

@b = :b
n.times do
@b
end

puts "Simple Read : #{Time.now - t} seconds"

t = Time.now

n.times do
@a ||= :a
end

puts " Or Equals : #{Time.now - t} seconds"

On `ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]`:

1000000 Times
Simple Read : 0.395487 seconds
Or Equals : 0.508428 seconds

On `ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-linux]`:

1000000 Times
Simple Read : 0.121280356 seconds
Or Equals : 0.183629043 seconds

Might be an interesting project.

 

[Ammar Ali]

[Note: parts of this message were removed to make it a legal post.]

Has anyone ever put together a suite of benchmarks that simply measure
interesting Ruby practices? For example, here is a comparison of using
||= vs not using it.

n = 1000000

puts "#{n} Times"

t = Time.now

@b = :b
n.times do
@b
end

puts "Simple Read : #{Time.now - t} seconds"

t = Time.now

n.times do
@a ||= :a
end

puts " Or Equals : #{Time.now - t} seconds"

On `ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]`:

1000000 Times
Simple Read : 0.395487 seconds
Or Equals : 0.508428 seconds

On `ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-linux]`:

1000000 Times
Simple Read : 0.121280356 seconds
Or Equals : 0.183629043 seconds

Might be an interesting project.

It would be interesting to have performance comparisons of the various
idioms in common use.

In these examples, however, it's not surprising that the first loop (without
||=) would be faster. It's not doing anything really. The second involves
testing wether the variable is defined or not, every iteration.

Regards,
Ammar

 

[Intransition]

It would be interesting to have performance comparisons of the various
idioms in common use.

In these examples, however, it's not surprising that the first loop (with= out
||=3D) would be faster. It's not doing anything really. The second involv= es
testing wether the variable is defined or not, every iteration.

Yep. I wrote it b/c I wanted to know *how much* slower using ||=3D is.

 

[Jeremy Bopp]

Yep. I wrote it b/c I wanted to know *how much* slower using ||= is.

The issue is one of relevance though. Of course using ||= is slower
than simply taking whatever value is in the variable, but why do you
care in the first place?

If you just wanted to take the value of a variable, you would never use
||= in the first place. If you have a need to assign a value to the
variable only if it's not already defined though, then you need ||= or
one of its less idiomatic equivalents.

The point of comparison profiling like this is to find ways to improve
your code without changing its ultimate function. Your comparison here
compares two fundamentally different operations. A more relevant
comparison would be as follows:

n = 1000000

puts "#{n} Times"

t = Time.now

@c = :c
n.times do
@c = :c unless @c
@c
end

puts " Optional set then get (verbose) : #{Time.now - t} seconds"

t = Time.now

@b = :b
n.times do
@b = @b || :b
end

puts "Optional set then get (less verbose) : #{Time.now - t} seconds"

t = Time.now

n.times do
@a ||= :a
end

puts " Optional set then get (idiomatic) : #{Time.now - t} seconds"


ruby 1.9.2p0 (2010-08-18) [i386-mingw32]
1000000 Times
Optional set then get (verbose) : 0.125004 seconds
Optional set then get (less verbose) : 0.140628 seconds
Optional set then get (idiomatic) : 0.156254 seconds

ruby 1.8.7 (2010-06-23 patchlevel 299) [i386-mingw32]
1000000 Times
Optional set then get (verbose) : 0.20313 seconds
Optional set then get (less verbose) : 0.218756 seconds
Optional set then get (idiomatic) : 0.187505 seconds

In this light, the idiomatic method compares favorably; however, I do
find it odd that Ruby 1.9.2 seems to take a hit on the relative
performance of the idiomatic method as compared to Ruby 1.8.7.

-Jeremy

 

[Rick DeNatale]

The point of comparison profiling like this is to find ways to improve
your code without changing its ultimate function. =A0Your comparison here
compares two fundamentally different operations

It struck me as odd that Trans is benchmarking two different things
against each other.

As another example, I just benchmarked the time it takes me to get up
out of my sofa and walk to my front door and back at about 30 seconds.
That's over 3 times FASTER than Fernando Alonso's fastest lap of
1:47.976 in the Singapore Grand Prix a couple of Sunday's ago.

So does that make me faster than a Ferrari F10, I don't think so, and
I doubt that it has a practical significance.

--=20
Rick DeNatale

Help fund my talk at Ruby Conf 2010:http://pledgie.com/campaigns/13677
Blog: http://talklikeaduck.denhaven2.com/
Github: http://github.com/rubyredrick
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 

[Intransition]

It struck me as odd that Trans is benchmarking two different things
against each other.

As another example, =A0I just benchmarked the time it takes me to get up
out of my sofa and walk to my front door and back at about 30 seconds.
=A0That's over 3 times FASTER than Fernando Alonso's fastest lap of
1:47.976 in the Singapore Grand Prix a couple of Sunday's ago.

So does that make me faster than a Ferrari F10, I don't think so, and
I doubt that it has a practical significance.

I think this is a good a reminder as any that it's hard to see what
someone else is getting at when you're not seeing the use case. There
is in fact a very good reason for my comparison. I often do:

class X
def f
@f ||=3D ( do_some_calc_or_lookup )
end
end

I like this idiom as it provides the speed bump of caching and yet
encapsulates the entire procedure for getting the value in one place.
However, if I am really concerned about squeezing out every bit of
speed, I would do:

class X
def initialize
@f =3D ( do_some_calc_or_lookup )
end
def f
@f
end
end

The purpose of the benchmark was to see just how much "squeeze" I am
getting in the exchange.

 

[Jeremy Bopp]

I think this is a good a reminder as any that it's hard to see what
someone else is getting at when you're not seeing the use case. There
is in fact a very good reason for my comparison. I often do:

class X
def f
@f ||= ( do_some_calc_or_lookup )
end
end

I like this idiom as it provides the speed bump of caching and yet
encapsulates the entire procedure for getting the value in one place.
However, if I am really concerned about squeezing out every bit of
speed, I would do:

class X
def initialize
@f = ( do_some_calc_or_lookup )
end
def f
@f
end
end

The purpose of the benchmark was to see just how much "squeeze" I am
getting in the exchange.

Take a look at
http://ruby-doc.org/docs/ProgrammingRuby/html/classes.html. On that
page you'll find some examples for defining a method named "once".
Search for "ExampleDate.once" under the Class and Module Definitions
section.

You can use the "once" method as a directive that will ensure that a
given method which may be expensive to compute is only run 1 time after
which the cached value will be returned. I made a generalized version
of "once" as suggested there for use in an internal project where I
work. That way I get the best of delayed computation, clear code, and
fast results for a few methods that really need it.

It might be interesting to profile the performance of a method wrapped
by "once" compared to your other solutions.

-Jeremy

 

[Ryan Davis]

However, if I am really concerned about squeezing out every bit of
speed, I would do:
=20
class X
def initialize
@f =3D ( do_some_calc_or_lookup )
end
def f
@f
end
end

There is a damn good reason why lazy initializing exists in the first =
place, and it invalidates your statement above.

If you were __really__ concerned about squeezing out every bit of speed, =
you'd profile the code to see how it is used. If #f isn't being called =
in a tight loop, or isn't even being called much at all, then you might =
be spending more time running initialize than you would have doing the =
lazy initialize. You'll never know that mentarbating over =
micro-benchmarks.

 

[Intransition]

There is a damn good reason why lazy initializing exists in the first pla=

ce, and it invalidates your statement above.

How?

If you were __really__ concerned about squeezing out every bit of speed, =

you'd profile the code to see how it is used. If #f isn't being called in a=
tight loop, or isn't even being called much at all, then you might be spen=
ding more time running initialize than you would have doing the lazy initia=
lize. You'll never know that mentarbating over micro-benchmarks.

You assume I am not aware of how frequently X.new is bring called in
comparison to X#f. In my case, its about 1 to N where N >=3D 1, and so
forth for X#f2, X#f3, X#f4, etc. If #f wasn't being used but on rare
occasion, I wouldn't worry about it. Moreover, by knowing the relative
difference between the two, one can make a better effort at optimizing
upfront, and potentially save time later.

And yes I do indeed profile code when I am "__really__ concerned about
squeezing out every bit of speed". In fact, if memory serves, I was
doing exactly that when I was trying to decide if giving up the
convenience of ||=3D was worth the performance gain.