memoize and yaml

B

Brian Buckley

Hello,

I am using the "memoize" module to eliminate having to redo complex,
time-consuming calculations. It works great. However, when I yamlize
my "memoized" object off to the database and then retrieve it back,
all the memoizing is lost.

My Ruby skills are still in development. In addition to normal
yamlizing, how could one also yamlize the memoize information, so that
yamlizing and back retains the methods that are to be memoized and
also retains any prior saved (memoized) calculations?

The test below on Foo illustrates what I am trying to do. Also below
is the whole Memoize module.

Any tips or pointers appreciated. Thanks!

--Brian Buckley


require 'test/unit'
require 'yaml'
require 'memoize'

class TestMemoizeAndYaml < Test::Unit::TestCase
class Foo
include Memoize
attr_reader :x, :y
def initialize(x, y)
@x, @y =3D x, y
end
def calc1(a, b)
sleep 1 # fakes big, complex calc
ans =3D calc2 + a + b
end
def calc2(c =3D 0)
sleep 1 # another big, complex calc
ans =3D x + y
end
end

def exercise_foo(foo)
3.times{ foo.calc1 10,20 }
end

def test_memoize_and_yaml

foo =3D Foo.new(1, 2) # first a foo without memoizing
exercise_foo(foo) # without memoizing exercise_foo takes 6 seconds
- good

# now memoize it
foo.memoize :calc1 # memoize calc1
foo.memoize :calc2 # and a 2nd method
exercise_foo(foo) # good - now it takes only 2 secs because of the memo=
izing
exercise_foo(foo) # and now no seconds because of the memoizing

foo =3D YAML::load(foo.to_yaml) # to the database and back
exercise_foo(foo) # but now exercise_foo is taking 6 secs again
# how can I retain memoized state?
# need to retain both the methods and the hashes of calculation histori=
es
end
end

#FYI... here is the Memoize module
module Memoize
MEMOIZE_VERSION =3D "1.1.0"
def memoize(name)
meth =3D method(name)
cache =3D {}
(class << self; self; end).class_eval do
define_method(name) do |*args|
cache.has_key?(args) ? cache[args] : cache[args] ||=3D
meth.call(*args)
end
end
cache
end
end
 
F

Florian Groß

Brian said:
I am using the "memoize" module to eliminate having to redo complex,
time-consuming calculations. It works great. However, when I yamlize
my "memoized" object off to the database and then retrieve it back,
all the memoizing is lost. [...]

#FYI... here is the Memoize module
module Memoize
MEMOIZE_VERSION = "1.1.0"
def memoize(name)
meth = method(name)
cache = {}
(class << self; self; end).class_eval do
define_method(name) do |*args|
cache.has_key?(args) ? cache[args] : cache[args] ||=
meth.call(*args)
end
end
cache
end
end

This memoize module stores the cache in a local variable that can be
referenced from within the memoize method wrapper.

It also memoizes the methods at the instance level meaning that after
storing and loading an instance the methods won't be memoized anymore.

Here's a version that stores the cache in an instance variable and that
replaces the method definitions at class definition time:

module Memoize
def memoize(name)
name = name.to_sym
old_method = instance_method(name)
remove_method(name)

define_method(name) do |*args|
@cache ||= {}
signature = [name] + args

if @cache.include?(signature) then
@cache[signature]
else
@cache[signature] = old_method.bind(self).call(*args)
end
end
end
end

Please note that you can use neither of these memoize() methods for
memoizing methods that can take a block argument as it will be dropped.

Oh, and if you call a memoized method with an argument that can't be
serialized by YAML this definition will probably cause trouble.
 
B

Brian Buckley

old_method =3D instance_method(name)

I see the intent. I have not worked at it yet but I dropped your code
in as you wrote it and I am getting a NoMethodError: undefined method
`instance_method' on the above line.

--Brian
 
F

Florian Groß

Brian said:
I see the intent. I have not worked at it yet but I dropped your code
in as you wrote it and I am getting a NoMethodError: undefined method
`instance_method' on the above line.

Make sure to use the code on a module or class:

class Foo
extend Memoize

def foo(x)
puts "Calculating #{x} ** 32"
x ** 32
end
memoize :foo
end
 
B

Brian Buckley

class Foo
extend Memoize

def foo(x)
puts "Calculating #{x} ** 32"
x ** 32
end
memoize :foo
end

Got it. I had used "include" rather than "extend". Your solution
applies memoization to all objects of a class. The original memoize
applied memoize on a per object basis.

I need to test and digest.
 
D

Daniel Berger

Brian said:
Got it. I had used "include" rather than "extend". Your solution
applies memoization to all objects of a class. The original memoize
applied memoize on a per object basis.

I need to test and digest.

Would folks like both options? memoize_all or something? Or just leave it alone?

- Dan
 
S

Sean O'Halpin

Would folks like both options? memoize_all or something? Or just leave =
it alone?

With a slight adjustment, Florian's version can be used at the top
level too (at the cost of the instance variable @cache in main):

module Memoize
def memoize(name)
name =3D name.to_sym
if self.to_s =3D=3D "main"
old_method =3D class<<self;self;end.instance_method(name)
klass =3D Object
else
old_method =3D instance_method(name)
klass =3D self
end
klass.send:)remove_method, name)
klass.send:)define_method, name) do |*args|
@cache ||=3D {}
signature =3D [name] + args

if @cache.include?(signature) then
@cache[signature]
else
@cache[signature] =3D old_method.bind(self).call(*args)
end
end
end
end

class D
extend Memoize
attr_accessor :value
def initialize(value)
@value =3D value
end
def t_memo(*args)
puts "calculating #{args.inspect}"
args.map { |x| x * value }
end
memoize:)t_memo)

end

d =3D D.new(10)
p d.t_memo(7,8)
p d.t_memo(7,8)

p d.t_memo(1,2)
p d.t_memo(1,2)

e =3D D.new(20)
p e.t_memo(1,2)
p e.t_memo(7,8)
p d.t_memo(7,8)

p d.instance_eval { @cache }

p e.instance_eval { @cache }

extend Memoize

def t_memo(*args)
puts "calculating #{args.inspect}"
args.map { |x| x * 5 }
end

memoize :t_memo

p t_memo(1,2,3)
p t_memo(1,2,3)
p t_memo(7,8)

p @cache


__END__
calculating [7, 8]
[70, 80]
[70, 80]
calculating [1, 2]
[10, 20]
[10, 20]
calculating [1, 2]
[20, 40]
calculating [7, 8]
[140, 160]
[70, 80]
{[:t_memo, 1, 2]=3D>[10, 20], [:t_memo, 7, 8]=3D>[70, 80]}
{[:t_memo, 1, 2]=3D>[20, 40], [:t_memo, 7, 8]=3D>[140, 160]}
calculating [1, 2, 3]
[5, 10, 15]
[5, 10, 15]
calculating [7, 8]
[35, 40]
{[:t_memo, 7, 8]=3D>[35, 40], [:t_memo, 1, 2, 3]=3D>[5, 10, 15]}

Regards,

Sean
 
B

Brian Buckley

module Memoize
def memoize(name)
name =3D name.to_sym
old_method =3D instance_method(name)
remove_method(name)

define_method(name) do |*args|
@cache ||=3D {}
signature =3D [name] + args

if @cache.include?(signature) then
@cache[signature]
else
@cache[signature] =3D old_method.bind(self).call(*args)
end
end
end
end

Two questions of this solution. (It works and is what I need now so
I'm just looking to round out Ruby skills.)

1 Why is the 'remove_method(name)' line necessary? Does not the next
define_method call get rid of the method anyway by overriding?

2 The module requires use of 'extend' instead of 'include'. Is there
a comparable 'include' solution, and (if there is) would it be
preferred because, for example, one could not use this solution on a
class that already extends something else?

Thanks!

Brian Buckley
 
F

Florian Groß

Brian said:
module Memoize
def memoize(name)
name = name.to_sym
old_method = instance_method(name)
remove_method(name)

define_method(name) do |*args|
@cache ||= {}
signature = [name] + args

if @cache.include?(signature) then
@cache[signature]
else
@cache[signature] = old_method.bind(self).call(*args)
end
end
end
end

Two questions of this solution. (It works and is what I need now so
I'm just looking to round out Ruby skills.)

1 Why is the 'remove_method(name)' line necessary? Does not the next
define_method call get rid of the method anyway by overriding?

Overriding methods usually produces warnings. It's best to remove the
methods first.
2 The module requires use of 'extend' instead of 'include'. Is there
a comparable 'include' solution, and (if there is) would it be
preferred because, for example, one could not use this solution on a
class that already extends something else?

Actually, I think this is the cleanest solution. In theory you could
also offer a custom append_features() which defines the methods on the
class or module including Memoize itself. That's more complex and not as
easy to understand, though, in my opinion.
 
B

Brian Buckley

Overriding methods usually produces warnings. It's best to remove the
methods first.

I was wondering if warnings might be the motive. I am glad to get confirma=
tion.
Actually, I think this is the cleanest solution. In theory you could
also offer a custom append_features() which defines the methods on the
class or module including Memoize itself. That's more complex and not as
easy to understand, though, in my opinion.

Cool. I had been thinking I wanted to use your version of Memoize in
conjunction with Rails' ActiveRecord::Base but ActiveRecord::Base
requires one to 'extend' it to use it. However it occurred to me that
since ActiveRecord::Base does not extend anything I could just extend
it instead of my actual class. It seems to work.

class ActiveRecord::Base
extend Memoize
end

class Foo < ActiveRecord::Base
def calc
...
end
memoize :calc
end

--Brian Buckley
 
F

Florian Groß

Brian said:
Cool. I had been thinking I wanted to use your version of Memoize in
conjunction with Rails' ActiveRecord::Base but ActiveRecord::Base
requires one to 'extend' it to use it. However it occurred to me that
since ActiveRecord::Base does not extend anything I could just extend
it instead of my actual class. It seems to work.

class ActiveRecord::Base
extend Memoize
end

class Foo < ActiveRecord::Base
def calc
...
end
memoize :calc
end

Please note that extend is not the same thing as inheriting from another
class. You can inherit from ActiveRecord::Base and still have a
extend(Memoize).
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

memoize 5
memoize to a file 13
memoize 1
memoize (reposting via ML) 4
declaratively caching results of a method 23
Probs with accessing variables in yaml file 2
status of cache in Memoizable 3
YAML and NArray 1

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top