storing values for later use

[progcat]

I have a group of expensive (as in slow) c functions that take an
arrays and
parameters, process them and return one or more arrays to Ruby.

These functions can take a long time to process and in some cases are
quite complex, so I really would like to keep them in c.

Example: My program calculates the first 1000 elements of a function.
My c function can calculate all 1000 elements
in about the same time as it take to calculate the 999'th element, so
rather calling c over and over is way too slow!

This function can have 1 to 6 parameters that are unknown until run
time
so I can't just "set them up" in advance.
I was thinking about making a hash so I could do something like
funcvals[(func_name + parm1.to_s + parm2.to_s
parm3.to_s).intern].storedvalue
but this is slow, ugly and messy!

Does anyone have a clever/fast idea how to do this best in Ruby?
(I think this is the "opposite" of a lazy function. Is there a term
for this
so I can sound smarter next time?)

Thanks,
Tom

 

[Axel Etzold]

-------- Original-Nachricht --------

Datum: Sun, 8 Jun 2008 05:53:49 +0900
Von: (e-mail address removed)
An: (e-mail address removed)
Betreff: storing values for later use

I have a group of expensive (as in slow) c functions that take an
arrays and
parameters, process them and return one or more arrays to Ruby.

These functions can take a long time to process and in some cases are
quite complex, so I really would like to keep them in c.

Example: My program calculates the first 1000 elements of a function.
My c function can calculate all 1000 elements
in about the same time as it take to calculate the 999'th element, so
rather calling c over and over is way too slow!

This function can have 1 to 6 parameters that are unknown until run
time
so I can't just "set them up" in advance.
I was thinking about making a hash so I could do something like
funcvals[(func_name + parm1.to_s + parm2.to_s
parm3.to_s).intern].storedvalue
but this is slow, ugly and messy!

Does anyone have a clever/fast idea how to do this best in Ruby?
(I think this is the "opposite" of a lazy function. Is there a term
for this
so I can sound smarter next time?)

Thanks,
Tom

Tom,

not quite sure whether I understood what you're after, but it might be
Marshalling:

http://www.ruby-doc.org/core/classes/Marshal.html

Best regards,

Axel

 

[David Masover]

Example: My program calculates the first 1000 elements of a function.
My c function can calculate all 1000 elements
in about the same time as it take to calculate the 999'th element, so
rather calling c over and over is way too slow!

This function can have 1 to 6 parameters that are unknown until run
time
so I can't just "set them up" in advance.

Depends what you mean by "in advance".

Sounds like you had the right idea with the hash, but probably better to wrap
it in some sort of object. For example:

class BigCStuff
...
def function_I_want_to_put_off *args
@argument_lists << args
end
...
def [] i
self.result_array
end
def result_array
@result_array ||= actual_C_call(@argument_lists)
end
end

I'm trying not to make too many assumptions about what you're doing here...

(I think this is the "opposite" of a lazy function.

By the way, what I've done above is actually a lazy object of sorts.

 

[Robert Klemme]

I have a group of expensive (as in slow) c functions that take an
arrays and
parameters, process them and return one or more arrays to Ruby.

These functions can take a long time to process and in some cases are
quite complex, so I really would like to keep them in c.

Example: My program calculates the first 1000 elements of a function.
My c function can calculate all 1000 elements
in about the same time as it take to calculate the 999'th element, so
rather calling c over and over is way too slow!

This function can have 1 to 6 parameters that are unknown until run
time
so I can't just "set them up" in advance.
I was thinking about making a hash so I could do something like
funcvals[(func_name + parm1.to_s + parm2.to_s
parm3.to_s).intern].storedvalue
but this is slow, ugly and messy!

I'd say you'd better use an array as Hash key.

Does anyone have a clever/fast idea how to do this best in Ruby?
(I think this is the "opposite" of a lazy function. Is there a term
for this
so I can sound smarter next time?)

Sounds like memoize would fit.

Kind regards

robert

 

[progcat]

Thanks for the quick reply.

I have no desire to store values except in the class while my program
is running.

f(a,b,c) returns a 1000 elemeent array. I want to call the function
with a=1,b=2 and c = -3). It takes ten seconds for it to call the c
function and fill the array.

I later call the same functions with different values of a,b and c.

Then I decide I want the values of the function with a=1, b=2 and c =
-3 again. I want
to avoid calling C again and recalculating the numbers I already have
calculated.

example: (sorry if this is not "quite" ruby, I am still learning!)
funcname = "thiscfunction"
x = myclass.callc(funcname,1,2,3)[55]
#first call, get all values for a=1,b=2,c=3 and return the
55'th
y = myclass.callc(funcname,3,2,1)[1] # new args, so C has to be
called again
z = myclass.callc(funcname,1,2,3)[11]
# second call with 1,2,3, don't call C again just lookup and the
11'th value

The program has many function and argument combinations (they are
defined at runtime)
but chances are I will reuse them many times withing the program, so
this litterally could make my program thousands of times faster.

Sorry if i did a poor job in explaining this.

Thanks,
Tom
Tom

 

[nicholasmabry]

Hey Tom,

The previous responders were right on. Memoization is the concept of
caching the results of a function call. Googling 'ruby memoization'
will yield far better descriptions than mine, but here's a simple
example using the ruby gem Memoize. When it is mixed-in to a class
definition, it adds the method memoize() to instances of the class.
When you want to begin storing results of a calculation, call memoize
on the object with the symbol name (the colon string) of the method
you'd like to begin caching.

require 'memoize'

class CalcLibrary
include Memoize
def call_c(function_name, *args_array)
# This method calls into the C extension (I assume)
# And returns an array (we'll just return randoms)
return_array = []
srand Time.now.usec
return_array << rand(10000)
return_array << rand(10000)
end
end

# A quick test
puts "-- No memoize() call yet --"
clib = CalcLibrary.new

puts "Result one:"
puts clib.call_c("rand")[1]
puts "Result two:"
puts clib.call_c("rand")[1]
puts "Result three:"
puts clib.call_c("rand", 2)[1]

puts "-- Memoized --"
clib.memoize :call_c

puts "Result one:"
puts clib.call_c("rand")[1]
puts "Result two:" # Should be the same as result 1 if
cached
puts clib.call_c("rand")[1]
puts "Result three:"
puts clib.call_c("rand", 2)[1]


The Memoize module also includes the ability to store and load caches
out to a file, but it's easy to get into trouble that way without
diving into how it's working under-the-hood. Good luck with your
project!