[SOLUTION] SerializableProc (#38)

[Robin Stocker]

Hi,

This is the second solution that I could finish in time. Well, it was
pretty easy.

I imagine my solution is not very fast, as each time a method on the
SerializableProc is called, a new Proc object is created.
The object could be saved in an instance variable @proc so that speed is
only low on the first execution. But that would require the definition
of custom dump methods for each Dumper so that it would not attempt to
dump @proc.

Here's my solution...
(Question: Is it better if I attach it or just paste it like this?)


class SerializableProc

def initialize( block )
@block = block
# Test if block is valid.
to_proc
end

def to_proc
# Raises exception if block isn't valid, e.g. SyntaxError.
eval "Proc.new{ #{@block} }"
end

def method_missing( *args )
to_proc.send( *args )
end

end


if $0 == __FILE__

require 'yaml'
require 'pstore'

code = SerializableProc.new %q{ |a,b| [b,a] }

# Marshal
File.open('proc.marshalled', 'w') { |file| Marshal.dump(code, file) }
code = File.open('proc.marshalled') { |file| Marshal.load(file) }

p code.call( 1, 2 )

# PStore
store = PStore.new('proc.pstore')
store.transaction do
store['proc'] = code
end
store.transaction do
code = store['proc']
end

p code.call( 1, 2 )

# YAML
File.open('proc.yaml', 'w') { |file| YAML.dump(code, file) }
code = File.open('proc.yaml') { |file| YAML.load(file) }

p code.call( 1, 2 )

p code.arity

end

 

[Ryan Leavengood]

And mine's attached to this mail.
=20
I wrote this a while ago and it works by extracting a proc's origin fil= e=20
name and line number from its .inspect string and using the source code= =20
(which usually does not have to be read from disc) -- it works with=20
procs generated in IRB, eval() calls and regular files. It does not wor= k=20
from ruby -e and stuff like "foo".instance_eval "lambda {}".source=20
probably doesn't work either.
=20
Usage:
=20
code =3D lambda { puts "Hello World" }
puts code.source
Marshal.load(Marshal.dump(code)).call
YAML.load(code.to_yaml).call

Interesting. I was considering taking this approach until I realized I'd=20
have to implement a partial Ruby parser, which is what I see you did.=20
Still, it is pretty cool, though obviously a bit hackish.

I wonder if YARV and Ruby byte-code will make it easier for procs to be=20
serialized? I'm not sure how the binding would work (hmmm, if it is just=20
objects maybe they could be serialized as normal), but the proc itself=20
could just be serialized as is if it is self-contained Ruby byte-code.

Does anyone know if this is how YARV will be? Because I'm just guessing=20
here.

Ryan

 

[Dominik Bathon]

def to_proc
# Raises exception if block isn't valid, e.g. SyntaxError.
eval "Proc.new{ #{@block} }"
end

def method_missing( *args )
to_proc.send( *args )
end

Nice idea, to avoid storing the Proc object in an instance variable and s=
o =20
being able to just use the default serializing. But I guess this is quite=
=20
slow ;-)

So, here is my solution. It should be almost as fast as normal procs, but=
=20
I had to implement custom serializing methods. I also implemented a custo=
m =20
=3D=3D, because that doesn't really work with method_missing/delegate.


require "delegate"
require "yaml"

class SProc < DelegateClass(Proc)

attr_reader roc_src

def initialize(proc_src)
super(eval("Proc.new { #{proc_src} }"))
@proc_src =3D proc_src
end

def =3D=3D(other)
@proc_src =3D=3D other.proc_src rescue false
end

def inspect
"#<SProc: #{@proc_src.inspect}>"
end
alias :to_s :inspect

def marshal_dump
@proc_src
end

def marshal_load(proc_src)
initialize(proc_src)
end

def to_yaml(opts =3D {})
YAML::quick_emit(self.object_id, opts) { |out|
out.map("!rubyquiz.com,2005/SProc" ) { |map|
map.add("proc_src", @proc_src)
}
}
end

end

YAML.add_domain_type("rubyquiz.com,2005", "SProc") { |type, val|
SProc.new(val["proc_src"])
}

if $0 =3D=3D __FILE__
require "pstore"

code =3D SProc.new %q{ |*args|
puts "Hello world"
print "Args: "
p args
}

orig =3D code

code.call(1)

File.open("proc.marshalled", "w") { |file| Marshal.dump(code, file) =
}
code =3D File.open("proc.marshalled") { |file| Marshal.load(file) }

code.call(2)

store =3D PStore.new("proc.pstore")
store.transaction do
store["proc"] =3D code
end
store.transaction do
code =3D store["proc"]
end

code.call(3)

File.open("proc.yaml", "w") { |file| YAML.dump(code, file) }
code =3D File.open("proc.yaml") { |file| YAML.load(file) }

code.call(4)

p orig =3D=3D code
end

 

[James Edward Gray II]

Here's my solution...

Here's what I came up with while building the quiz:

class SerializableProc
def self._load( proc_string )
new(proc_string)
end

def initialize( proc_string )
@code = proc_string
@proc = nil
end

def _dump( depth )
@code
end

def method_missing( method, *args )
if to_proc.respond_to? method
@proc.send(method, *args)
else
super
end
end

def to_proc( )
return @proc unless @proc.nil?

if @code =~ /\A\s*(?:lambda|proc)(?:\s*\{|\s+do).*(?:\}|end)
\s*\Z/
@proc = eval @code
elsif @code =~ /\A\s*(?:\{|do).*(?:\}|end)\s*\Z/
@proc = eval "lambda #{@code}"
else
@proc = eval "lambda { #{@code} }"
end
end

def to_yaml( )
@proc = nil
super
end
end

(Question: Is it better if I attach it or just paste it like this?)

It doesn't much matter, but I favor inlining it when it's a single file.

James Edward Gray II

 

[Dave Burt]

Proc's documentation tells us that "Proc objects are blocks of code that
have been bound to a set of local variables." (That is, they are "closures"
with "bindings".) Do any of the proposed solutions so far store local
variables?

# That is, can the following Proc be serialized?
local_var = 42
code = proc { local_var += 1 } # <= what should that look like in YAML?
code.call #=> 43
File.open("proc.marshalled", "w") { |file| Marshal.dump(code, file) }

# New context, e.g. new file:
code = File.open("proc.marshalled") { |file| Marshal.load(file) }
code.call #=> 44
local_var #=> NameError - undefined here

AFAICT, the only one is Christian Neukirchen's Nodewrap suggestion, which
looks very cool. From <http://rubystuff.org/nodewrap/>:

Sample code
This will dump the class Foo (including its instance methods, class
variables, etc.) and re-load it as an anonymous class:
class Foo
def foo; puts "this is a test..."; end
end

s = Marshal.dump(Foo)
p Marshal.load(s) #=> #<Class 0lx4027be20>

Here's another, trickier test for SerializableProcs. Can multiple Procs
sharing context, as returned by the following method, be made to behave
consistently across serialization? If the Procs are serialized
independently, I believe this is impossible - an inherent problem with the
idea of serializing Procs (or anything with shared context).
def two_procs
x = 1
[proc { x }, proc { x += 1 }]
end

p1, p2 = two_procs
[p1.call, p2.call, p1.call, p2.call] #=> [1, 2, 2, 3]
q1, q2 = Marshal.load(Marshal.dump(p1)), Marshal.load(Marshal.dump(p2))
[q1.call, q2.call, q1.call, q2.call] #=> [3, 4, 4, 5]
# I expect Nodewrap can get [3, 4, 3, 5] for this last result.

Dave

 

[Ryan Davis]

Granted, we cheated, quite a bit at that, but I think the solution we
came up with is pretty:

require 'r2c_hacks'

class ProcStore # We have to have this because yaml calls allocate on
Proc
def initialize(&proc)
@p = proc.to_ruby
end

def call(*args)
eval(@p).call(*args)
end
end

code = ProcStore.new { |x| return x+1 }
=> #<ProcStore:0x3db25c @p="proc do |x|\n return (x + 1)\nend">

The latest release of ZenHacks added Proc.to_ruby among other things.
Granted, it doesn't preserve the actual closure, just the code, but
it looks like that is a limitation of the other solutions as well, so
we aren't crying too much.

Our original solution just patched Proc and added _load/_store on it,
but it choked on the YAML serialization side of things. Not entirely
sure why, and we were too tired to care at the time.

To see what we do to implement Proc.to_ruby:

class Proc
ProcStoreTmp = Class.new unless defined? ProcStoreTmp
def to_ruby
ProcStoreTmp.senddefine_method, :myproc, self)
m = ProcStoreTmp.new.methodmyproc)
result = m.to_ruby.sub!(/def myproc\(([^\)]+)\)/, 'proc do |\1|')
return result
end
end