Concerning Marshalling

Discussion in 'Ruby' started by christophe.poucet@gmail.com, Sep 24, 2005.

  1. Guest

    Hello,

    I was trying to marshal some objects, however some of these objects
    contain references to singletons which of course can't be marshalled.
    That's why I tried something as follows:

    class Container
    end

    class SpecialContainer < Container
    include Singleton
    end

    class Test
    attr_accessor :container
    def _dump(depth)
    old = @container
    if old == SpecialContainer.instance
    @container = nil
    end
    result = super
    @container = old
    return result
    end
    def self._load(str)
    super
    if @container == nil
    @container = SpecialContainer.instance
    end
    self
    end
    end

    Sadly, however, it seems there is no default implementation for _dump,
    which means that in this case the super call results in a
    NoMethodError. Is there any other way to resolve this problem without
    custom-coding a full _dump method?

    Regards,
    Christophe
     
    , Sep 24, 2005
    #1
    1. Advertising

  2. ts Guest

    >>>>> "c" == christophe poucet@gmail com <> writes:

    c> That's why I tried something as follows:

    Perhaps best to use marshal_dump/marshal_load

    Guy Decoux
     
    ts, Sep 25, 2005
    #2
    1. Advertising

  3. ts wrote:
    >>>>>>"c" == christophe poucet@gmail com <> writes:

    >
    >
    > c> That's why I tried something as follows:
    >
    > Perhaps best to use marshal_dump/marshal_load
    >
    > Guy Decoux


    That's a good suggestion. The marshal_dump/marshal_load pair can be very
    useful, because it integrates well with the whole Marshal.dump or
    Marshal.load process. An inherent problem with _dump and _load is that
    you have to use strings, so you are responsible for somehow preserving
    and restoring references to other objects that are also being dumped.
    This is critical when the original dump call was on an object that
    referred to your object, not directly to your object.

    With marshal_dump/marshal_load, you produce/consume an arbitrary
    Marshallable object, possibly containing references, and all the code in
    marshal.c takes over the hard task of preserving and restoring
    references. Probably an example will say it better:

    class Node
    attr_accessor :prev_node, :next_node

    attr_accessor :temp_data
    # don't want this to persist (maybe it's a cache, or simply
    # undumpable, like a proc, or a file, or a singleton).

    def marshal_dump
    dumped_obj = [prev_node, next_node]
    dumped_obj
    end

    def marshal_load(dumped_obj)
    @prev_node, @next_node = dumped_obj
    end
    end

    node1 = Node.new
    node2 = Node.new
    node1.next_node = node2
    node2.prev_node = node1
    node1.temp_data = STDOUT
    node2.temp_data = proc {"foo"}

    p node1

    p Marshal.load(Marshal.dump(node1))

    __END__

    output:

    #<Node:0xb7dff098 @next_node=#<Node:0xb7dfef08
    @prev_node=#<Node:0xb7dff098 ...>, @temp_data=#<Proc:0xb7dff37c@-:23>>,
    @temp_data=#<IO:0xb7e4c078>>
    #<Node:0xb7dfed64 @prev_node=nil, @next_node=#<Node:0xb7dfed28
    @prev_node=#<Node:0xb7dfed64 ...>, @next_node=nil>>

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    Joel VanderWerf, Sep 25, 2005
    #3
  4. Ara.T.Howard Guest

    On Sun, 25 Sep 2005, wrote:

    > Hello,
    >
    > I was trying to marshal some objects, however some of these objects
    > contain references to singletons which of course can't be marshalled.
    > That's why I tried something as follows:
    >
    > class Container
    > end
    >
    > class SpecialContainer < Container
    > include Singleton
    > end
    >
    > class Test
    > attr_accessor :container
    > def _dump(depth)
    > old = @container
    > if old == SpecialContainer.instance
    > @container = nil
    > end
    > result = super
    > @container = old
    > return result
    > end
    > def self._load(str)
    > super
    > if @container == nil
    > @container = SpecialContainer.instance
    > end
    > self
    > end
    > end
    >
    > Sadly, however, it seems there is no default implementation for _dump,
    > which means that in this case the super call results in a
    > NoMethodError. Is there any other way to resolve this problem without
    > custom-coding a full _dump method?
    >
    > Regards,
    > Christophe


    for some reason the mailing list seemed to have eaten my post about this so
    here it is again - sorry if it's a dup:


    harp:~ > cat dumpable_singleton.rb

    module DumpableSingleton
    module InstanceMethods
    attr_accessor 'ikey'
    def _dump *a, &b
    Marshal::dump ikey, *a, &b
    end
    def == other
    self.ikey == other.ikey
    end
    alias_method 'eql?', '=='
    end
    module ClassMethods
    def new *a, &b
    ikey = []
    ikey.push *a
    ikey.push b if b
    if instances.has_key? ikey
    instances[ ikey ]
    else
    begin
    ikey.each{|k| Marshal::dump k}
    rescue TypeError, Exception
    raise ArgumentError, "all arguments must be dumpable!"
    end
    instances[ (obj = super(*a, &b)).ikey = ikey ] = obj
    end
    end
    def _load buf, *a, &b
    new(*(ikey = Marshal::load(buf, *a, &b)))
    end
    attr_accessor 'instances'
    end
    def self::included other
    other.module_eval{ include InstanceMethods }
    other.extend ClassMethods
    other.instances = {}
    other
    end
    end

    class C
    include DumpableSingleton
    def initialize(*a, &b); end
    end

    x = C::new 42
    y = C::new 42
    z = C::new 43

    puts '-' * 42
    C::instances.each {|k,c| p k => c}
    p x == y
    p x == z
    p y == z

    xd, yd, zd = [x, y, z].map{|c| Marshal::load(Marshal::dump(c))}

    puts '-' * 42
    C::instances.each {|k,c| p k => c}
    p xd == yd
    p xd == zd
    p yd == zd

    puts '-' * 42
    C::instances.each {|k,c| p k => c}
    p x == xd
    p y == yd
    p z == zd


    harp:~ > ruby dumpable_singleton.rb

    ------------------------------------------
    {[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
    {[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
    true
    false
    false
    ------------------------------------------
    {[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
    {[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
    true
    false
    false
    ------------------------------------------
    {[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
    {[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
    true
    true
    true

    i've used something like this in in a few small projects.

    hth.

    -a
    --
    ===============================================================================
    | email :: ara [dot] t [dot] howard [at] noaa [dot] gov
    | phone :: 303.497.6469
    | Your life dwells amoung the causes of death
    | Like a lamp standing in a strong breeze. --Nagarjuna
    ===============================================================================
     
    Ara.T.Howard, Sep 26, 2005
    #4
  5. Hello,

    Guy and Joel: Thank you for your helpful comments, this was exactly
    what I was looking for and this was sadly not mentioned in the RDoc of
    Marshal :/. For the moment I had been using YAML, but this will not be
    a sustainable solution once the number of objects to dump grows in
    size. Eventually I want to be able to store the objects into a
    database based upon a key that is unique per object (all my classes
    derive from Managed that ensures a unique identifier) As for the
    references being kept, great!

    Ara: Your solution is definitely an interesting one, however I fear
    that it is not a sustainable solution for larger projects and the
    problem was in more than just singletons, singletons being a symptom of
    the problem.

    Cheers,
    Christophe / vincenz
     
    christophe (dot) poucet (at) gmail (dot) com, Sep 27, 2005
    #5
  6. Concerning tracking references.

    I want to store each object as a separate value. I can assure that
    each object has a unique id (all the objects that need to be stored
    separately are instance_of? Managed), however if I marshal the objects
    one by one, then the references they might hold internally to each
    other will of course no long work. How would I solve this problem?
    Obviously I need to replace the references by some sort value that
    refers to it, or otherwise anything that is referred will be stored
    multiple times. However I'm not quite sure how to put the hooks in
    place for this.

    Christophe.
     
    christophe (dot) poucet (at) gmail (dot) com, Sep 27, 2005
    #6
  7. christophe (dot) poucet (at) gmail (dot) com wrote:
    > Concerning tracking references.
    >
    > I want to store each object as a separate value. I can assure that
    > each object has a unique id (all the objects that need to be stored
    > separately are instance_of? Managed), however if I marshal the objects
    > one by one, then the references they might hold internally to each
    > other will of course no long work. How would I solve this problem?
    > Obviously I need to replace the references by some sort value that
    > refers to it, or otherwise anything that is referred will be stored
    > multiple times. However I'm not quite sure how to put the hooks in
    > place for this.
    >
    > Christophe.
    >


    Here's one way:

    class Node
    @@nodes = Hash.new {|h,k| raise ArgumentError, "Missing node
    #{k.inspect}"}

    # The id of a node can be anything but +nil+.
    attr_reader :id

    def initialize id
    @id = id
    @@nodes[id] = self
    end

    attr_accessor :prev_node, :next_node

    attr_accessor :temp_data
    # don't want this to persist (maybe it's a cache, or simply
    # undumpable, like a proc, or a file, or a singleton).

    def marshal_dump
    [prev_node, next_node].map {|node| node && node.id}
    end

    def marshal_load(dumped_obj)
    @prev_node, @next_node = dumped_obj.map do |node_id|
    node_id.nil? ? nil : @@nodes[node_id]
    end
    end

    def self.delete id
    @@nodes.delete id
    end
    end

    node1 = Node.new "One"
    node2 = Node.new 2

    node1.next_node = node2
    node2.prev_node = node1

    node1.temp_data = STDOUT
    node2.temp_data = proc {"foo"}

    p node1
    node1_dumped = Marshal.dump(node1)
    p Marshal.load(Marshal.dump(node1))

    Node.delete 2
    p Marshal.load(node1_dumped)
    # This should raise an exception, to show that the referred node is not
    # stored in node1_dumped, and not in the @@nodes hash.

    __END__

    #<Node:0xb7dfe260 @temp_data=#<IO:0xb7e4c078>, @id="One",
    @next_node=#<Node:0xb7dfe24c
    @temp_data=#<Proc::40>, @id=2,
    @prev_node=#<Node:0xb7dfe260 ...>>>
    #<Node:0xb7dfdeb4 @next_node=#<Node:0xb7dfe24c
    @temp_data=#<Proc::40>, @id=2,
    @prev_node=#<Node:0xb7dfe260 @temp_data=#<IO:0xb7e4c078>, @id="One",
    @next_node=#<Node:0xb7dfe24c ...>>>, @prev_node=nil>
    marshal-refs2.rb:2: Missing node 2 (ArgumentError)
    from marshal-refs2.rb:2:in `call'
    from marshal-refs2.rb:24:in `default'
    from marshal-refs2.rb:24:in `[]'
    from marshal-refs2.rb:24:in `marshal_load'
    from marshal-refs2.rb:23:in `map'
    from marshal-refs2.rb:23:in `marshal_load'
    from marshal-refs2.rb:47:in `load'
    from marshal-refs2.rb:47


    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    Joel VanderWerf, Oct 1, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    607
  2. Peter Flynn
    Replies:
    2
    Views:
    454
    Peter Flynn
    Aug 9, 2004
  3. Terence
    Replies:
    2
    Views:
    371
    Frank Schmitt
    Nov 12, 2003
  4. dev_chandok

    Marshalling in COM

    dev_chandok, Oct 26, 2004, in forum: C++
    Replies:
    1
    Views:
    718
    red floyd
    Oct 26, 2004
  5. gk
    Replies:
    8
    Views:
    10,458
    Kent Paul Dolan
    May 24, 2006
Loading...

Share This Page