custom serialization problem 2 (marshal_dump, marshal_load)

[sayoyo Sayoyo]

Hi,

I have a problem with the marshal_dump and marshal_load...
I built two classes XXX, YYY and implemented both methods like this:

class YYY
@data = "string"
@version = 1

def marshal_dump()
return [@version,@data]
end

def marshal_load(var)
@version = var[0]
case @version
when 1
@data = var[1]
else
#do something else
end
end
end

class XXX
@data = Array of YYY
@version = 1

def marshal_dump()
return [@version,@data]
end

def marshal_load(var)
@version = var[0]
case @version
when 1
@data = var[1]
else
#do something else
end
end
end

The serialization works correctly when the number of objects YYY is
small, but when it got more than 5000, the serialization doesn't work.
(the serializaed file has more and less 1 meg)


When I try to not to use marshal_dump and marshal_load. the
serialization works again????

Can someone tell me, where is the difference between customized
serialization and no-customized one???

Thanks you very much

Sayoyo

 

[Joel VanderWerf]

Hi,

I have a problem with the marshal_dump and marshal_load...
I built two classes XXX, YYY and implemented both methods like this:

class YYY
@data = "string"
@version = 1

def marshal_dump()
return [@version,@data]
end

def marshal_load(var)
@version = var[0]
case @version
when 1
@data = var[1]
else
#do something else
end
end
end

class XXX
@data = Array of YYY
@version = 1

def marshal_dump()
return [@version,@data]
end

def marshal_load(var)
@version = var[0]
case @version
when 1
@data = var[1]
else
#do something else
end
end
end

The serialization works correctly when the number of objects YYY is
small, but when it got more than 5000, the serialization doesn't work.
(the serializaed file has more and less 1 meg)


When I try to not to use marshal_dump and marshal_load. the
serialization works again????

Can someone tell me, where is the difference between customized
serialization and no-customized one???

Thanks you very much

Sayoyo

What version of ruby? What platform?

Same problem when you serialize to memory instead of file?

What exactly is the observed problem? Crash, exception, data error?

The following version of your code seems to work:

class YYY
attr_reader :data, :version

def initialize
@data = "string"
@version = 1
end

def marshal_dump()
return [@version,@data]
end

def marshal_load(var)
@version = var[0]
case @version
when 1
@data = var[1]
else
#do something else
end
end
end

class XXX
attr_reader :data, :version

def initialize
@data = (0..1000).map { YYY.new }
@version = 1
end

def marshal_dump()
return [@version,@data]
end

def marshal_load(var)
@version = var[0]
case @version
when 1
@data = var[1]
else
#do something else
end
end
end

xxx = XXX.new

x2 = Marshal.load(Marshal.dump(xxx))

p x2.data[777]

File.open('/tmp/foo', "wb") do |f|
Marshal.dump(xxx, f)
end

x3 = File.open('/tmp/foo', "rb") do |f|
Marshal.load(f)
end

p x3.data[888]

 

[Joel VanderWerf]

@data = (0..1000).map { YYY.new }

I tried it for 1_000_000 with no problem, and that generated a 1.4Mb file.

Ruby 1.8.6-p114, linux.

 

[sayoyo Sayoyo]

Hi,

I'm using ruby 1.8.6 (2007-09-24 patchlevel 111) [x86_64-linux] and
Rails 1.2.5

The problem I have, is the class XXX should have an array of YYY as
data, but it turn out I got only NIL when I deserialized, then I got an
exception for unable to use [] for a NIL. It happens only when I have a
lot of data.

The example I posted here is an simplification of my data, actually I
have 5 classes, the structure:
A--
|-Array of B
|- C

B--
|-Array of D

C--
|-Array of E

and each of them have more and less 10 attributes... When I have 5000 of
B, the serialized file has almost 1 meg. And I also realized each time I
serialized the same data, the file got a different size each time...

How can I verify the serailization?

 

[Joel VanderWerf]

Hi,

I'm using ruby 1.8.6 (2007-09-24 patchlevel 111) [x86_64-linux] and
Rails 1.2.5

The problem I have, is the class XXX should have an array of YYY as
data, but it turn out I got only NIL when I deserialized, then I got an
exception for unable to use [] for a NIL. It happens only when I have a
lot of data.

The example I posted here is an simplification of my data, actually I
have 5 classes, the structure:
A--
|-Array of B
|- C

B--
|-Array of D

C--
|-Array of E

and each of them have more and less 10 attributes... When I have 5000 of
B, the serialized file has almost 1 meg. And I also realized each time I
serialized the same data, the file got a different size each time...

How can I verify the serailization?

I would try to come up with a minimal characterization of the
problem--minimize the classes, attrs, objects so that if you take away
anything else the problem goes away. That should help point to the problem.

Also, you could try running this on other systems and ruby versions....