Serialization ruby 1.8 vs ruby 1.9

[Florian Odronitz]

Hi,

I am trying to marshal objects with ruby 1.8 and read them with 1.9 but
I get an error on date objects:
ruby 1.8:
f=File.new('/tmp/date', 'w+'); f.puts Marshal.dump(Date.today); f.close

ruby 1.9:
m=File.read('/tmp/date'); Marshal.load(m)

gives me:
class Date needs to have method `_load' (TypeError)

The versions of Marshal are identical, but in 1.8, Date has a _dump and
_load method, in 1.9 it does not.

Am I missing something or is this a bug?
Any hints are appreciated.

Thanks,
Florian

 

[Robert Klemme]

Hi,

I am trying to marshal objects with ruby 1.8 and read them with 1.9 but
I get an error on date objects:
ruby 1.8:
f=File.new('/tmp/date', 'w+'); f.puts Marshal.dump(Date.today); f.close

ruby 1.9:
m=File.read('/tmp/date'); Marshal.load(m)

gives me:
class Date needs to have method `_load' (TypeError)

The versions of Marshal are identical, but in 1.8, Date has a _dump and
_load method, in 1.9 it does not.

Am I missing something or is this a bug?
Any hints are appreciated.

James is right: the format of Marshal is not guaranteed to be portable
across Ruby versions. Nevertheless some types do work.

Btw, Marshal's format is binary. Which means two things

1. you should open files in binary mode,

2. printing methods like #puts are not guaranteed to work.

If you observe these rules you can make it work with Time...

robert@fussel:~$ ruby1.8 -e
'File.open("x","wb"){|i|Marshal.dump(Time.now,i)}'
robert@fussel:~$ ls -l x
-rw-r--r-- 1 robert robert 18 2010-01-07 13:44 x
robert@fussel:~$ ruby19 -e 'p(File.open("x","rb"){|i|Marshal.load(i)})'
2010-01-07 13:44:40 +0100
robert@fussel:~$

.... but apparently neither Date nor DateTime:

robert@fussel:~$ ruby1.8 -r date -e
'File.open("x","wb"){|i|Marshal.dump(Date.today,i)}'
robert@fussel:~$ ruby19 -r date -e
'p(File.open("x","rb"){|i|Marshal.load(i)})'
-e:1:in `load': class Date needs to have method `_load' (TypeError)
from -e:1:in `block in <main>'
from -e:1:in `open'
from -e:1:in `<main>'
robert@fussel:~$

robert@fussel:~$ ruby1.8 -r date -e
'File.open("x","wb"){|i|Marshal.dump(DateTime.now,i)}'
robert@fussel:~$ ruby19 -r date -e
'p(File.open("x","rb"){|i|Marshal.load(i)})'
-e:1:in `load': class DateTime needs to have method `_load' (TypeError)
from -e:1:in `block in <main>'
from -e:1:in `open'
from -e:1:in `<main>'
robert@fussel:~$

It does work for Date and DateTime when only using one version

robert@fussel:~$ ruby19 -r date -e
'File.open("x","wb"){|i|Marshal.dump(DateTime.now,i)}'
robert@fussel:~$ ruby19 -r date -e
'p(File.open("x","rb"){|i|Marshal.load(i)})'
#<DateTime: 2010-01-07T13:54:15+01:00
(2356995876177376393/960000000000,1/24,2299161)>
robert@fussel:~$ ruby19 -r date -e
'File.open("x","wb"){|i|Marshal.dump(Date.today,i)}'
robert@fussel:~$ ruby19 -r date -e
'p(File.open("x","rb"){|i|Marshal.load(i)})'
#<Date: 2010-01-07 (4910407/2,0,2299161)>
robert@fussel:~$

You have quite a few options:

1. use a type that works, e.g. String:

robert@fussel:~$ ruby1.8 -r date -e
'File.open("x","wb"){|i|Marshal.dump(Date.today.strftime,i)}'
robert@fussel:~$ ruby19 -r date -e
'p(Date.strptime(File.open("x","rb"){|i|Marshal.load(i)}))'
#<Date: 2010-01-07 (4910407/2,0,2299161)>
robert@fussel:~$

2. create custom Marshalling and demarshalling methods which use types
that work.

3. create your custom date type which encapsulates a Date but uses
customized Marshal serialization.

You can see an example for customized persistence in section "Custom
Persistence" on
http://blog.rubybestpractices.com/posts/rklemme/018-Complete_Class.html .

4. you use another serialization mechanism, like Yaml:

robert@fussel:~$ ruby1.8 -r date -r yaml -e
'File.open("x","wb"){|i|YAML.dump(Date.today,i)}'
robert@fussel:~$ ruby19 -r date -r yaml -e
'p(File.open("x","rb"){|i|YAML.load(i)})'
#<Date: 2010-01-07 (4910407/2,0,2299161)>
robert@fussel:~$

5. use yet another completely different format.

There are probably more options...

Kind regards

robert

 

[Florian Odronitz]

Thank you for your very informative answers.

I was missing the thing with the binary mode. In the real use case I am
writing to and reading from Memcache.

I tried to serialize the data with YAML and JSON which turned out to be
far too slow.

I followed your suggestion and am now converting dates to strings and
parsing them at the receiving end which works fine.

Cheers,
Florian

 

[Robert Klemme]

Thank you for your very informative answers.

You're welcome!

I was missing the thing with the binary mode. In the real use case I am
writing to and reading from Memcache.

I tried to serialize the data with YAML and JSON which turned out to be
far too slow.

I followed your suggestion and am now converting dates to strings and
parsing them at the receiving end which works fine.

If performance is crucial you should probably check parsing performance.
I remember I did the parsing myself once because #strptime was too
slow. Ah, found it:

http://github.com/rklemme/muppet-laboratories/blob/master/bin/sample-animal.rb

If you can better use a numeric type like Time#to_i or Time#to_f. It
may be that you can get at the rational inside Date.

irb(main):001:0> d=Date.today
=> #<Date: 2010-01-07 (4910407/2,0,2299161)>
irb(main):002:0> d.instance_variables
=> [sg, of, ajd, __ca__]
irb(main):003:0> d.instance_variables.map {|iv| d.instance_variable_get(iv)}
=> [2299161, 0, (4910407/2), {252120=>2455204, 258504=>[2010, 1, 7]}]

Then you "only" need to transform that back into a Date.

Kind regards

robert