Marshal's handling of floats

[Brian Palmer]

I was thinking about writing a patch to modify how Marshal handles
floats, right now it dumps them using sprintf(3) and stores the
resulting string in the Marshal stream. I'd like to see it handle
floats the same way that Array#pack does:

[400.53].pack('g').length == 4
[400.53].pack('G').length == 8

while

Marshal.dump(400.53).length - 3 == 22
(and is slower, to boot)

I want to make sure, though, that this would be an acceptable patch.
I can't think why it would be OK for Array#pack to work this way and
not Marshal, but is there any particular reason why it can't be done?
Obviously it would break backwards compatability with older Marshal
dumps, but I don't think they're often used for long-term storage,
are they?

-- Brian Palmer

 

[ara.t.howard]

I was thinking about writing a patch to modify how Marshal handles floats,
right now it dumps them using sprintf(3) and stores the resulting string in
the Marshal stream. I'd like to see it handle floats the same way that
Array#pack does:

[400.53].pack('g').length == 4
[400.53].pack('G').length == 8

while

Marshal.dump(400.53).length - 3 == 22
(and is slower, to boot)

I want to make sure, though, that this would be an acceptable patch. I can't
think why it would be OK for Array#pack to work this way and not Marshal, but
is there any particular reason why it can't be done? Obviously it would break
backwards compatability with older Marshal dumps, but I don't think they're
often used for long-term storage, are they?

-- Brian Palmer

i've never tried to use marshaled data across a big and little endian machine
- but this would break it. consider drb: if you had a mac and a linux box
talking on the wire you might see

harp:~ > ruby -e' puts [1.44417819733316e-41].pack("g").reverse.unpack("g")[0].to_i '
42

which could be confusing. then again maybe i'm overlooking something.

cheers.

-a

 

[Brian Palmer]

Hi Ara,

The 'g' and 'G' flags for Array#pack/String#unpack are in network
byte order, so they're in a platform-independent format, as far as I
know. I actually tested this by Packing a couple thousand floats on
my Mac, sending them in UDP packets and unpacking them on my AMD64
desktop, they all came across correctly.

-- Brian

I was thinking about writing a patch to modify how Marshal handles
floats, right now it dumps them using sprintf(3) and stores the
resulting string in the Marshal stream. I'd like to see it handle
floats the same way that Array#pack does:

[400.53].pack('g').length == 4
[400.53].pack('G').length == 8

while

Marshal.dump(400.53).length - 3 == 22
(and is slower, to boot)

I want to make sure, though, that this would be an acceptable
patch. I can't think why it would be OK for Array#pack to work
this way and not Marshal, but is there any particular reason why
it can't be done? Obviously it would break backwards compatability
with older Marshal dumps, but I don't think they're often used for
long-term storage, are they?

-- Brian Palmer

i've never tried to use marshaled data across a big and little
endian machine
- but this would break it. consider drb: if you had a mac and a
linux box
talking on the wire you might see

harp:~ > ruby -e' puts [1.44417819733316e-41].pack
("g").reverse.unpack("g")[0].to_i '
42

which could be confusing. then again maybe i'm overlooking something.

cheers.

-a
--
suffering increases your inner strength. also, the wishing for
suffering
makes the suffering disappear.
- h.h. the 14th dali lama

 

[Robert Klemme]

Hi Ara,

The 'g' and 'G' flags for Array#pack/String#unpack are in network
byte order, so they're in a platform-independent format, as far as I
know. I actually tested this by Packing a couple thousand floats on
my Mac, sending them in UDP packets and unpacking them on my AMD64
desktop, they all came across correctly.

Yeah, but I still see serious drawbacks of your proposal:

- it will break existing code (YAML data)

- it's not human readable which is one of the key advantages (YAML files
seem to be frequently written by hand as a nice user interface to config
data)

- it's not portable and it might not be compliant with YAML spec

In sum all these are serious showstoppers for a general change of YAML
behavior.

Another solution - if you want to make use of this: AFAIK you can customize
YAMLfication of classes. If that was true, you can pretty easily perform
the conversion during to conversion to and from YAML. That would be the way
to go IMHO.

Kind regards

robert

 

[Joel VanderWerf]

Yeah, but I still see serious drawbacks of your proposal:

- it will break existing code (YAML data)

- it's not human readable which is one of the key advantages (YAML files
seem to be frequently written by hand as a nice user interface to config
data)

- it's not portable and it might not be compliant with YAML spec

In sum all these are serious showstoppers for a general change of YAML
behavior.

I thought the OP wanted to change the behavior of Marshal, not YAML. Or
are you saying that the YAML rep'n would also have to change so that
both formats have the same lossiness?