Marshal's handling of floats

Discussion in 'Ruby' started by Brian Palmer, Jul 9, 2006.

  1. Brian Palmer

    Brian Palmer Guest

    I was thinking about writing a patch to modify how Marshal handles
    floats, right now it dumps them using sprintf(3) and stores the
    resulting string in the Marshal stream. I'd like to see it handle
    floats the same way that Array#pack does:

    [400.53].pack('g').length == 4
    [400.53].pack('G').length == 8

    while

    Marshal.dump(400.53).length - 3 == 22
    (and is slower, to boot)

    I want to make sure, though, that this would be an acceptable patch.
    I can't think why it would be OK for Array#pack to work this way and
    not Marshal, but is there any particular reason why it can't be done?
    Obviously it would break backwards compatability with older Marshal
    dumps, but I don't think they're often used for long-term storage,
    are they?

    -- Brian Palmer
     
    Brian Palmer, Jul 9, 2006
    #1
    1. Advertising

  2. Brian Palmer

    Guest

    On Sun, 9 Jul 2006, Brian Palmer wrote:

    > I was thinking about writing a patch to modify how Marshal handles floats,
    > right now it dumps them using sprintf(3) and stores the resulting string in
    > the Marshal stream. I'd like to see it handle floats the same way that
    > Array#pack does:
    >
    > [400.53].pack('g').length == 4
    > [400.53].pack('G').length == 8
    >
    > while
    >
    > Marshal.dump(400.53).length - 3 == 22
    > (and is slower, to boot)
    >
    > I want to make sure, though, that this would be an acceptable patch. I can't
    > think why it would be OK for Array#pack to work this way and not Marshal, but
    > is there any particular reason why it can't be done? Obviously it would break
    > backwards compatability with older Marshal dumps, but I don't think they're
    > often used for long-term storage, are they?
    >
    > -- Brian Palmer


    i've never tried to use marshaled data across a big and little endian machine
    - but this would break it. consider drb: if you had a mac and a linux box
    talking on the wire you might see

    harp:~ > ruby -e' puts [1.44417819733316e-41].pack("g").reverse.unpack("g")[0].to_i '
    42

    which could be confusing. then again maybe i'm overlooking something.

    cheers.

    -a
    --
    suffering increases your inner strength. also, the wishing for suffering
    makes the suffering disappear.
    - h.h. the 14th dali lama
     
    , Jul 9, 2006
    #2
    1. Advertising

  3. Brian Palmer

    Brian Palmer Guest

    Hi Ara,

    The 'g' and 'G' flags for Array#pack/String#unpack are in network
    byte order, so they're in a platform-independent format, as far as I
    know. I actually tested this by Packing a couple thousand floats on
    my Mac, sending them in UDP packets and unpacking them on my AMD64
    desktop, they all came across correctly.

    -- Brian

    On Jul 8, 2006, at 11:40 PM, wrote:

    > On Sun, 9 Jul 2006, Brian Palmer wrote:
    >
    >> I was thinking about writing a patch to modify how Marshal handles
    >> floats, right now it dumps them using sprintf(3) and stores the
    >> resulting string in the Marshal stream. I'd like to see it handle
    >> floats the same way that Array#pack does:
    >>
    >> [400.53].pack('g').length == 4
    >> [400.53].pack('G').length == 8
    >>
    >> while
    >>
    >> Marshal.dump(400.53).length - 3 == 22
    >> (and is slower, to boot)
    >>
    >> I want to make sure, though, that this would be an acceptable
    >> patch. I can't think why it would be OK for Array#pack to work
    >> this way and not Marshal, but is there any particular reason why
    >> it can't be done? Obviously it would break backwards compatability
    >> with older Marshal dumps, but I don't think they're often used for
    >> long-term storage, are they?
    >>
    >> -- Brian Palmer

    >
    > i've never tried to use marshaled data across a big and little
    > endian machine
    > - but this would break it. consider drb: if you had a mac and a
    > linux box
    > talking on the wire you might see
    >
    > harp:~ > ruby -e' puts [1.44417819733316e-41].pack
    > ("g").reverse.unpack("g")[0].to_i '
    > 42
    >
    > which could be confusing. then again maybe i'm overlooking something.
    >
    > cheers.
    >
    > -a
    > --
    > suffering increases your inner strength. also, the wishing for
    > suffering
    > makes the suffering disappear.
    > - h.h. the 14th dali lama
    >
     
    Brian Palmer, Jul 9, 2006
    #3
  4. Brian Palmer <> wrote:
    > Hi Ara,
    >
    > The 'g' and 'G' flags for Array#pack/String#unpack are in network
    > byte order, so they're in a platform-independent format, as far as I
    > know. I actually tested this by Packing a couple thousand floats on
    > my Mac, sending them in UDP packets and unpacking them on my AMD64
    > desktop, they all came across correctly.


    Yeah, but I still see serious drawbacks of your proposal:

    - it will break existing code (YAML data)

    - it's not human readable which is one of the key advantages (YAML files
    seem to be frequently written by hand as a nice user interface to config
    data)

    - it's not portable and it might not be compliant with YAML spec

    In sum all these are serious showstoppers for a general change of YAML
    behavior.

    Another solution - if you want to make use of this: AFAIK you can customize
    YAMLfication of classes. If that was true, you can pretty easily perform
    the conversion during to conversion to and from YAML. That would be the way
    to go IMHO.

    Kind regards

    robert
     
    Robert Klemme, Jul 10, 2006
    #4
  5. Robert Klemme wrote:
    > Brian Palmer <> wrote:
    >> Hi Ara,
    >>
    >> The 'g' and 'G' flags for Array#pack/String#unpack are in network
    >> byte order, so they're in a platform-independent format, as far as I
    >> know. I actually tested this by Packing a couple thousand floats on
    >> my Mac, sending them in UDP packets and unpacking them on my AMD64
    >> desktop, they all came across correctly.

    >
    > Yeah, but I still see serious drawbacks of your proposal:
    >
    > - it will break existing code (YAML data)
    >
    > - it's not human readable which is one of the key advantages (YAML files
    > seem to be frequently written by hand as a nice user interface to config
    > data)
    >
    > - it's not portable and it might not be compliant with YAML spec
    >
    > In sum all these are serious showstoppers for a general change of YAML
    > behavior.


    I thought the OP wanted to change the behavior of Marshal, not YAML. Or
    are you saying that the YAML rep'n would also have to change so that
    both formats have the same lossiness?

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
     
    Joel VanderWerf, Jul 10, 2006
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?RWx0b24gUmFiZWxsbw==?=

    Marshal in Asp.Net

    =?Utf-8?B?RWx0b24gUmFiZWxsbw==?=, Feb 29, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    3,509
    =?Utf-8?B?RWx0b24gUmFiZWxsbw==?=
    Feb 29, 2004
  2. Kosio

    Floats to chars and chars to floats

    Kosio, Sep 16, 2005, in forum: C Programming
    Replies:
    44
    Views:
    1,290
    Tim Rentsch
    Sep 23, 2005
  3. Giff
    Replies:
    4
    Views:
    513
    Lionel B
    May 15, 2007
  4. Replies:
    10
    Views:
    534
    Aaron Watters
    Jun 18, 2008
  5. Michael Davis

    Ruby 1.8 and Marshal.load/Marshal.dump

    Michael Davis, Oct 10, 2003, in forum: Ruby
    Replies:
    0
    Views:
    171
    Michael Davis
    Oct 10, 2003
Loading...

Share This Page