YAML vs. Marshal

Discussion in 'Ruby' started by Bill Atkins, May 3, 2004.

  1. Bill Atkins

    Bill Atkins Guest

    Is there any reason to use Marshal instead of YAML? Is there anything
    Marshal can do that YAML cannot?
     
    Bill Atkins, May 3, 2004
    #1
    1. Advertising

  2. Bill Atkins wrote:
    > Is there any reason to use Marshal instead of YAML? Is there anything
    > Marshal can do that YAML cannot?


    Here's one:

    > Marshal.dump [String, Integer]

    # =>"\004\010[\ac\vStringc\fInteger"
    > YAML.dump [String, Integer]

    ArgumentError: can't dump anonymous class Class

    However, you can get around that--see [ruby-talk:95432].

    Marshal is, in my experience, a bit faster.
     
    Joel VanderWerf, May 3, 2004
    #2
    1. Advertising

  3. Joel VanderWerf wrote:

    > Marshal is, in my experience, a bit faster.


    For small documents, my benchmarks have Marshal running at 3x the speed
    of Ruby's YAML parser. For larger documents, this often decreases to
    2x. For output, I'm sure YAML is quite a bit slower than Marshal. I
    haven't benched it, though.

    I'm also working on the YAML bytecode parser, which should run as
    quickly as Marshal does. Yep.

    Mostly you want to use YAML in place of Marshal if:
    * You need portable serialization.
    * You will be hand-editing your objects.
    * You need to read saved objects (for debugging).

    YAML bytecode will only really solve the first of those.

    _why
     
    why the lucky stiff, May 4, 2004
    #3
  4. Bill Atkins

    Hal Fulton Guest

    why the lucky stiff wrote:
    > Mostly you want to use YAML in place of Marshal if:
    > * You need portable serialization.
    > * You will be hand-editing your objects.
    > * You need to read saved objects (for debugging).
    >
    > YAML bytecode will only really solve the first of those.


    Can you explain this to me? I sense some irony here. You're taking a
    format which is renowned for making persistent objects readable in
    plain text, and you're making it unreadable.

    No flame intended. I like YAML, I use it all the time, and I appreciate
    all your hard work.

    But YAML bytecode does sound to me a bit like a step backward.

    Reminds me a bit of the early days of Java:
    1. Hey, let's make a virtual machine that's PROCESSOR INDEPENDENT!
    2. Hey, let's make a new chip that will run this natively!

    I don't know that they ever did, though.


    Hal
     
    Hal Fulton, May 4, 2004
    #4
  5. Hal Fulton wrote:

    > Can you explain this to me? I sense some irony here. You're taking a
    > format which is renowned for making persistent objects readable in
    > plain text, and you're making it unreadable.


    Well, the bytecode is sort of a sideproject and it's one of those things
    that probably doesn't make sense but actually fills a need. I can't
    think of anything else really that is "one of those things," so I'm
    proud to present a first-of-breed nonsensical necessity.

    Why not provide a wire format that is more efficient than YAML and keep
    its portability? My big use case is this: I love DRb. I want it to
    work across languages. I guess you could do this with plain YAML. But
    we don't really need YAML's readability. And we could use the
    efficiency a bit.

    It took me an afternoon to write the bytecode parser, so I don't think
    it's been expensive. But I want to rewrite it again, having some new
    insights, which will take another afternoon. So, I just need to decide
    if it's worth an afternoon. Keep in mind that summer afternoons are
    prettier than any other kind of afternoon. Perhaps I will wait till
    October.

    > No flame intended. I like YAML, I use it all the time, and I appreciate
    > all your hard work.


    Didn't you just say "No flame intended" in another thread? Seriously.
    It's starting to sound really insincere, Hal.

    _why
     
    why the lucky stiff, May 4, 2004
    #5
  6. Bill Atkins

    Hal Fulton Guest

    why the lucky stiff wrote:

    > Why not provide a wire format that is more efficient than YAML and keep
    > its portability? My big use case is this: I love DRb. I want it to
    > work across languages. I guess you could do this with plain YAML. But
    > we don't really need YAML's readability. And we could use the
    > efficiency a bit.


    OK, I'll buy that.

    > Didn't you just say "No flame intended" in another thread? Seriously.
    > It's starting to sound really insincere, Hal.


    Haha, I noticed that after it was too late.

    Maybe I should introduce a new Usenet abbreviation, NFI.

    <flame>You know, _why, sometimes your code isn't as high-quality as it
    usually is.</flame>


    Hal
     
    Hal Fulton, May 4, 2004
    #6
  7. Hal Fulton wrote:

    > <flame>You know, _why, sometimes your code isn't as high-quality as it
    > usually is.</flame>


    I swear: the only quality to my code is that which Ruby inherently brings.

    _why
     
    why the lucky stiff, May 4, 2004
    #7
  8. On Wednesday, May 5, 2004, 3:35:08 AM, why wrote:

    > Why not provide a wire format that is more efficient than YAML and keep
    > its portability? My big use case is this: I love DRb. I want it to
    > work across languages. I guess you could do this with plain YAML. But
    > we don't really need YAML's readability. And we could use the
    > efficiency a bit.



    Doesn't DRb use Marshal, and isn't Marshal faster than YAML? Won't
    YAML bytecode be no faster than Marshal? Isn't Marshal suitably
    cross-platform?

    No flame intended, but you got a lot of questions to answer :)

    Gavin
     
    Gavin Sinclair, May 4, 2004
    #8
  9. Gavin Sinclair wrote:
    > On Wednesday, May 5, 2004, 3:35:08 AM, why wrote:
    >
    >
    >>Why not provide a wire format that is more efficient than YAML and keep
    >>its portability? My big use case is this: I love DRb. I want it to
    >>work across languages. I guess you could do this with plain YAML. But
    >>we don't really need YAML's readability. And we could use the
    >>efficiency a bit.

    >
    >
    >
    > Doesn't DRb use Marshal, and isn't Marshal faster than YAML? Won't
    > YAML bytecode be no faster than Marshal? Isn't Marshal suitably
    > cross-platform?
    >
    > No flame intended, but you got a lot of questions to answer :)


    I guess _why's suggestion is to make it possible for the drb protocol to
    talk between languages, if someone wanted to develop DRb for Python or
    Perl. But that does raise a lot of questions...
     
    Joel VanderWerf, May 4, 2004
    #9
  10. Bill Atkins

    Ara.T.Howard Guest

    On Wed, 5 May 2004, Joel VanderWerf wrote:

    > Gavin Sinclair wrote:
    > > On Wednesday, May 5, 2004, 3:35:08 AM, why wrote:
    > >
    > >
    > >>Why not provide a wire format that is more efficient than YAML and keep
    > >>its portability? My big use case is this: I love DRb. I want it to
    > >>work across languages. I guess you could do this with plain YAML. But
    > >>we don't really need YAML's readability. And we could use the
    > >>efficiency a bit.

    > >
    > >
    > >
    > > Doesn't DRb use Marshal, and isn't Marshal faster than YAML? Won't
    > > YAML bytecode be no faster than Marshal? Isn't Marshal suitably
    > > cross-platform?
    > >
    > > No flame intended, but you got a lot of questions to answer :)

    >
    > I guess _why's suggestion is to make it possible for the drb protocol to
    > talk between languages, if someone wanted to develop DRb for Python or
    > Perl. But that does raise a lot of questions...


    wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention
    for sure...

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
     
    Ara.T.Howard, May 4, 2004
    #10
  11. Ara.T.Howard wrote:

    >On Wed, 5 May 2004, Joel VanderWerf wrote:
    >
    >
    >> I guess _why's suggestion is to make it possible for the drb protocol to
    >>
    >>talk between languages, if someone wanted to develop DRb for Python or
    >>Perl. But that does raise a lot of questions...
    >>
    >>

    >
    >wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention
    >for sure...
    >
    >

    Reinvention? YAML is already a serialization language. All we're
    talking about is providing a bytecode to allow efficiency where
    readability isn't needed. It's no different from Rite in that regard.
    We don't need to tax the parser all the time.

    You could use the same useful YAML APIs but have Marshal's tightness. I
    don't know about you, but I'm convinced!

    _why
     
    why the lucky stiff, May 4, 2004
    #11
  12. Bill Atkins

    Ara.T.Howard Guest

    On Wed, 5 May 2004, why the lucky stiff wrote:

    > Ara.T.Howard wrote:
    >
    > >On Wed, 5 May 2004, Joel VanderWerf wrote:
    > >
    > >
    > >> I guess _why's suggestion is to make it possible for the drb protocol to
    > >>
    > >>talk between languages, if someone wanted to develop DRb for Python or
    > >>Perl. But that does raise a lot of questions...
    > >>
    > >>

    > >
    > >wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention
    > >for sure...
    > >
    > >

    > Reinvention? YAML is already a serialization language. All we're
    > talking about is providing a bytecode to allow efficiency where
    > readability isn't needed. It's no different from Rite in that regard.
    > We don't need to tax the parser all the time.
    >
    > You could use the same useful YAML APIs but have Marshal's tightness. I
    > don't know about you, but I'm convinced!
    >
    > _why


    don't get me wrong, i'm running four peices of code that use yaml heavily as
    we speak...

    by reinvention i meant the 'object access protocol' bit. i mean, soap is a
    lot more than just serialization, it's __what__ you are serializing (an object
    request) that has had much work done... but i guess if you leveraged drb...
    then who knows...

    my latest yaml project is an nfs accessed on disk priority queue that i'm
    using to distribute jobs to consumers located on 36 machines. the cool thing
    about it is that grep and vi work to modifiy it - too cool.... so - i'm
    definitely a believer...

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
     
    Ara.T.Howard, May 4, 2004
    #12
  13. Bill Atkins

    Jamis Buck Guest

    why the lucky stiff wrote:

    > Reinvention? YAML is already a serialization language. All we're
    > talking about is providing a bytecode to allow efficiency where
    > readability isn't needed. It's no different from Rite in that regard.
    > We don't need to tax the parser all the time.
    >
    > You could use the same useful YAML APIs but have Marshal's tightness. I
    > don't know about you, but I'm convinced!
    >


    I'll speak up in _why's defense here, and say I'm convinced, too.
    Besides: where in Ruby is there only a single way to do anything? It
    certainly can't hurt to have YAML bytecode, and once it is available,
    people will find ways to use it. Possibly even ways that _why didn't
    foresee.

    I say, go for it!

    --
    Jamis Buck

    http://www.jamisbuck.org/jamis

    ruby -h | ruby -e
    'a=[];readlines.join.scan(/-(.)\[e|Kk(\S*)|le.l(..)e|#!(\S*)/) {|r| a <<
    r.compact.first };puts "\n>#{a.join(%q/ /)}<\n\n"'
     
    Jamis Buck, May 4, 2004
    #13
  14. Bill Atkins

    Ara.T.Howard Guest

    On Wed, 5 May 2004, Jamis Buck wrote:

    > I say, go for it!


    i stand corrected. ;-)

    -a
    --
    ===============================================================================
    | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
    | PHONE :: 303.497.6469
    | ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
    | URL :: http://www.ngdc.noaa.gov/stp/
    | TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
    ===============================================================================
     
    Ara.T.Howard, May 5, 2004
    #14
  15. Ara.T.Howard wrote:

    >by reinvention i meant the 'object access protocol' bit. i mean, soap is a
    >lot more than just serialization, it's __what__ you are serializing (an object
    >request) that has had much work done... but i guess if you leveraged drb...
    >then who knows...
    >
    >

    Right, yeah. Method dispatch stuff would have to be worked out. That
    would be a bit of reinvention.

    >my latest yaml project is an nfs accessed on disk priority queue that i'm
    >using to distribute jobs to consumers located on 36 machines. the cool thing
    >about it is that grep and vi work to modifiy it - too cool.... so - i'm
    >definitely a believer...
    >

    Your faith is unimpeachable, Ara. And give my best to your 36 machines.

    _why
     
    why the lucky stiff, May 5, 2004
    #15
  16. Bill Atkins

    Kevin M Guest

    > >wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention
    > >for sure...
    > >

    > Reinvention? YAML is already a serialization language. All we're
    > talking about is providing a bytecode to allow efficiency where
    > readability isn't needed.


    I certainly think it would be cool to have both formats -- both
    functionally equivalent, but one being faster and the other being a
    lot handier for debugging.

    Maybe it's not so much reinventing the wheel, as constructing a
    unicycle... :)

    Kevin
     
    Kevin M, May 5, 2004
    #16
  17. Hi,

    I'm interested in this (YAML byte code) project. I have questions, but
    feel free to ignore these questions, because I may be expecting too much.

    why the lucky stiff wrote:
    > My big use case is this: I love DRb. I want it to
    > work across languages.


    Does 'DRb' here mean 'RPC'? 'ORPC'? Or do you expect some DRb specific
    paradigm with your YAML byte code, such as object by reference,
    distributed garbage collection and distributed anonymous code block
    handling? Or more such as message boxing or object activation?

    Does the goal of YAML byte code (+ message invoking?) includes these
    features across languages?

    Currently, Ruby's SOAP marshalling implementation passes all existing
    unit test for Ruby's marshaler (I hope YAML do, too [ruby-talk:95352]).
    So I can use SOAP as wire format of DRb environment as you can see the
    figure in http://rrr.jin.gr.jp/rwiki?cmd=view;name=SOAP-DRb router
    (written in 840 days ago). But it does not mean "cross language DRb
    environment" though it uses SOAP serialization format which is language
    independent. Cross language object environment has some common paradigm
    as I listed above.

    I wrote too much '?'s. I summarize my question. What is the goal of
    YAML byte code project?

    Regards,
    // NaHi
     
    NAKAMURA, Hiroshi, May 5, 2004
    #17
  18. On Wed, 5 May 2004, Hal Fulton wrote:

    > Reminds me a bit of the early days of Java:
    > 1. Hey, let's make a virtual machine that's PROCESSOR INDEPENDENT!
    > 2. Hey, let's make a new chip that will run this natively!
    >
    > I don't know that they ever did, though.


    http://www.sun.com/microelectronics/picoJava/overview.html

    There seems to be a www.picojava.com as well.
    >
    >
    > Hal
    >
    >

    Hugh
     
    Hugh Sasse Staff Elec Eng, May 5, 2004
    #18
  19. On Wednesday 05 May 2004 11:38, Kevin M wrote:
    > > >wouldn't that be soap? ;-) i hate xml, but this would be wheel
    > > > reinvention for sure...

    > >
    > > Reinvention? YAML is already a serialization language. All we're
    > > talking about is providing a bytecode to allow efficiency where
    > > readability isn't needed.

    >
    > I certainly think it would be cool to have both formats -- both
    > functionally equivalent, but one being faster and the other being a
    > lot handier for debugging.
    >
    > Maybe it's not so much reinventing the wheel, as constructing a
    > unicycle... :)

    I agree. Having a couple of standards for one action is always a good thing.
    So you can choose. Choice is great. That's my opinion.

    > Kevin


    --
    sdmitry -=- Dmitry V. Sabanin
    MuraveyLabs.
    Spam Here ->
     
    Dmitry V. Sabanin, May 5, 2004
    #19
  20. Bill Atkins

    Dick Davies Guest

    * Hal Fulton <> [0535 17:35]:

    > Reminds me a bit of the early days of Java:
    > 1. Hey, let's make a virtual machine that's PROCESSOR INDEPENDENT!
    > 2. Hey, let's make a new chip that will run this natively!
    >
    > I don't know that they ever did, though.


    Oh yeah, they built the chip. By the time it came out the API
    had changed, and the hardware had about the power of a 90-Mhz cpu
    right as 300Mhz hit commodity prices.

    Silly billies...

    --
    Rasputin :: Jack of All Trades - Master of Nuns
     
    Dick Davies, May 5, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    10
    Views:
    551
    Aaron Watters
    Jun 18, 2008
  2. Michael Davis

    Ruby 1.8 and Marshal.load/Marshal.dump

    Michael Davis, Oct 10, 2003, in forum: Ruby
    Replies:
    0
    Views:
    192
    Michael Davis
    Oct 10, 2003
  3. RubyQuestions
    Replies:
    0
    Views:
    242
    RubyQuestions
    Dec 3, 2003
  4. Paul Battley

    YAML.dump/YAML.load bug

    Paul Battley, Aug 3, 2005, in forum: Ruby
    Replies:
    0
    Views:
    206
    Paul Battley
    Aug 3, 2005
  5. anne001
    Replies:
    4
    Views:
    142
    James Edward Gray II
    Aug 2, 2006
Loading...

Share This Page