YAML vs. Marshal

[Bill Atkins]

Is there any reason to use Marshal instead of YAML? Is there anything
Marshal can do that YAML cannot?

 

[Joel VanderWerf]

Is there any reason to use Marshal instead of YAML? Is there anything
Marshal can do that YAML cannot?

Here's one:

Marshal.dump [String, Integer]

# =>"\004\010[\ac\vStringc\fInteger"

YAML.dump [String, Integer]

ArgumentError: can't dump anonymous class Class

However, you can get around that--see [ruby-talk:95432].

Marshal is, in my experience, a bit faster.

 

[why the lucky stiff]

Marshal is, in my experience, a bit faster.

For small documents, my benchmarks have Marshal running at 3x the speed
of Ruby's YAML parser. For larger documents, this often decreases to
2x. For output, I'm sure YAML is quite a bit slower than Marshal. I
haven't benched it, though.

I'm also working on the YAML bytecode parser, which should run as
quickly as Marshal does. Yep.

Mostly you want to use YAML in place of Marshal if:
* You need portable serialization.
* You will be hand-editing your objects.
* You need to read saved objects (for debugging).

YAML bytecode will only really solve the first of those.

_why

 

[Hal Fulton]

Mostly you want to use YAML in place of Marshal if:
* You need portable serialization.
* You will be hand-editing your objects.
* You need to read saved objects (for debugging).

YAML bytecode will only really solve the first of those.

Can you explain this to me? I sense some irony here. You're taking a
format which is renowned for making persistent objects readable in
plain text, and you're making it unreadable.

No flame intended. I like YAML, I use it all the time, and I appreciate
all your hard work.

But YAML bytecode does sound to me a bit like a step backward.

Reminds me a bit of the early days of Java:
1. Hey, let's make a virtual machine that's PROCESSOR INDEPENDENT!
2. Hey, let's make a new chip that will run this natively!

I don't know that they ever did, though.


Hal

 

[why the lucky stiff]

Can you explain this to me? I sense some irony here. You're taking a
format which is renowned for making persistent objects readable in
plain text, and you're making it unreadable.

Well, the bytecode is sort of a sideproject and it's one of those things
that probably doesn't make sense but actually fills a need. I can't
think of anything else really that is "one of those things," so I'm
proud to present a first-of-breed nonsensical necessity.

Why not provide a wire format that is more efficient than YAML and keep
its portability? My big use case is this: I love DRb. I want it to
work across languages. I guess you could do this with plain YAML. But
we don't really need YAML's readability. And we could use the
efficiency a bit.

It took me an afternoon to write the bytecode parser, so I don't think
it's been expensive. But I want to rewrite it again, having some new
insights, which will take another afternoon. So, I just need to decide
if it's worth an afternoon. Keep in mind that summer afternoons are
prettier than any other kind of afternoon. Perhaps I will wait till
October.

No flame intended. I like YAML, I use it all the time, and I appreciate
all your hard work.

Didn't you just say "No flame intended" in another thread? Seriously.
It's starting to sound really insincere, Hal.

_why

 

[Hal Fulton]

Why not provide a wire format that is more efficient than YAML and keep
its portability? My big use case is this: I love DRb. I want it to
work across languages. I guess you could do this with plain YAML. But
we don't really need YAML's readability. And we could use the
efficiency a bit.

OK, I'll buy that.

Didn't you just say "No flame intended" in another thread? Seriously.
It's starting to sound really insincere, Hal.

Haha, I noticed that after it was too late.

Maybe I should introduce a new Usenet abbreviation, NFI.

<flame>You know, _why, sometimes your code isn't as high-quality as it
usually is.</flame>


Hal

 

[why the lucky stiff]

<flame>You know, _why, sometimes your code isn't as high-quality as it
usually is.</flame>

I swear: the only quality to my code is that which Ruby inherently brings.

_why

 

[Gavin Sinclair]

Why not provide a wire format that is more efficient than YAML and keep
its portability? My big use case is this: I love DRb. I want it to
work across languages. I guess you could do this with plain YAML. But
we don't really need YAML's readability. And we could use the
efficiency a bit.

Doesn't DRb use Marshal, and isn't Marshal faster than YAML? Won't
YAML bytecode be no faster than Marshal? Isn't Marshal suitably
cross-platform?

No flame intended, but you got a lot of questions to answer

Gavin

 

[Joel VanderWerf]

Doesn't DRb use Marshal, and isn't Marshal faster than YAML? Won't
YAML bytecode be no faster than Marshal? Isn't Marshal suitably
cross-platform?

No flame intended, but you got a lot of questions to answer

I guess _why's suggestion is to make it possible for the drb protocol to
talk between languages, if someone wanted to develop DRb for Python or
Perl. But that does raise a lot of questions...

 

[Ara.T.Howard]

I guess _why's suggestion is to make it possible for the drb protocol to
talk between languages, if someone wanted to develop DRb for Python or
Perl. But that does raise a lot of questions...

wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention
for sure...

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[why the lucky stiff]

wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention
for sure...

Reinvention? YAML is already a serialization language. All we're
talking about is providing a bytecode to allow efficiency where
readability isn't needed. It's no different from Rite in that regard.
We don't need to tax the parser all the time.

You could use the same useful YAML APIs but have Marshal's tightness. I
don't know about you, but I'm convinced!

_why

 

[Ara.T.Howard]

Reinvention? YAML is already a serialization language. All we're
talking about is providing a bytecode to allow efficiency where
readability isn't needed. It's no different from Rite in that regard.
We don't need to tax the parser all the time.

You could use the same useful YAML APIs but have Marshal's tightness. I
don't know about you, but I'm convinced!

_why

don't get me wrong, i'm running four peices of code that use yaml heavily as
we speak...

by reinvention i meant the 'object access protocol' bit. i mean, soap is a
lot more than just serialization, it's __what__ you are serializing (an object
request) that has had much work done... but i guess if you leveraged drb...
then who knows...

my latest yaml project is an nfs accessed on disk priority queue that i'm
using to distribute jobs to consumers located on 36 machines. the cool thing
about it is that grep and vi work to modifiy it - too cool.... so - i'm
definitely a believer...

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[Jamis Buck]

Reinvention? YAML is already a serialization language. All we're
talking about is providing a bytecode to allow efficiency where
readability isn't needed. It's no different from Rite in that regard.
We don't need to tax the parser all the time.

You could use the same useful YAML APIs but have Marshal's tightness. I
don't know about you, but I'm convinced!

I'll speak up in _why's defense here, and say I'm convinced, too.
Besides: where in Ruby is there only a single way to do anything? It
certainly can't hurt to have YAML bytecode, and once it is available,
people will find ways to use it. Possibly even ways that _why didn't
foresee.

I say, go for it!

--
Jamis Buck
(e-mail address removed)
http://www.jamisbuck.org/jamis

ruby -h | ruby -e
'a=[];readlines.join.scan(/-(.)\[e|Kk(\S*)|le.l(..)e|#!(\S*)/) {|r| a <<
r.compact.first };puts "\n>#{a.join(%q/ /)}<\n\n"'

 

[Ara.T.Howard]

I say, go for it!

i stand corrected. ;-)

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL :: http://www.ngdc.noaa.gov/stp/
| TRY :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done
===============================================================================

 

[why the lucky stiff]

by reinvention i meant the 'object access protocol' bit. i mean, soap is a
lot more than just serialization, it's __what__ you are serializing (an object
request) that has had much work done... but i guess if you leveraged drb...
then who knows...

Right, yeah. Method dispatch stuff would have to be worked out. That
would be a bit of reinvention.

my latest yaml project is an nfs accessed on disk priority queue that i'm
using to distribute jobs to consumers located on 36 machines. the cool thing
about it is that grep and vi work to modifiy it - too cool.... so - i'm
definitely a believer...

Your faith is unimpeachable, Ara. And give my best to your 36 machines.

_why

 

[Kevin M]

wouldn't that be soap? ;-) i hate xml, but this would be wheel reinvention

Reinvention? YAML is already a serialization language. All we're
talking about is providing a bytecode to allow efficiency where
readability isn't needed.

I certainly think it would be cool to have both formats -- both
functionally equivalent, but one being faster and the other being a
lot handier for debugging.

Maybe it's not so much reinventing the wheel, as constructing a
unicycle...

Kevin

 

[NAKAMURA, Hiroshi]

Hi,

I'm interested in this (YAML byte code) project. I have questions, but
feel free to ignore these questions, because I may be expecting too much.

My big use case is this: I love DRb. I want it to
work across languages.

Does 'DRb' here mean 'RPC'? 'ORPC'? Or do you expect some DRb specific
paradigm with your YAML byte code, such as object by reference,
distributed garbage collection and distributed anonymous code block
handling? Or more such as message boxing or object activation?

Does the goal of YAML byte code (+ message invoking?) includes these
features across languages?

Currently, Ruby's SOAP marshalling implementation passes all existing
unit test for Ruby's marshaler (I hope YAML do, too [ruby-talk:95352]).
So I can use SOAP as wire format of DRb environment as you can see the
figure in http://rrr.jin.gr.jp/rwiki?cmd=view;name=SOAP-DRb+router
(written in 840 days ago). But it does not mean "cross language DRb
environment" though it uses SOAP serialization format which is language
independent. Cross language object environment has some common paradigm
as I listed above.

I wrote too much '?'s. I summarize my question. What is the goal of
YAML byte code project?

Regards,
// NaHi

 

[Hugh Sasse Staff Elec Eng]

Reminds me a bit of the early days of Java:
1. Hey, let's make a virtual machine that's PROCESSOR INDEPENDENT!
2. Hey, let's make a new chip that will run this natively!

I don't know that they ever did, though.

http://www.sun.com/microelectronics/picoJava/overview.html

There seems to be a www.picojava.com as well.

Hal

Hugh

 

[Dmitry V. Sabanin]

I certainly think it would be cool to have both formats -- both
functionally equivalent, but one being faster and the other being a
lot handier for debugging.

Maybe it's not so much reinventing the wheel, as constructing a
unicycle...

I agree. Having a couple of standards for one action is always a good thing.
So you can choose. Choice is great. That's my opinion.

 

[Dick Davies]

Reminds me a bit of the early days of Java:
1. Hey, let's make a virtual machine that's PROCESSOR INDEPENDENT!
2. Hey, let's make a new chip that will run this natively!

I don't know that they ever did, though.

Oh yeah, they built the chip. By the time it came out the API
had changed, and the hardware had about the power of a 90-Mhz cpu
right as 300Mhz hit commodity prices.

Silly billies...