Serialisation formats

Discussion in 'Java' started by Tom Anderson, Aug 3, 2008.

  1. Tom Anderson

    Tom Anderson Guest

    Hi again,

    I'm posting this partly as question and partly as an aide-memoire: what
    formats and/or techniques are there for serialising/persisting/storing
    graphs of objects? So far, i've got:

    - hand-hacked binary storage
    - java.io serialization
    - XML done manually
    - XML via java.beans.XMLEncoder
    - XML via JAXB
    - XML via Swing archiver
    - JSON
    - ASN.1 - various encoding rules
    - CORBA IIOP marshalling (not sure how to get at this)

    Any other significant ones?

    I'm wondering about something like persisting objects to a database
    (HypersonicSQL or similar) and then taking the database table files. It's
    not really a serialisation format per se, but it might be interesting to
    compare.

    tom

    --
    unstable orbits in the space of lies
    Tom Anderson, Aug 3, 2008
    #1
    1. Advertising

  2. Tom Anderson

    Arne Vajhøj Guest

    Tom Anderson wrote:
    > I'm wondering about something like persisting objects to a database
    > (HypersonicSQL or similar) and then taking the database table files.
    > It's not really a serialisation format per se, but it might be
    > interesting to compare.


    HypersonicSQL is relational not object oriented AFAIK.

    My guess would be that most/all databases uses
    binary serialization.

    Arne
    Arne Vajhøj, Aug 3, 2008
    #2
    1. Advertising

  3. Tom Anderson

    Tom Anderson Guest

    On Sun, 3 Aug 2008, Arne Vajhøj wrote:

    > Tom Anderson wrote:
    >> I'm wondering about something like persisting objects to a database
    >> (HypersonicSQL or similar) and then taking the database table files. It's
    >> not really a serialisation format per se, but it might be interesting to
    >> compare.

    >
    > HypersonicSQL is relational not object oriented AFAIK.


    Yes. I was thinking of using an ORM to put things into an RDBMS. Sorry, i
    should have made myself clearer there.

    > My guess would be that most/all databases uses binary serialization.


    Yes. I have come across toy databases that don't, and there are situations
    like using MS Jet to use a CVS file as a database, but i'd be surprised if
    there was a serious database that didn't. And i'd be interested to see how
    efficient that binary format was as an object storage medium, once you
    include any overhead from the ORM.

    tom

    --
    unstable orbits in the space of lies
    Tom Anderson, Aug 3, 2008
    #3
  4. Tom Anderson

    Stefan Ram Guest

    Tom Anderson <> writes:
    >Any other significant ones?


    It is remarkable how fast YAML is being forgotten,
    it has been replaced by JSON in the public mind.

    Well-known formats you have not mentioned are the
    two classic ones: S expressions and SGML.

    A variant of S expressions augmented with attributes
    is called Unotal, and was implemented for Java as Junotal:

    http://www.purl.org/stefan_ram/pub/junotal

    Searching for languages that directly implement cyclic
    graphs, one also might find RDF and its variants.
    Knowledge-Languages, such as CycL or Prolog, also can
    be used to store graphs.

    Even Java could be used. The graph A<--B-->C--. might be
    rendered in Java as: ^ |
    '----------'
    class A {}
    class B { static A $0; static C $1; }
    class C { static A $0; }

    I am also working on general graph API for Java, but
    this does not yet have serialisation:

    http://www.purl.org/stefan_ram/pub/de.dclj.ram.system.space
    Stefan Ram, Aug 3, 2008
    #4
  5. Gordon Beaton, Aug 3, 2008
    #5
  6. Tom Anderson

    Tom Anderson Guest

    On Sun, 3 Aug 2008, Stefan Ram wrote:

    > Tom Anderson <> writes:
    >> Any other significant ones?

    >
    > It is remarkable how fast YAML is being forgotten,
    > it has been replaced by JSON in the public mind.


    Partly because YAML was invented by semi-insane perl guys, and betrays its
    roots.

    > Well-known formats you have not mentioned are the
    > two classic ones: S expressions and SGML.


    I love S-expressions, but they're not exactly a standard format. I guess i
    could do Scheme sexps or something. But how do you express DAGs? Can't
    they only do trees?

    Actually, i think JSON has that problem too. I came up with an extended
    form of JSON that gets around it, where you have a sequence of statements,
    not a single expression; it's not compatible with existing JSON
    implementations, though.

    That reminds me, there's bencoding:

    http://en.wikipedia.org/wiki/Bencode

    > A variant of S expressions augmented with attributes
    > is called Unotal, and was implemented for Java as Junotal:
    >
    > http://www.purl.org/stefan_ram/pub/junotal


    Interesting. For me, one of the beautiful things about S-expressions is
    their simplicity - they don't have this duality of attributes and
    subelements that XML has. Unotal seems to discard that, and indeed to be
    more complicated (although in some way more uniform) than XML.

    > Searching for languages that directly implement cyclic
    > graphs, one also might find RDF and its variants.
    > Knowledge-Languages, such as CycL or Prolog, also can
    > be used to store graphs.


    True, i hadn't thought of that! But at this point, i think we're beyond
    practical territory and into the realm of interesting ideas.

    > Even Java could be used. The graph A<--B-->C--. might be
    > rendered in Java as: ^ |
    > '----------'
    > class A {}
    > class B { static A $0; static C $1; }
    > class C { static A $0; }


    I would certainly agree that java objects are capable of representing the
    java objects i am working with.

    tom

    --
    Imagine a city where graffiti wasn't illegal, a city where everybody
    could draw wherever they liked. Where every street was awash with a
    million colours and little phrases. Where standing at a bus stop was never
    boring. A city that felt like a living breathing thing which belonged to
    everybody, not just the estate agents and barons of big business. Imagine
    a city like that and stop leaning against the wall - it's wet. -- Banksy
    Tom Anderson, Aug 4, 2008
    #6
  7. Tom Anderson

    Tom Anderson Guest

    On Sun, 3 Aug 2008, Gordon Beaton wrote:

    > On Sun, 3 Aug 2008 15:26:49 +0100, Tom Anderson wrote:
    >> I'm posting this partly as question and partly as an aide-memoire:
    >> what formats and/or techniques are there for
    >> serialising/persisting/storing graphs of objects?

    >
    > Here's a comprehensive list:
    > http://web.archive.org/web/20060325012720/www.pault.com/xmlalternatives.html


    Oh, wow. That's kind of a markup format graveyard / hall of fame. Thanks!

    tom

    --
    Imagine a city where graffiti wasn't illegal, a city where everybody
    could draw wherever they liked. Where every street was awash with a
    million colours and little phrases. Where standing at a bus stop was never
    boring. A city that felt like a living breathing thing which belonged to
    everybody, not just the estate agents and barons of big business. Imagine
    a city like that and stop leaning against the wall - it's wet. -- Banksy
    Tom Anderson, Aug 4, 2008
    #7
  8. Tom Anderson wrote:

    > Any other significant ones?


    Fast Object Serialization (uka.transport), developed for the kaRMI
    project of the University if Karlsruhe in Germany. Claimed to be 10
    times faster, but maybe this is no longer valid for newer Java versions.

    "uka.transport is 100% compatible with the regular Java serialization
    mechanism. This means that you can marshal all objects that are
    serializable by means of regular Java object serialization. This is
    achieved by a fall-back mechanism that uses the regular Java object
    serialization if an object is encountered during marshaling that is not
    declared transportable."

    http://www.ipd.uka.de/JavaParty/ukatransport.html

    Best Regards
    --
    Michael Justin
    SCJP, SCJA
    betasoft - Software for Delphiâ„¢ and for the Javaâ„¢ platform
    http://www.mikejustin.com - http://www.betabeans.de
    Michael Justin, Aug 5, 2008
    #8
  9. Tom Anderson

    Stefan Ram Guest

    Stefan Ram, Aug 5, 2008
    #9
  10. Tom Anderson

    Roedy Green Guest

    On Sun, 3 Aug 2008 15:26:49 +0100, Tom Anderson <>
    wrote, quoted or indirectly quoted someone who said :

    >- hand-hacked binary storage
    >- java.io serialization
    >- XML done manually
    >- XML via java.beans.XMLEncoder
    >- XML via JAXB
    >- XML via Swing archiver
    >- JSON
    >- ASN.1 - various encoding rules
    >- CORBA IIOP marshalling (not sure how to get at this)


    see my list of 13 alternatives at:
    http://mindprod.com/jgloss/serialization.html#ALTERNATIZES

    I recently updated it to include those of yours I did not already
    have.

    --

    Roedy Green Canadian Mind Products
    The Java Glossary
    http://mindprod.com
    Roedy Green, Aug 5, 2008
    #10
  11. Tom Anderson

    Tom Anderson Guest

    On Tue, 5 Aug 2008, Michael Justin wrote:

    > Tom Anderson wrote:
    >
    >> Any other significant ones?

    >
    > Fast Object Serialization (uka.transport), developed for the kaRMI project of
    > the University if Karlsruhe in Germany.


    Ooh, i hadn't heard of that. Thanks!

    tom

    --
    The girlfriend of my friend is my enemy. -- old Arabic proverb
    Tom Anderson, Aug 6, 2008
    #11
  12. Tom Anderson

    Tom Anderson Guest

    On Tue, 5 Aug 2008, Stefan Ram wrote:

    > Michael Justin <> writes:
    >> Fast Object Serialization (uka.transport), developed for the kaRMI

    >
    > If self-proclaimed-as-fast formats are the topic, there also is:
    >
    > http://code.google.com/apis/protocolbuffers/


    I've heard of this, but i don't know what it is. Time to look into it!

    tom

    --
    The girlfriend of my friend is my enemy. -- old Arabic proverb
    Tom Anderson, Aug 6, 2008
    #12
  13. Tom Anderson

    Tom Anderson Guest

    On Tue, 5 Aug 2008, Roedy Green wrote:

    > On Sun, 3 Aug 2008 15:26:49 +0100, Tom Anderson <>
    > wrote, quoted or indirectly quoted someone who said :
    >
    >> - hand-hacked binary storage
    >> - java.io serialization
    >> - XML done manually
    >> - XML via java.beans.XMLEncoder
    >> - XML via JAXB
    >> - XML via Swing archiver
    >> - JSON
    >> - ASN.1 - various encoding rules
    >> - CORBA IIOP marshalling (not sure how to get at this)

    >
    > see my list of 13 alternatives at:
    > http://mindprod.com/jgloss/serialization.html#ALTERNATIZES
    >
    > I recently updated it to include those of yours I did not already
    > have.


    Lovely, thankyou!

    tom

    --
    The girlfriend of my friend is my enemy. -- old Arabic proverb
    Tom Anderson, Aug 6, 2008
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nick Gilbert
    Replies:
    4
    Views:
    1,492
  2. Roedy Green

    Serialisation inefficiency

    Roedy Green, Sep 16, 2003, in forum: Java
    Replies:
    8
    Views:
    369
    Robert Olofsson
    Sep 18, 2003
  3. Michael Binz
    Replies:
    0
    Views:
    297
    Michael Binz
    Oct 29, 2003
  4. VisionSet
    Replies:
    1
    Views:
    317
    Roedy Green
    Jul 3, 2004
  5. ickz
    Replies:
    1
    Views:
    329
    Esmond Pitt
    May 13, 2005
Loading...

Share This Page