Discussion in 'C++' started by Nick Baumbach, Jan 17, 2014.

  1. Nick Baumbach

    Öö Tiib Guest

    If someone asks for C++ solution for serialization then he
    gets lot of answers like protocol buffers, jsoncpp,
    codesynthesis xsd, tinyxml, boost serialize, and so
    on. Like you see all are capable to serialize into language
    neutral formats. Other languages have same or
    different tools for serialization since what matters is
    the format.
    Lot of network services are not written in C++. I trust majority.
    I speak of experience. Boost tends to have lower amount of
    defects than other C++ libraries. If there is a defect then it is
    usually fixed soon and also workaround is found. Anyway if
    it looks too complex for me to fix it myself then I avoid it.
    Öö Tiib, Jan 26, 2014
    1. Advertisements

  2. Nick Baumbach

    woodbrian77 Guest

    The C++ Middleware Writer has support for serializing
    some of Boost. G-d willing we'll add support for other
    types in the future including more of Boost. It will
    depend on the needs of users.

    I'm talking about binary serialization here.

    Do you disagree with my claim that C++ is most important
    language for services?
    Some of Boost is as you describe. Other parts don't
    get much love.

    Ebenezer Enterprises
    woodbrian77, Jan 26, 2014
    1. Advertisements

  3. Nick Baumbach

    Ian Collins Guest

    While C++ is an important language for services, it has to inter-operate
    with servers and clients written in other languages. This is why most
    serialisation formats in common use today (SOAP, XML-RPC, JSON) are
    textual or binary extensions of them (BSON).
    Ian Collins, Jan 26, 2014
  4. Nick Baumbach

    Öö Tiib Guest

    The binary formats are less used than text formats when traffic
    is small enough. There are plenty of binary formats and most
    have C++ or C serialization/deserialization available. Protocol
    Buffers have language neutral binary format from above.

    Boost Serialization binary format may have differences between
    versions of library and it is C++ only. However it may be useful
    for usage within single application.
    No. Claiming what is more or less important programming language
    misses the point about importance of formats. I was saying that
    there are lot of services that are not written in C++. These are
    important too and the programs need to interoperate so need to
    use common format.
    Most of the libraries in Boost seem to be made with love.
    Öö Tiib, Jan 26, 2014
  5. Nick Baumbach

    woodbrian77 Guest

    If you had said --

    it sometimes has to inter-operate

    I'd agree. I know of a few companies that write C++
    programs that communicate with each other. That's
    not exactly unheard of.

    And as I said I'm open to working on adding support
    for another language, but I don't think that language
    will be Java. Python or C# is more likely.

    I think scientific apps and games often use binary serialization.

    I believe the quality of my software is good and believe
    that thinking about the software/service through another
    project's eyes will help me to improve the software service.

    Ebenezer Enterprises
    woodbrian77, Jan 26, 2014
  6. Nick Baumbach

    Ian Collins Guest

    Probably the most common type of "service" these days is the web
    service. Web services almost exclusively use SOAP. If you are going to
    write the back ends for web pages in C++, you will probably have to use
    JSON to communicate with the client JavaScript.
    By using a proprietary binary format, you are restricting yourself to a
    narrow range of applications. The easiest way to support other
    languages is to use a well known on wire format.
    Ian Collins, Jan 26, 2014
  7. Nick Baumbach

    woodbrian77 Guest

    Maybe. Currently I'm doing byte-level marshalling, but am
    thinking about switching to bit-level marshalling. In that
    case a boolean would be marshalled as one bit rather than
    taking a whole byte.

    I'm aware of HDF, netCDF and ASN 1. Do any of them
    support bit-level marshalling? Do any of the standards
    have support for polymorphism?

    I marshall string lengths as variable length integers. Do
    any of the standards require fixed length integers for string
    lengths? I don't think I want that so am asking to find out
    what standards to avoid.

    Ebenezer Enterprises - In G-d we trust.
    woodbrian77, Jan 27, 2014
  8. Nick Baumbach

    Ian Collins Guest

    Um, why? Surely the extra overhead would outweigh transmission time
    ASN.1 is awfully complex and out of favour these says. I'm not familiar
    with the others.

    It looks like you set out to serialise the object model rather than
    simply serialising data. Isn't that tying things too tightly to C++?
    Most interchange standards currently in vogue value simplicity over
    minimising the payload size.
    Ian Collins, Jan 27, 2014
  9. Nick Baumbach

    Jorgen Grahn Guest

    ASN.1 with the PER (packed encoding rules) does. But PER is also the
    one you need expert help to generate a parser for ... I used it
    2000--2001 with some telecom protocol (RANAP?) and it wasn't much fun

    Personally I suspect a (possibly gzipped) text stream is good enough
    almost all the time.

    Jorgen Grahn, Jan 27, 2014
  10. Nick Baumbach

    woodbrian77 Guest

    Thanks and thanks to Ian for his comment on ASN.1. I think I'll
    steer clear of that.
    I think those working on scientific apps would disagree.
    The serialization of their data is faster with binary and
    there's probably no need for an additional compression step.

    Ebenezer Enterprises
    woodbrian77, Jan 28, 2014
  11. Nick Baumbach

    Jorgen Grahn Guest

    Quite possible for some kinds of scientific apps -- I think my "almost
    all the time" gets me off that particular hook!

    Jorgen Grahn, Jan 28, 2014
  12. Nick Baumbach

    woodbrian77 Guest

    I've been reading about JSON some. The following is
    maybe due to some misunderstanding, but it seems
    JSON can only deal with user defined objects.

    I have a message template (not a C++ template) that
    looks like this:

    @out :):cmw::marshalling_integer, const char*)

    (A function is generated based on that.)

    IIuc, with JSON, I'd have to create a class that has
    those two types in it and then marshal an instance of
    that type. I like not having to do that as there's
    no need currently for such a class. Those two pieces
    of data are from the command line and I just forward
    them to the marshalling function without having to
    create an object that's only use would be marshalling
    it's data members. I will admit it wouldn't take long
    to construct such an object, but there's a wrinkle.
    If the class were:

    class front_tier_type
    cmw::marshalling_integer account_number;
    char* path;

    That class won't work for me in my middle tier where
    it doesn't hurt to store the data from the path variable
    in a std::string. I have to store it somewhere in the
    middle tier so I choose to put it in a std::string.

    Constructing a front_tier_type would be fast, but if
    I changed it to have a std:string instead of a char*
    it gets more expensive. I don't want to have to pay
    for that in the front tier. So iiuc, JSON based
    approach would have one class on sending side and
    another on the receiving side. The other approach I
    described is also different on the sending and receiving
    ends, but is more efficient.

    If there were more than two pieces of data, constructing
    an object becomes more of a waste.

    The C++ route seems to offer more flexibility than
    the JSON approach.

    The format isn't a secret.
    I may go back to this angle.

    Ebenezer Enterprises
    woodbrian77, Jan 29, 2014
  13. Nick Baumbach

    Ian Collins Guest

    It depends on how you use JSON and your chosen library. I would write
    something like:

    json::Object blob;
    blob["number"] = accountNumber;
    blob["name"] = accountName;

    std::cout << blob;

    In cases where speed of serialisation isn't critical (most cases I'd
    suggest), the flexibility and interoperability of text on the wire wins out.

    I often use all or part of the received JSON object directly in the
    sever code, so it never gets converted to something else. I guess this
    habit comes form having written a lot of JavaScript.
    But you's have to convince others to adopt it.
    Ian Collins, Jan 30, 2014
  14. Nick Baumbach

    woodbrian77 Guest

    That kind of helps, but the indices ("number" and "name")
    are a problem. I don't have a means of getting that
    information from a user like I do if they provide a header
    file with data member names.

    Let me change the messsage template a little

    @out (int, const char*)

    The generated function for that will have a1 and a2
    for the names of the arguments. So

    json::Object blob;
    blob["a1"] = a1;
    blob["a2"] = a2;

    That doesn't work for me on the receiving side where
    I have a user defined type.
    Well where speed is critical, there's a good chance they'll
    be using C++ so that much is promising from my perspective.
    I recall your saying something like that previously. I'd like
    to know how accessing a JSON object compares to accessing a C++
    object. Is it a lot slower? I recall Juha talking about
    something that may have been similar with Objective C.

    Ebenezer Enterprises
    woodbrian77, Jan 30, 2014
  15. Nick Baumbach

    Ian Collins Guest

    Given a JSON object is a C++ object, the access will depend on the
    complexity of the object. My library is designed for programmer
    convenience first, efficiency a close second. I wanted to support
    arbitrary nesting, such as

    json::Object blob;

    blob["one"]["two"]["three"]["four"] << 1 << 2 << "hello";

    which creates this object:


    This requires a degree of indirection than a more basic implementation

    If I want to use an internal C++ object, I code generate (usually from
    an Open Office document) to/from JSON methods for the object's data.
    Ian Collins, Jan 30, 2014
  16. Nick Baumbach

    woodbrian77 Guest

    What about streaming video?
    woodbrian77, Jan 31, 2014
  17. Nick Baumbach

    Dombo Guest

    Op 26-Jan-14 23:46, schreef:
    When communicating with a web server chances are it is written in Java.
    Python has its own serialization facilities as does .NET, as does Java.
    Focusing on one or a very few languages puts your solution at a
    disadvantage compared to the more established alternatives. If you want
    people to convince to use your solution over the well known, proven,
    well supported and industry standard solutions (SOAP, JSON, BSON...etc),
    you have to provide some compelling reason, i.e. a unique selling point,
    which as of yet I haven't seen in spite of your frequent postings
    promoting your library. I think you need to do a bit more market
    research before investing in reinventing the wheel.
    Dombo, Feb 1, 2014
  18. Nick Baumbach

    woodbrian77 Guest

    I'm pioneering on line code generation. These people

    are also working on on line code generation, but
    they use Java. Most of what they say in terms of
    marketing their service is also true of the C++
    Middleware Writer.

    10 years ago there was some doubt about the
    path, but I don't think there's any doubt today.
    On line code generation is here to stay.

    Ebenezer Enterprises - "Imitation is the sincerest
    form of flattery."
    woodbrian77, Feb 1, 2014
  19. Nick Baumbach

    Dombo Guest

    Op 01-Feb-14 17:16, schreef:
    I'd hardly call that pioneering.
    Springfuse makes rather generic claims, the kind of marketing speak one
    expects for just about any software development tool. And above all it
    still not answers the questions how your product differentiates itself
    from competing, well established, solutions.
    Ask yourself what is the benefit of online code generation for the
    client. If it is the only option for code generation it would be a
    considered to be a serious disadvantage by many because:
    1. It undesirable to make your build process dependant on some external
    service not under your own control;
    2. But more importantly...what if the company that provides the online
    service goes out of business or just decides to pull the plug?
    Dombo, Feb 1, 2014
  20. Nick Baumbach

    Jorgen Grahn Guest

    No. But it seems to me that's a different problem compared to the
    ones discussed so far.

    Jorgen Grahn, Feb 1, 2014
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.