Discussion in 'C++' started by Nick Baumbach, Jan 17, 2014.

  1. Nick Baumbach

    woodbrian77 Guest

    Would someone be willing to supply me with the
    Protocol Buffers pieces to serialize and deserialize
    a ::std::vector<::std::deque<double> > ?

    If so, I'll work on writing a test with CMW code
    that compares these. Tia.

    Ebenezer Enterprises
    woodbrian77, Feb 4, 2014
    1. Advertisements

  2. Nick Baumbach

    woodbrian77 Guest

    I forgot about Wave. Good point.

    Having executables, that you ship, that are based
    on a library you also ship is a form of eating your
    own dog food. Our service is made up of 3 executables.
    If any of them don't work right, the service itself
    is affected. It comes down to fixing either an
    executable or the library to get things working again.

    So we are set up like an external user. We use the
    same code (machine generated code and the library) to
    build *programs* as a user does. Imo our examples of
    how to use the code (the executables that I've been
    talking about) are more detailed and realistic than
    those given by the serialization library in Boost.
    Here's an example:

    localMessages.Marshal(localsendbuf, false, cmwBuf.GiveCharStar());

    That line is from this file:

    The ambassador (middle tier) is getting an error message
    from the back tier and is in the process of sending it
    to the front tier. GiveCharStar() just hands us a
    pointer to a null terminated string without anyone
    having to copy it. You shouldn't expect to be able to
    use the pointer indefinitely, and we don't need to as
    it is used right away by the Marshal function.

    We are treating the user like we want to be treated --
    like a grown up. Why would we do that? Because we
    are users also.

    That function's signature is:

    char const* GiveCharStar ()

    Is that possible with the serialization library in
    Boost or other competitors?

    Ebenezer Enterprises
    woodbrian77, Feb 4, 2014
    1. Advertisements

  3. Nick Baumbach

    Öö Tiib Guest

    You sell non-standard language-specific alternative to
    wide set of well-established open and language-neutral
    solutions. You sell it to engineers. Yet you openly tell
    that you need help in finding out if it is outstanding in
    anything at all?
    Öö Tiib, Feb 6, 2014
  4. Nick Baumbach

    woodbrian77 Guest

    It's free.
    I have tests that compare with the serialization
    library in Boost.

    I haven't updated my site yet, but I'm updating
    those tests using Boost 1.55. For the
    vector<deque<double> > test, the Boost version is
    over 2 times slower than the CMW version. The
    Boost-based executable is over 3 times larger than
    the CMW-based executable.

    I'm not sure Protocol Buffers is worth comparing
    against. If someone is interested in seeing that,
    they can do a little bit to make it happen. I'm
    happy to turn over my side of the test so they
    can do the comparison at their location also.

    Ebenezer Enterprises - A stone the builders refused
    has become head of a corner. Psalms 118:22
    woodbrian77, Feb 6, 2014
  5. Nick Baumbach

    Öö Tiib Guest

    Its usage is very narrow (being language- and version-specific).
    Basically clipboard operations (undo/redo/cut/paste) within single
    application and dumping objects into text or xml file or log for
    debugging/problem reporting/code generation purposes.
    For those things it is adequate; it does not matter if undo takes
    25 or 50 milliseconds. CMW feels even more narrow in sense that
    it does not look like suitable for most of named use-cases.

    It is more likely that someone wanting to actually serialise
    something between applications/services using boost uses
    Boost.PropertyTree instead.
    Yes, but what use case it is that you compare?

    You push that 'std::vector<std::deque<double> >' reflection?
    Typically it is implementation detail that may change between
    versions. It is likely that profiling shows in version 2.1 that
    'std::valarray' trumps 'std::deque' but it can't be done since
    that would make CMW of 2.2 not compatible with 2.1. Telling
    that it is indexed collection of indexed collections of floating
    point values gives it more longevity and is neutral to
    programming language.

    If C++ reflection is major property for you then there is cereal. (has JSON/XML and
    various binary representations). There are even attempts of
    making reflection libraries:
    I'm also not sure. Those seem to be on same market
    between small enough for using text-based protocols
    and big enough for using real binary formats (audio,
    graphics, database etc.). Protobuf lacks reflection
    and CMW lacks language neutrality.

    In current world it is hard to find need for entirely new
    type of service. If it is not entirely new then you need
    to collaborate with existing applications.
    Öö Tiib, Feb 7, 2014
  6. Nick Baumbach

    woodbrian77 Guest

    CMW doesn't support JSON or text formats, but I
    don't think it's difficult for application written
    using CMW to work with an application written in
    a language other than C++. I don't have examples
    of that, but there aren't difficult technical
    problems behind doing that.
    It's a low-level test. Applications are made up
    of low-level operations ... I'm open to other
    ideas of what to test.
    From a protocol perspective deque and valarray are
    the same. First we marshal the size of the container
    and then the elements. So it should be possible to
    make the change you mention without compatibility

    The following type is from that link:

    struct SomeData
    int32_t id;
    std::shared_ptr<std::unordered_map<uint32_t, MyRecord>> data;

    template <class Archive>
    void save( Archive & ar ) const
    ar( data );

    template <class Archive>
    void load( Archive & ar )
    static int32_t idGen = 0;
    id = idGen++;
    ar( data );

    The load function looks funny to me. In order
    for something to be loaded, it has to have been
    saved. How is id being set in code that's not
    shown and can it be coherent with the static in
    the load function?

    I agree with their requiring C++ 2011 or newer.
    But Cereal looks similar to the serialization
    library in Boost. Users have to maintain
    serialization functions and the dependence on iostream.

    I think Protobuf has some support for reflection.

    I don't think support for other languages is hard
    to reach. Assuming we stick with byte-level
    marshalling, I don't think there are difficult
    problems in making applications written with
    CMW work with applications in another language.
    Also assuming IEEE 754 floating point.

    Switching to bit-level marshalling would make
    it more work to support other languages so
    probably we won't do that.

    Ebenezer Enteprises - In G-d we trust.
    woodbrian77, Feb 8, 2014
  7. Nick Baumbach

    James Kanze Guest

    I don't know. The way it's implemented often creates more
    problems than it's worth.

    Reference counted pointers seem irrelevant for a lot of
    applications: some won't use any dynamic memory, except within
    the standard containers, and many others will only use it for
    entity objects, which have deterministic lifetimes for which
    shared_ptr is totally irrelevant. And even in the few cases
    where intensive use of reference counted pointers is relevant,
    you probably want some sort of invasive counting, to avoid
    errors. We experimented with `std::shared_ptr` when we moved to
    C++11, but found that it didn't work for us, and went back to
    our home written one.
    James Kanze, Feb 14, 2014
  8. Nick Baumbach

    James Kanze Guest

    Maybe he's not using an Intel. Where I used to work (four and
    a half years ago), I "inherited" a 32 core Sparc because it
    wasn't fast enough for production (where 128 cores was the
    minimum). And that was some time ago.
    James Kanze, Feb 14, 2014
  9. Nick Baumbach

    Öö Tiib Guest

    It seems to be implemented rather well. It is sort of
    heavyweight smart pointer but not overly.
    If the objects for 'std::shared_ptr' are allocated with
    'make_shared' then I haven't observed much performance
    difference with intrusive ref-counting. What are the key features
    it lacks? Issue may be that some functionality it has (like dynamic
    "deleter" or "weak count") are unneeded for particular application
    but errors it does not seem to have.
    Öö Tiib, Feb 14, 2014
  10. Nick Baumbach

    Ian Collins Guest

    Like any software tool, it depends how you use it. Smart pointers and
    reference counting were a couple of the C++ tricks that first pulled me
    over from C and I've been using them ever since.
    Invasive counting should have a slight performance edge, but the
    programmer doesn't always have control of the type contained by the
    pointer. In cases where objects are manipulated more often than that
    are allocated, the advantages are less obvious. Most of my use cases
    for std::shared_ptr are build a tree, process the tree data where the
    process phase dominates the run time.

    Care to explain "to avoid errors"?
    I went the other way...
    Ian Collins, Feb 14, 2014
  11. Nick Baumbach

    Öö Tiib Guest

    Is it some sort of mid-way refactoring where we have raw pointers in mix
    with smarts?
    Yes, typically constructor is protected and there are factories that
    'make_shared'. What is the drawback with that approach?
    Öö Tiib, Feb 14, 2014
  12. Nick Baumbach

    Öö Tiib Guest

    I can't find case when I want that. You can leave particular constructor
    public as performance optimization but it is becoming rather dim for me
    what type of object we are talking about.
    Öö Tiib, Feb 15, 2014
  13. Nick Baumbach

    Jorgen Grahn Guest

    That's where I am. My objects are either on the stack or owned by
    containers. No doubt smart pointers have a place in C++, but I have
    not yet come across a problem where I needed them.

    And that takes us back to Ian Collins' "Maybe you don't but others
    certainly do!" above. "Others" doesn't mean "everyone else" but
    "enough people to make it worth standardizing".

    Jorgen Grahn, Feb 15, 2014
  14. But that's also a bit dangerous. If a function takes a reference as
    parameter/raw pointer I usually expect that it is not taking (any kind of)
    ownership of the object.

    I always wondered if it useful to have a "smart object" template like:

    template <typename T>
    class RC : public T
    /* add perfect forwarding constructors here */
    void addRef();
    void release();
    int count;

    theoretically it combines the the advantages of internal and external
    reference counts:
    - you can use T on the stack and RC<T> as smart object, as with external
    - performance of internal counts.
    - safety/"ownership contract" of external count: you can only create smart
    pointers from RC<T>* not from T*.
    - You can pass by raw pointer/reference and create smart pointers on
    demand, but still have the "ownership contract"

    Any disadvantages?

    Tobias Müller, Feb 15, 2014
  15. Nick Baumbach

    Öö Tiib Guest

    The 'boost::intrusive_ref_counter' is bit better since one can regulate
    thread safety of access to that 'count'. Also CRTP makes it simpler,
    no need for variadic template constructor for template (that makes
    most readers head spin).
    Öö Tiib, Feb 15, 2014
  16. But boost::intrusive_ref_counter is basically just a helper class for
    implementing a plain normal intrusive count.
    It has none of the advantages of the above solution.

    My RC template works with any existing class, and with appropriate
    specialization also with primitive types. Just by using RC<SomeClass> or
    RC<int>. No need to declare a wrapper. No need to decide at design time
    whether a class should be "smart".

    Tobias Müller, Feb 16, 2014
  17. Nick Baumbach

    Öö Tiib Guest

    None? It may be I do not understand the advantages. We can use 'T' on
    the stack and 'shared_ptr<T>' as shared object. Performance of internal
    counts is there with 'make_shared'. There is 'shared_ptr::use_count'.
    Usage of raw pointers directly in code does not make sense for me. I
    either use standard iterators/smart pointers/references or some self-made
    things. 'boost::intrusive_ref_counter' may be useful for designs
    Usage of 'RC<int>' is more likely over-engineering than
    'std::shared_ptr<int>' (where 'int' may be "handle" of something needing
    special "deleter"). The question is not about "smartness". There are
    objects that are never "cloned" (may be "copied") and there are objects
    that are never "copied" (may be "cloned"). It needs to be clear design
    time what type of object it is there and smart pointers make sense only
    for the second type.
    Öö Tiib, Feb 17, 2014
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.