Re: What is the right way to do this simple thing

Discussion in 'C Programming' started by ImpalerCore, Mar 10, 2012.

  1. ImpalerCore

    ImpalerCore Guest

    On Mar 10, 2:26 pm, (Richard Harter) wrote:
    > Suppose that I have some data that is pointer free.  It isn't
    > necessarily serialized in some format such as JSON but I would like to
    > be able to copy it from one from one machine to another compatible
    > machine and use it without trouble.  Also I don't want to pass around
    > a collection of items and I want everything to be properly aligned.
    >
    > Naturally I think of using the struct hack and a structure something
    > like this:
    >
    >    struct package {
    >        message_type type;       /* The type has access info   */
    >        size_t       total_len;  /* Total length of the package */
    >        size_t       data_len;   /* Length of the data         */
    >        char[1]      data;       /* The actual data             */
    >    };
    >
    > Presumably at some point I will copy said data into an instance of the
    > package which can then be passed around.  Also I really would like to
    > be able to use the data upon receipt without copying it out, e.g.,
    >
    >    local_data = (data_type *)(pkg->data);


    You can rely on malloc generating an aligned pointer for the largest
    alignment necessary for that system. If one has a header of unknown
    size, to properly place a data pointer stacked onto a header, the
    easiest method is to find the largest whole multiple of ALIGN_MAX that
    exceeds the size of your header, which in this case is 'sizeof
    (message_type) + 2 * sizeof (size_t)'.

    > It occurs to me that this might not work there is no guarantee that
    > pkg->data is correctly aligned.  There are a whole bunch of other
    > potential gotchas that I can foresee.  This is particularly troubling
    > to me because my PC has four cores, each with a different DS9000 chip.
    > What's worse my boss is on a "compiler of the week" binge.  Every week
    > we get to test a new compiler.  I really want to get this code
    > bullet-proof.
    >
    > Help!  What is the right way to do this simple thing?


    You can look in some of Chris Thomasson's posts for ALIGN_MAX. I have
    my own version based off his.

    \code snippet
    #define alignof( type ) (offsetof (struct { char c; type t; }, t))

    #if !defined(ALIGN_MAX)

    /*! \cond IGNORE */

    enum gc_align_e
    {
    GC_ALIGN_ENUM_1,
    GC_ALIGN_ENUM_2
    };
    struct gc_align_s
    {
    double d;
    };

    /*
    * The function pointer type is typedef to avoid syntax problems with
    * the definition of the alignof macro.
    */
    typedef double (*gc_align_func)( double );

    /*
    * Ideally, the 'gc_align_max' union should contain every possible C
    * type. That is simply not possible, so the best one can do is use
    * a wide variety of common C types.
    */
    union gc_align_max
    {
    char c;
    short int si;
    int i;
    long int li;
    #if defined(LLONG_MAX)
    long long int lli;
    #endif
    float f;
    double d;
    size_t sz;
    ptrdiff_t diff;
    void* p;
    enum gc_align_e ae;
    struct gc_align_s as;
    gc_align_func af;
    };

    /*! \endcond */

    /*!
    * \brief An estimate of the maximum alignment for any type.
    *
    * The \c ALIGN_MAX constant is not a guaranteed maximum alignment
    * requirement because its "union of all types" may not include a
    * type with a larger alignment requirement.
    *
    * If one discovers a type with an alignment restriction larger than
    * set of types used to determine the \c ALIGN_MAX constant, just add
    * the type to the <tt>union gc_align_max</tt> declaration.
    */
    #define ALIGN_MAX (alignof (union gc_align_max))
    #endif
    \endcode

    Once you have an estimate of ALIGN_MAX, or if you have support for the
    constant in a C11 compiler, you should be able to do something like
    this.

    \code
    /*!
    * \brief Get the size of a \c type rounded up to the nearest
    * multiple of \c ALIGN_MAX.
    * \param type_sz The \c type size.
    * \return The size aligned to the next largest multiple of
    * \c ALIGN_MAX.
    */
    size_t gc_maxalign_sizeof( size_t type_sz )
    { return ((type_sz + ALIGN_MAX - 1) / ALIGN_MAX) * ALIGN_MAX; }

    /*!
    * \brief Get the size of a \c type that conforms to the maximum
    * alignment defined by \c ALIGN_MAX.
    * \param type The type.
    * \return The size of \c type in bytes rounded up to the next
    * multiple of \c ALIGN_MAX.
    *
    * The \c c_maxalign_sizeof macro allows one to stack objects of
    * different types with varying alignment requirements in a single
    * memory block, with the cost of additional memory overhead.
    *
    * \usage
    * \include align/c_maxalign_sizeof_example.c
    *
    * The example above should display something similar to the
    * following.
    *
    * \code
    * c_maxalign_sizeof example
    * -----------------------------------------------
    * | type | sizeof | c_maxalign_sizeof |
    * -----------------------------------------------
    * char 1 8
    * short int 2 8
    * int 4 8
    * long int 4 8
    * long long int 8 8
    * float 4 8
    * double 8 8
    * long double 12 16
    * size_t 4 8
    * ptrdiff_t 4 8
    * void* 4 8
    * struct c_list 12 16
    * struct c_array 28 32
    *
    * ALIGN_MAX = 8
    * \endcode
    */
    #define c_maxalign_sizeof( type ) ( (gc_maxalign_sizeof( sizeof
    (type) )) )
    \endcode

    Hence, all you need to do is place your data pointer at an offset
    'c_maxalign_sizeof ( sizeof (message_type) + 2 * sizeof (size_t) )'
    from a pointer allocated using malloc (which is guaranteed to be
    aligned for any type).

    Hopefully I haven't made any obvious errors, maybe subtle ones exist.

    Best regards,
    John D.
    ImpalerCore, Mar 10, 2012
    #1
    1. Advertising

  2. ImpalerCore

    Ian Collins Guest

    On 03/11/12 12:31 PM, Richard Harter wrote:
    > On Sat, 10 Mar 2012 14:12:17 -0800 (PST), ImpalerCore
    > <> wrote:
    >
    >> On Mar 10, 2:26 pm, (Richard Harter) wrote:
    >>> Suppose that I have some data that is pointer free. =A0It isn't
    >>> necessarily serialized in some format such as JSON but I would like to
    >>> be able to copy it from one from one machine to another compatible
    >>> machine and use it without trouble. Also I don't want to pass around
    >>> a collection of items and I want everything to be properly aligned.

    >
    > [snip remainder of statement, and comments on an approach]
    >
    > I thought of doing an align_max kind of thing but it seems to me that
    > there are problems with that. One is that align_max is not a C89
    > thing and that is one of the constraints. A second is that it gets a
    > bit cumbersome.
    >
    > I thought of using a bit of anti-social casting. Start out with
    >
    > struct load_package {
    > data_type type;
    > size_t total_len;
    > size_t data_len;
    > int offset;
    > T data;
    > };
    >
    > Then the package struct is the same as the load_package struct except
    > for the final char[] hack. Compute the offset of the data, stuff it
    > in the struct, and blithely set
    >
    > pkg = (struct package *)&load_pkg;
    >
    > I think that works, modulo typoes, but it's a bit of a crock because
    > you need a separate struct for every data type.


    Why don't you just use the usual union in a struct technique?

    --
    Ian Collins
    Ian Collins, Mar 11, 2012
    #2
    1. Advertising

  3. ImpalerCore

    Ian Collins Guest

    On 03/11/12 01:44 PM, Richard Harter wrote:
    > On Sun, 11 Mar 2012 13:06:18 +1300, Ian Collins<>
    > wrote:
    >
    >> On 03/11/12 12:31 PM, Richard Harter wrote:
    >>> On Sat, 10 Mar 2012 14:12:17 -0800 (PST), ImpalerCore
    >>> <> wrote:
    >>>
    >>>> On Mar 10, 2:26 pm, (Richard Harter) wrote:
    >>>>> Suppose that I have some data that is pointer free. =A0It isn't
    >>>>> necessarily serialized in some format such as JSON but I would like to
    >>>>> be able to copy it from one from one machine to another compatible
    >>>>> machine and use it without trouble. Also I don't want to pass around
    >>>>> a collection of items and I want everything to be properly aligned.
    >>>
    >>> [snip remainder of statement, and comments on an approach]
    >>>
    >>> I thought of doing an align_max kind of thing but it seems to me that
    >>> there are problems with that. One is that align_max is not a C89
    >>> thing and that is one of the constraints. A second is that it gets a
    >>> bit cumbersome.
    >>>
    >>> I thought of using a bit of anti-social casting. Start out with
    >>>
    >>> struct load_package {
    >>> data_type type;
    >>> size_t total_len;
    >>> size_t data_len;
    >>> int offset;
    >>> T data;
    >>> };
    >>>
    >>> Then the package struct is the same as the load_package struct except
    >>> for the final char[] hack. Compute the offset of the data, stuff it
    >>> in the struct, and blithely set
    >>>
    >>> pkg = (struct package *)&load_pkg;
    >>>
    >>> I think that works, modulo typoes, but it's a bit of a crock because
    >>> you need a separate struct for every data type.

    >>
    >> Why don't you just use the usual union in a struct technique?

    >
    > You might spell out what you think "the usual" is; I looked at
    > several variations and it looked like there were subtle problems with
    > all of them. Perhaps I missed something.


    struct load_package {
    message_type type; /* The type has access info */
    size_t total_len; /* Total length of the package */
    size_t data_len; /* Length of the data */
    union {
    payload_type1;
    payload_type2;
    } payload;
    };

    --
    Ian Collins
    Ian Collins, Mar 11, 2012
    #3
  4. ImpalerCore

    Ian Collins Guest

    On 03/11/12 05:46 PM, Richard Harter wrote:
    > On Sun, 11 Mar 2012 14:15:52 +1300, Ian Collins<>
    > wrote:
    >
    >> On 03/11/12 01:44 PM, Richard Harter wrote:
    >>> On Sun, 11 Mar 2012 13:06:18 +1300, Ian Collins<>
    >>> wrote:
    >>>>
    >>>> Why don't you just use the usual union in a struct technique?
    >>>
    >>> You might spell out what you think "the usual" is; I looked at
    >>> several variations and it looked like there were subtle problems with
    >>> all of them. Perhaps I missed something.

    >>
    >> struct load_package {
    >> message_type type; /* The type has access info */
    >> size_t total_len; /* Total length of the package */
    >> size_t data_len; /* Length of the data */
    >> union {
    >> payload_type1;
    >> payload_type2;
    >> } payload;
    >> };

    >
    > Unless you have something in mind that I'm missing this won't do at
    > all. First of all there is no guarantee of alignment - there may have
    > to be padding between data_len and the union.


    So? Using a union solves the alignment issue - it will be correctly
    aligned.

    > Secondly I really hope
    > you aren't suggesting that every different payload type in a program
    > must be in that union.


    Well that depends on the relationship between the types. If they are
    related, why not? The X11 Event structure has a couple of dozen or so
    structs in its union member.

    > The point of the second bit of code that I did is that it does take
    > care of alignment and doesn't drag in stuff from all other
    > serializations.


    ?

    --
    Ian Collins
    Ian Collins, Mar 11, 2012
    #4
  5. (Richard Harter) writes:

    > On Sun, 11 Mar 2012 14:15:52 +1300, Ian Collins <>
    > wrote:

    <snip>
    >>>> Why don't you just use the usual union in a struct technique?
    >>>
    >>> You might spell out what you think "the usual" is; I looked at
    >>> several variations and it looked like there were subtle problems with
    >>> all of them. Perhaps I missed something.

    >>
    >>struct load_package {
    >> message_type type; /* The type has access info */
    >> size_t total_len; /* Total length of the package */
    >> size_t data_len; /* Length of the data */
    >> union {
    >> payload_type1;
    >> payload_type2;
    >> } payload;
    >>};

    >
    > Unless you have something in mind that I'm missing this won't do at
    > all. First of all there is no guarantee of alignment - there may have
    > to be padding between data_len and the union. Secondly I really hope
    > you aren't suggesting that every different payload type in a program
    > must be in that union.


    There is another "usual" union method where the members are used only to
    ensure alignment and not to designate payloads:

    struct load_package {
    message_type type; /* The type has access info */
    size_t total_len; /* Total length of the package */
    size_t data_len; /* Length of the data */
    union {
    long double dummy_1;
    double dummy_2;
    long long int dummy_3;
    void *dummy_4;
    struct dummy { int i; } dummy_5;
    /* and so on... */
    } aligned[];
    } example;

    You never use any of the dummy members, their purpose is only to ensure
    that

    (void *)example.aligned;

    is aligned for (almost) any type. (Sometimes it is worth giving some of
    the members sane names because it gives you a way to get typed pointers
    to the payload without a cast.) Obviously, you can hide the ugliness
    away in another header if you prefer.

    The main problems is obvious -- can you be sure that maximal alignment
    has been achieved? I think that, for all practical purposes, you can,
    but it's not an intellectually gratifying solution.

    The empty []s are C99. You may need to use [1]

    Note that I've not read your original post, so I'm replying just on this
    detailed point. It's therefore possible that the above fails some other
    requirement, but it's easier for you to check that than for me go find
    (and read) your OP!

    <snip>
    --
    Ben.
    Ben Bacarisse, Mar 11, 2012
    #5
  6. ImpalerCore

    Ian Collins Guest

    On 03/12/12 06:23 AM, Richard Harter wrote:
    > On Sun, 11 Mar 2012 21:27:56 +1300, Ian Collins wrote:
    >> On 03/11/12 05:46 PM, Richard Harter wrote:

    >
    >>> The point of the second bit of code that I did is that it does take
    >>> care of alignment and doesn't drag in stuff from all other
    >>> serializations.

    >>
    >> ?

    >
    > Did you read that code? It's pretty simple. The key is that
    > different packages can have different offsets for the deliverable
    > data; all that is really required is that the payload is correctly
    > aligned and that the package specifies where it starts.
    >
    > The issue with it is where the code for doing the load lives.


    I see. Well the generic union approach Ben quoted will take care of
    alignment issues.

    I use JSON for passing objects unless there is proven performance issue.

    If I have to serialise a selection of types as binary, I use something
    other than C to define the types (usually XML) and generate the struct
    definitions and serialisation code.

    --
    Ian Collins
    Ian Collins, Mar 11, 2012
    #6
  7. ImpalerCore

    ImpalerCore Guest

    On Mar 10, 7:31 pm, (Richard Harter) wrote:
    > On Sat, 10 Mar 2012 14:12:17 -0800 (PST), ImpalerCore
    >
    > <> wrote:
    > >On Mar 10, 2:26 pm, (Richard Harter) wrote:
    > >> Suppose that I have some data that is pointer free. =A0It isn't
    > >> necessarily serialized in some format such as JSON but I would like to
    > >> be able to copy it from one from one machine to another compatible
    > >> machine and use it without trouble. Also I don't want to pass around
    > >> a collection of items and I want everything to be properly aligned.

    >
    > [snip remainder of statement, and comments on an approach]
    >
    > I thought of doing an align_max kind of thing but it seems to me that
    > there are problems with that.  One is that align_max is not a C89
    > thing and that is one of the constraints.  A second is that it gets a
    > bit cumbersome.


    The union and offsetof approach works well enough to use on C89. It's
    just that the value of ALIGN_MAX is not guaranteed as it would be in
    C11. I don't have C11, and in my limited experience, I haven't run
    into issues using this construct on C89 compilers.

    > I thought of using a bit of anti-social casting.  Start out with
    >
    >     struct load_package {
    >         data_type type;
    >         size_t    total_len;
    >         size_t    data_len;
    >         int       offset;
    >         T         data;
    >     };
    >
    > Then the package struct is the same as the load_package struct except
    > for the final char[] hack.  Compute the offset of the data, stuff it
    > in the struct, and blithely set
    >
    >         pkg = (struct package *)&load_pkg;
    >
    > I think that works, modulo typoes, but it's a bit of a crock because
    > you need a separate struct for every data type.


    One option is to prepackage the data through an API that automatically
    places the package header as a prefix to the data pointer.

    \code
    #include <stdio.h>
    #include <stdlib.h>
    #include <clover/pstdint.h>
    #include <clover/align.h>

    size_t current_memory = 0;

    void* track_malloc( size_t size );
    void track_free( void* p );

    int main( void )
    {
    int* i;
    double* d;
    char* s;

    printf( "size_t [sizeof,alignof,c_maxalign_sizeof] = "
    "[%" PRIuSIZE ",%" PRIuSIZE ",%" PRIuSIZE "]\n",
    sizeof (size_t), alignof (size_t),
    c_maxalign_sizeof (size_t) );
    printf( "\n" );

    i = track_malloc( 10 * sizeof (int) );
    printf( "current memory in use: %3" PRIuSIZE " bytes\n",
    current_memory );

    d = track_malloc( 8 * sizeof (double) );
    printf( "current memory in use: %3" PRIuSIZE " bytes\n",
    current_memory );

    s = track_malloc( 32 * sizeof (char) );
    printf( "current memory in use: %3" PRIuSIZE " bytes\n",
    current_memory );

    track_free( i );
    printf( "current memory in use: %3" PRIuSIZE " bytes\n",
    current_memory );

    track_free( d );
    printf( "current memory in use: %3" PRIuSIZE " bytes\n",
    current_memory );

    track_free( s );
    printf( "current memory in use: %3" PRIuSIZE " bytes\n",
    current_memory );

    return 0;
    }

    void* track_malloc( size_t size )
    {
    void* mem_blk = NULL;
    void* p = NULL;

    /*
    * To track the amount of used memory, each memory allocation is
    * prefixed with a size_t object that allows the free function to
    * update 'current_memory' correctly when releasing memory.
    */
    mem_blk = malloc( c_maxalign_sizeof (size_t) + size );

    if ( mem_blk )
    {
    *((size_t*)mem_blk) = size;
    p = (unsigned char*)mem_blk + c_maxalign_sizeof (size_t);

    current_memory += size;
    }

    return p;
    }

    void track_free( void* p )
    {
    void* mem_blk = NULL;
    size_t p_size;

    if ( p )
    {
    mem_blk = (unsigned char*)p - c_maxalign_sizeof (size_t);
    p_size = *((size_t*)mem_blk);

    free( mem_blk );

    current_memory -= p_size;
    }
    }
    \endcode

    \result
    size_t [sizeof,alignof,c_maxalign_sizeof] = [4,4,8]

    current memory in use: 40 bytes
    current memory in use: 104 bytes
    current memory in use: 136 bytes
    current memory in use: 96 bytes
    current memory in use: 32 bytes
    current memory in use: 0 bytes
    \end result

    This is a simple example that stores the size as a 'size_t' header of
    each allocation to the "left" of the data pointer. For my purposes, I
    use the concept to introduce artificial out of memory limits for API
    testing.

    You can apply the same principle to your packaging scheme. Just
    replace 'size_t' with 'struct package_header', and the pointer you
    return will be to the payload, aligned to a multiple of ALIGN_MAX.

    \code
    void* payload_malloc( size_t size, ... payload parameters ... )
    {
    void* mem_blk = NULL;
    void* p = NULL;

    mem_blk = malloc( c_maxalign_sizeof (struct package_header) +
    size );

    if ( mem_blk )
    {
    /* Set the payload to the payload values provided. */
    ((struct payload_header*)mem_blk)->type = msg_type;
    ((struct payload_header*)mem_blk)->total_length = ttl_length;
    ...
    p = (unsigned char*)mem_blk + c_maxalign_sizeof (struct
    package_header);
    }

    return p;
    }

    void package_free( void* p )
    {
    void* mem_blk = NULL;
    struct package_header* pck_hdr;

    if ( p )
    {
    mem_blk = (unsigned char*)p - c_maxalign_sizeof (struct
    package_header);
    pck_hdr = ((struct package_header*)mem_blk);

    free( mem_blk );
    }
    }

    struct payload_header* get_payload_header( void* p )
    {
    return (unsigned char*)p - c_maxalign_sizeof (struct
    package_header);
    }
    \endcode

    Just don't mix and match API calls to your "package" data pointers
    with regular "malloc" pointers and vice versa.

    Best regards,
    John D.
    ImpalerCore, Mar 12, 2012
    #7
  8. ImpalerCore

    ImpalerCore Guest

    On Mar 12, 10:28 am, ImpalerCore <> wrote:
    > On Mar 10, 7:31 pm, (Richard Harter) wrote:


    [snip]

    > You can apply the same principle to your packaging scheme.  Just
    > replace 'size_t' with 'struct package_header', and the pointer you
    > return will be to the payload, aligned to a multiple of ALIGN_MAX.
    >
    > \code
    > void* payload_malloc( size_t size, ... payload parameters ... )
    > {
    >   void* mem_blk = NULL;
    >   void* p = NULL;
    >
    >   mem_blk = malloc( c_maxalign_sizeof (struct package_header) +
    > size );
    >
    >   if ( mem_blk )
    >   {
    >     /* Set the payload to the payload values provided. */
    >     ((struct payload_header*)mem_blk)->type = msg_type;
    >     ((struct payload_header*)mem_blk)->total_length = ttl_length;
    >     ...
    >     p = (unsigned char*)mem_blk + c_maxalign_sizeof (struct
    > package_header);
    >   }
    >
    >   return p;
    >
    > }
    >
    > void package_free( void* p )
    > {
    >   void* mem_blk = NULL;
    >   struct package_header* pck_hdr;
    >
    >   if ( p )
    >   {
    >     mem_blk = (unsigned char*)p - c_maxalign_sizeof (struct
    > package_header);
    >     pck_hdr = ((struct package_header*)mem_blk);
    >
    >     free( mem_blk );
    >   }
    >
    > }
    >
    > struct payload_header* get_payload_header( void* p )
    > {
    >   return (unsigned char*)p - c_maxalign_sizeof (struct
    > package_header);}
    >
    > \endcode
    >
    > Just don't mix and match API calls to your "package" data pointers
    > with regular "malloc" pointers and vice versa.


    Oops! s/payload/package/g
    ImpalerCore, Mar 12, 2012
    #8
  9. ImpalerCore

    Gene Guest

    On Mar 11, 12:51 pm, (Richard Harter) wrote:
    > On Sun, 11 Mar 2012 11:59:29 +0000, Ben Bacarisse
    >
    > [ elision ]
    >
    > I've used that one in the past.  As you say, for all practical
    > purposes it works.  At least it does until a wave of architecture
    > changes arrives.  One of the nasty things about these "for all
    > practical purposes" solutions is that they can make for latent bugs
    > that don't show up until many years later.
    >
    > What I am hoping for is a coding technique that is "bullet proof",
    > i.e., it is guaranteed to work if one has a conforming C compiler.


    At an abstract level, alignment of the payload and its fields is a
    pure function of type information in the header. C provides no
    way to introspectively learn what the payload alignment is. You
    approached it with the "offset" field in the header, but this
    doesn't quite get it if you are planning to access fields directly
    from the receive buffer: you have no way of being absolutely sure
    the payload is properly aligned. I think this is provably
    an unsolvable problem.

    So any bulletproof strategy will have to implement the package as
    two separate values: a header with type information and a blob
    that is the payload. Receiving is always in two stages: Get the
    header
    in order to recover the payload type, then either receive the blob
    directly into a variable of payload type or receive the blob into an
    unaligned buffer and copy it to a variable of payload type. I've
    used both approaches (sort of). The interesting thing is with
    care you can use the same interface for both:

    struct header {
    msg_tag_t tag;
    size_t size;
    ... blah blah
    } hdr[1];

    struct foo_payload {
    ... blah blah
    } foo[1];

    to send:
    hdr->tag = FOO;
    hdr->size = sizeof *foo;

    to send

    write(f, hdr, sizeof *hdr);
    write(f, foo, hdr->size);

    to receive with copying:

    struct raw_msg {
    struct header hdr[1];
    char *data;
    } msg[1];

    void receive_message(int f, struct raw_msg *msg)
    {
    read(f, msg->hdr);
    msg->data = allocate_buffer(msg->hdr->size);
    read(f, msg->data, msg->hdr->size);
    }

    void unpack(void *data, struct raw_msg *msg)
    {
    memcpy(data, msg->data, msg->hdr->size);
    deallocate(msg->data);
    }

    .... receive_message(msg);

    Now pass the raw message from the communications code to the point of
    use and unpack there.

    void foo_processor(struct raw_msg *msg)
    {
    struct foo_payload foo[1];

    if (msg->hdr->type != FOO) error(...);
    unpack(foo, msg);
    }

    To use the "lazy" method where you receive without copying, these
    substitutions:

    struct raw_msg {
    struct header hdr[1];
    int h; // file handle for data blob
    } msg[1];

    void receive_message(struct raw_msg *msg)
    {
    read(f, msg->hdr);
    msg->h = f;
    }

    void unpack(void *data, struct raw_msg *msg)
    {
    read(f, data, msg->hdr->size);
    }

    Of course this has the huge disadvantage that processing must be
    synchronous.
    Gene, Mar 13, 2012
    #9
  10. ImpalerCore

    Ian Collins Guest

    On 03/14/12 10:18 AM, Richard Harter wrote:
    > On Mon, 12 Mar 2012 10:01:39 +1300, Ian Collins<>
    > wrote:
    >
    >> On 03/12/12 06:23 AM, Richard Harter wrote:
    >>> On Sun, 11 Mar 2012 21:27:56 +1300, Ian Collins wrote:
    >>>> On 03/11/12 05:46 PM, Richard Harter wrote:
    >>>
    >>>>> The point of the second bit of code that I did is that it does take
    >>>>> care of alignment and doesn't drag in stuff from all other
    >>>>> serializations.
    >>>>
    >>>> ?
    >>>
    >>> Did you read that code? It's pretty simple. The key is that
    >>> different packages can have different offsets for the deliverable
    >>> data; all that is really required is that the payload is correctly
    >>> aligned and that the package specifies where it starts.
    >>>
    >>> The issue with it is where the code for doing the load lives.

    >>
    >> I see. Well the generic union approach Ben quoted will take care of
    >> alignment issues.

    >
    > Probably, but with no guarantees. Different versions of C, different
    > types, you know.


    Such as?

    >> I use JSON for passing objects unless there is proven performance issue.
    >>
    >> If I have to serialise a selection of types as binary, I use something
    >> other than C to define the types (usually XML) and generate the struct
    >> definitions and serialisation code.

    >
    > Thank you for the comments. I should have made it clearer that I was
    > interested in light weight packaging. Performance is an issue.


    I'd still opt for generated serialisation code. If you decide to change
    the packet structure, you only have to change the code generator.

    --
    Ian Collins
    Ian Collins, Mar 13, 2012
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark_Galeck
    Replies:
    12
    Views:
    424
    Mark Space
    Jul 29, 2008
  2. Dmitry S. Makovey

    is decorator the right thing to use?

    Dmitry S. Makovey, Sep 24, 2008, in forum: Python
    Replies:
    33
    Views:
    968
    Dmitry S. Makovey
    Sep 28, 2008
  3. Zam
    Replies:
    1
    Views:
    218
    Mark Schupp
    Mar 14, 2005
  4. Kirk Haines
    Replies:
    4
    Views:
    86
    Graham Nicholls
    Jul 9, 2004
  5. Tim Rentsch

    Re: What is the right way to do this simple thing

    Tim Rentsch, Mar 10, 2012, in forum: C Programming
    Replies:
    3
    Views:
    251
    Tim Rentsch
    Mar 19, 2012
Loading...

Share This Page