structs for data transfer?

Discussion in 'C++' started by Oliver Gerlich, Sep 19, 2004.

  1. Hello,
    I want to transfer messages between a client and a server (over TCP
    sockets). A message consists of a message type (like a message "subject"
    :), the size of the attached data, and the data itself. The data part
    should then be able to contain some information whose layout depends on
    the message type...
    So now I thought I could define some structs which represent the layout
    of the additional information, like this:

    typedef struct
    {
    char versionString[30]
    bool paused;
    int uptime;
    } MsgCoreInfoStruct;

    And then I use something like this:

    MsgCoreInfoStruct myData;
    strcpy(myData.versionString, "Server V1.0");
    myData.uptime = getUptime();
    myData.paused = false;

    Message m;
    m.setData( (char*)(&myData) , sizeof(myData) );

    The Message object then sends the data to the other side.

    Question: When I receive such a message, can I just cast the char*
    (which points to the data) into MsgCoreInfoStruct* myRecvData?
    And can I then use myRecvData->uptime to get the value I sent out?
    And, last but not least ;) , can I use this concept if client and server
    are running on different platforms (in this case, Linux and Win)?

    I have doubts about this (because of data packing, and because
    sizeof(bool) might be inconsistent between compilers...), but I'm not
    sure...

    Can someone tell me if this concept is right or wrong? Or has someone a
    better solution for this problem (maybe something else than structs)?

    Thanks in advance,
    Oliver Gerlich
    Oliver Gerlich, Sep 19, 2004
    #1
    1. Advertising

  2. Oliver Gerlich

    Phlip Guest

    Oliver Gerlich wrote:

    > I want to transfer messages between a client and a server (over TCP
    > sockets).


    I did that, many many winters ago, and wish I didn't. When you communicate,
    TCP is too low-level for data. It can handle the low-level handshaking and
    message metadata required, but not all the high-level details. When you need
    them, you will find yourself adding packets to your packets.

    You are asking "how can I make my program very inflexible for no reason?"

    You need to serialize your data into XML, and then transfer that in a higher
    level protocol, such as HTTP. After you go with a pre-existing system like
    those, they will answer all of your low-level questions, such as how do I
    make this string arbitary length, or how do I add a new field, or how do I
    change a field's meaning, or how to I interpret a transmission error, etc.

    The C languages specify the memory layout of raw structures. Only use them
    when you have a real reason not to use a higher level protocol. Premature
    optimization is the root of all evil.

    --
    Phlip
    http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
    Phlip, Sep 19, 2004
    #2
    1. Advertising

  3. "Oliver Gerlich" <> wrote in message
    news:cikc2c$qet$01$-online.com...
    > Hello,
    > I want to transfer messages between a client and a server (over TCP
    > sockets). A message consists of a message type (like a message "subject"
    > :), the size of the attached data, and the data itself. The data part
    > should then be able to contain some information whose layout depends on
    > the message type...
    > So now I thought I could define some structs which represent the layout of
    > the additional information, like this:
    >
    > typedef struct
    > {
    > char versionString[30]
    > bool paused;
    > int uptime;
    > } MsgCoreInfoStruct;
    >
    > And then I use something like this:
    >
    > MsgCoreInfoStruct myData;
    > strcpy(myData.versionString, "Server V1.0");
    > myData.uptime = getUptime();
    > myData.paused = false;
    >
    > Message m;
    > m.setData( (char*)(&myData) , sizeof(myData) );
    >
    > The Message object then sends the data to the other side.
    >
    > Question: When I receive such a message, can I just cast the char* (which
    > points to the data) into MsgCoreInfoStruct* myRecvData?
    > And can I then use myRecvData->uptime to get the value I sent out?
    > And, last but not least ;) , can I use this concept if client and server
    > are running on different platforms (in this case, Linux and Win)?
    >
    > I have doubts about this (because of data packing, and because
    > sizeof(bool) might be inconsistent between compilers...), but I'm not
    > sure...


    You are right to doubt.

    >
    > Can someone tell me if this concept is right or wrong? Or has someone a
    > better solution for this problem (maybe something else than structs)?
    >


    Write some code to convert the struct you want to send into a char array.
    Write some code to turn that char array back into a struct. Use the first
    piece of code when you send, the second when you recieve. Trying to send
    anything more complicated than char arrays between different types of
    computers is asking for trouble.

    > Thanks in advance,
    > Oliver Gerlich


    john
    John Harrison, Sep 19, 2004
    #3
  4. Oliver Gerlich

    nospam Guest

    Oliver Gerlich wrote:
    > I have doubts about this (because of data packing, and because
    > sizeof(bool) might be inconsistent between compilers...), but I'm not
    > sure...


    Yes you are right :)
    1. You can not be sure not only about bool, but also about byte order(endian
    type) and int size.
    2. Worst than that, there may be alignament difference into the elements of
    the stuct.

    What you have to do is to "serialize" data for sending and "deserialize" for
    receiving. Doing that means to define some rules about how each data type
    is converted into a byte array and back.

    > Can someone tell me if this concept is right or wrong? Or has someone a
    > better solution for this problem (maybe something else than structs)?


    You can always send a structure safelly, after you have defined a method of
    serialization of that struct :)

    If you have time, a good book to read it will be Stevens "Unix Network
    Programming" for a clear ideea about sockets.
    nospam, Sep 19, 2004
    #4
  5. Oliver Gerlich

    nospam Guest

    Phlip wrote:

    > You need to serialize your data into XML, and then transfer that in a
    > higher level protocol, such as HTTP. After you go with a pre-existing
    > system like those, they will answer all of your low-level questions, such
    > as how do I make this string arbitary length, or how do I add a new field,
    > or how do I change a field's meaning, or how to I interpret a transmission
    > error, etc.



    Today, many people use XML over HTTP or even SOAP for the job
    regardles if that is good or bad for the application.
    The core ideea to keep in mind is the ammount of comunication and answer
    time you want.
    If these are not a issue (i.e. you send 2 msg/seccond and a 1/2...1 second
    is ok) you can use XML over HTTP. If your client is going to exchange
    hundreds or more msg/seccond with the server, and you want a very fast
    response, then stay away of XML/HTTP.

    About SOAP, my advice is use it only when is mandatory: i.e. when your
    software have to comunicate with a given product who speak only SOAP.
    Soap have a poor design (just look at Corba for comparation) and the
    overhead is unbelivable: A Soap message with XML serialised data for the
    struct of OP will waste about 20..50 times (yes, TIMES) as much bandwidth
    and will waste about 10..100 times as much processing power on the server
    side compared with when you serialize data by yourself in binary endian
    independent format.
    nospam, Sep 19, 2004
    #5
  6. Oliver Gerlich

    Phlip Guest

    John Harrison wrote:

    > Write some code to convert the struct you want to send into a char array.
    > Write some code to turn that char array back into a struct. Use the first
    > piece of code when you send, the second when you recieve. Trying to send
    > anything more complicated than char arrays between different types of
    > computers is asking for trouble.


    Oh, and then you must figure out either a length system for each part of the
    string, or sentinels and delimeters for the ends of strings. Then you must
    figure out how to escape the delimiters if your users type them inside
    strings. Then you might need a system to name each data element.

    Oh, and then you might localize, and need to pack UTF-8 into your strings.

    Use XML. If HTTP is slow, even to a server you programmed, then rewrite a
    simpler version of it.

    --
    Phlip
    http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
    Phlip, Sep 20, 2004
    #6
  7. Oliver Gerlich

    away Guest

    nospam <> wrote in message
    news:...
    > Phlip wrote:
    >
    > > You need to serialize your data into XML, and then transfer that in a
    > > higher level protocol, such as HTTP. After you go with a pre-existing
    > > system like those, they will answer all of your low-level questions,

    such
    > > as how do I make this string arbitary length, or how do I add a new

    field,
    > > or how do I change a field's meaning, or how to I interpret a

    transmission
    > > error, etc.

    >
    >
    > Today, many people use XML over HTTP or even SOAP for the job
    > regardles if that is good or bad for the application.
    > The core ideea to keep in mind is the ammount of comunication and answer
    > time you want.
    > If these are not a issue (i.e. you send 2 msg/seccond and a 1/2...1 second
    > is ok) you can use XML over HTTP. If your client is going to exchange
    > hundreds or more msg/seccond with the server, and you want a very fast
    > response, then stay away of XML/HTTP.
    >

    Why XML/HTTP is slow?

    > About SOAP, my advice is use it only when is mandatory: i.e. when your
    > software have to comunicate with a given product who speak only SOAP.
    > Soap have a poor design (just look at Corba for comparation) and the
    > overhead is unbelivable: A Soap message with XML serialised data for the
    > struct of OP will waste about 20..50 times (yes, TIMES) as much bandwidth
    > and will waste about 10..100 times as much processing power on the server
    > side compared with when you serialize data by yourself in binary endian
    > independent format.
    >
    >

    What consists overheads of SOAP?

    Thanks for your insights in advance!
    away, Sep 21, 2004
    #7
  8. "Phlip" <> wrote in message
    news:qxB3d.21906$...
    > John Harrison wrote:
    >
    >> Write some code to convert the struct you want to send into a char array.
    >> Write some code to turn that char array back into a struct. Use the first
    >> piece of code when you send, the second when you recieve. Trying to send
    >> anything more complicated than char arrays between different types of
    >> computers is asking for trouble.

    >
    > Oh, and then you must figure out either a length system for each part of
    > the
    > string, or sentinels and delimeters for the ends of strings. Then you must
    > figure out how to escape the delimiters if your users type them inside
    > strings. Then you might need a system to name each data element.
    >
    > Oh, and then you might localize, and need to pack UTF-8 into your strings.
    >
    > Use XML. If HTTP is slow, even to a server you programmed, then rewrite a
    > simpler version of it.
    >


    Good advice but I got the impression that XML might be a bit beyond the OP.

    john
    John Harrison, Sep 21, 2004
    #8
  9. Oliver Gerlich

    nospam Guest

    away wrote:

    > Why XML/HTTP is slow?


    on a standard x86 compiler
    short n=htons(h);
    will translate into 2 CPU instruction if optimisation is on

    ostringstream str;
    str<<"<val>"<<h<<"</val>";
    string n=str.str();

    will be at least hundreds if no more. It will require heap alocation etc...

    in first example sizeof(s) will be 2
    into the second s.size() will be at least 12 and at most 16.
    If you add the <?xml version="1.0" ..... or namespaces
    you got the point.

    A dedicated TCP protocol implemented can send messages using only
    a minimum overhead (like you can define your own message header with
    one byte message type and a long the message size). Just compare the size
    of a minimal http header with that. The http being stateless, you may need
    to implement your own state keeping procedure if you need that. The
    amount of the code executed inside of a application server prior to the
    message reaching your handlers can be estimated very conservative to at
    least thousands CPU instructions.


    For most applications this is OK. If your request is going to make a query
    into a database with milions of records, the overhead not significant
    compared with the time spent in query. However, for some aplications
    it may be unacceptable.

    The point I made was NOT that you do not have to use XML. XML is very good
    for most of the applications. But you can not state that this is the only
    solution, as your answer:
    """
    You need to serialize your data into XML, and then transfer that in a higher
    level protocol, such as HTTP.
    """

    without having a good insight of OP problem.

    And by the way, there are over there some modern C++ socket libraries
    and with a good implementation of message serialization, a custom socket
    protocol may be way easier to implement that J2EE server side solution :)

    > What consists overheads of SOAP?


    Usually, SOAP just double the amount of data to be send
    (serialized/send/deserialized) over a simple POST over HTTP.
    Just look at the SOAP envelope required to send a single integer
    as specified above :)

    Again, if your application do a query at every 2 seconds over a SQL
    database, it do not mater. But at 100+ msg/s a SOAP implementation
    will just collapse.


    > Thanks for your insights in advance!


    You are welcome.
    nospam, Sep 22, 2004
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Patricia  Van Hise

    structs with fields that are structs

    Patricia Van Hise, Apr 5, 2004, in forum: C Programming
    Replies:
    5
    Views:
    624
    Al Bowers
    Apr 5, 2004
  2. Chris Hauxwell

    const structs in other structs

    Chris Hauxwell, Apr 23, 2004, in forum: C Programming
    Replies:
    6
    Views:
    549
    Chris Hauxwell
    Apr 27, 2004
  3. Paminu
    Replies:
    5
    Views:
    631
    Eric Sosman
    Oct 11, 2005
  4. Daniel Rudy
    Replies:
    15
    Views:
    1,376
    Keith Thompson
    Apr 10, 2006
  5. Tuan  Bui
    Replies:
    14
    Views:
    464
    it_says_BALLS_on_your forehead
    Jul 29, 2005
Loading...

Share This Page