Help - trying to serialize a struct

E

erbm

hello guys. I'm trying to convert a struct to a buffer of bytes, to
send to a server using sockets... I tryed a lot of solutions, but
nothing worked. I can convert the struct to a char* (string
representation) , but that isn't the point... I must send bytes to the
server and he must be able to deserialize the information. The
functions are:

//return the number of alocated bytes
int message_to_buffer(struct message_t *msg, char **buffer);

//Deserialize the struct
struct message_t *buffer_to_message(int buflen, char *buffer);


the seralization should be something like this: (an array of bytes)

OPCODE KEYSIZE KEY DATASIZE DATA
[2byte] [4byte] [KEYSIZE bytes] [4 byte] [DATASIZE bytes]


And the structs:

struct message_t {
short opcode;
struct key_t key;
struct data_t data;
};

struct key_t{
int keysize;
char *key;
};

struct data_t{
int datasize;
char *data;
};

Does anyone have an ideia, how I can do that?
thanks
 
U

user923005

hello guys. I'm trying to convert a struct to a buffer of bytes, to
send to a server using sockets... I tryed a lot of solutions, but
nothing worked. I can convert the struct to a char* (string
representation) , but that isn't the point... I must send bytes to the
server and he must be able to deserialize the information. The
functions are:

//return the number of alocated bytes
int message_to_buffer(struct message_t *msg, char **buffer);

//Deserialize the struct
struct message_t *buffer_to_message(int buflen, char *buffer);

the seralization should be something like this: (an array of bytes)

OPCODE  KEYSIZE      KEY         DATASIZE    DATA
 [2byte]   [4byte]   [KEYSIZE bytes] [4 byte] [DATASIZE bytes]

And the structs:

struct message_t {
       short opcode;
       struct key_t key;
       struct data_t data;

};

struct key_t{
       int keysize;
       char *key;

};

struct data_t{
       int datasize;
       char *data;

};

Does anyone have an ideia, how I can do that?

Your question is closely related to these questions from the C-FAQ:

2.11: How can I read/write structures from/to data files?

A: It is relatively straightforward to write a structure out using
fwrite():

fwrite(&somestruct, sizeof somestruct, 1, fp);

and a corresponding fread invocation can read it back in.
However, data files so written will *not* be portable (see
questions 2.12 and 20.5). Also, if the structure contains any
pointers, only the pointer values will be written, and they are
most unlikely to be valid when read back in. Finally, note that
for widespread portability you must use the "b" flag when
opening the files; see question 12.38.

A more portable solution, though it's a bit more work initially,
is to write a pair of functions for writing and reading a
structure, field-by-field, in a portable (perhaps even human-
readable) way.

References: H&S Sec. 15.13 p. 381.

2.12: My compiler is leaving holes in structures, which is wasting
space and preventing "binary" I/O to external data files. Why?
Can I turn this off, or otherwise control the alignment of
structure fields?

A: Those "holes" provide "padding", which may be needed in order to
preserve the "alignment" of later fields of the structure. For
efficient access, most processors prefer (or require) that
multibyte objects (e.g. structure members of any type larger
than char) not sit at arbitrary memory addresses, but rather at
addresses which are multiples of 2 or 4 or the object size.

Your compiler may provide an extension to give you explicit
control over struct alignment (perhaps involving a #pragma; see
question 11.20), but there is no standard method.

See also question 20.5.

References: K&R2 Sec. 6.4 p. 138; H&S Sec. 5.6.4 p. 135.

2.13: Why does sizeof report a larger size than I expect for a
structure type, as if there were padding at the end?

A: Padding at the end of a structure may be necessary to preserve
alignment when an array of contiguous structures is allocated.
Even when the structure is not part of an array, the padding
remains, so that sizeof can always return a consistent size.
See also question 2.12 above.

References: H&S Sec. 5.6.7 pp. 139-40.

2.14: How can I determine the byte offset of a field within a
structure?

A: ANSI C defines the offsetof() macro in <stddef.h>, which lets
you compute the offset of field f in struct s as
offsetof(struct s, f). If for some reason you have to code this
sort of thing yourself, one possibility is

#define offsetof(type, f) ((size_t) \
((char *)&((type *)0)->f - (char *)(type *)0))

This implementation is not 100% portable; some compilers may
legitimately refuse to accept it.

References: ISO Sec. 7.1.6; Rationale Sec. 3.5.4.2; H&S
Sec. 11.1 pp. 292-3.
 
E

erbm

There is no universal method of serializing.  What method does
the server expect?  You have to use *THAT* method if you expect
the server to understand it.
Are these functions you are expected to write, or functions you
already have code for?


Those are functions i'm supose ot write... the client would use the
first one to serialize the struct and the server the second one to
deserialize.

//return the number of alocated bytes
int message_to_buffer(struct message_t *msg, char **buffer);
//Deserialize the struct
struct message_t *buffer_to_message(int buflen, char *buffer);
the seralization should be something like this: (an array of bytes)
OPCODE  KEYSIZE      KEY         DATASIZE    DATA
[2byte]   [4byte]   [KEYSIZE bytes] [4 byte] [DATASIZE bytes]

There are all sorts of possible ways to do this, including issues
about the endianness of multi-byte integers, which need not match
the endianness of either server or client.


I will use htons(), htonl(), ntohs(), ntohl() to fix the problem of
endianness, but first i need to have an idea of how to implement those
functions above
 
M

Mark Wooding

erbm said:
the seralization should be something like this: (an array of bytes)

OPCODE KEYSIZE KEY DATASIZE DATA
[2byte] [4byte] [KEYSIZE bytes] [4 byte] [DATASIZE bytes]


And the structs:

struct message_t {
short opcode;
struct key_t key;
struct data_t data;
};

struct key_t{
int keysize;
char *key;
};

struct data_t{
int datasize;
char *data;
};

Does anyone have an ideia, how I can do that?

Are struct key_t and struct data_t different just to be annoying?
(They're certainly misnamed. The `_t' suffix suggests a type name,
rather than a struct tag.) Is there some reason you desperately want to
be able to represent negative sizes in your structures?

My solution to all of this mess is as follows.

* I have a small library which works on structures like this:

typedef struct buf {
octet *base, *p, *limit;
unsigned f;
#define BF_BROKEN 1u
} buf;

This refers to a buffer of octets (8-bit unsigned quantities, stored
in elements of type unsigned char) starting at base, and ending just
below limit; p is a pointer somewhere into this area, and is stepped
along as we read or write. The flag BF_BROKEN is set if we run out
of space. (This saves a certain amount of tedious error checking.)

* I wrote a bunch of macros to translate 8-, 16-, 24-, 32- and 64-bit
quantities between the `obvious' C types and arrays of octets. As
an example, here are the 32-bit big-endian load and store macros.

#define MASK8 0xffu
#define U8(x) ((octet)((x) & MASK8))
#define GETBYTE(p, o) (((octet *)(p))[o] & MASK8)
#define PUTBYTE(p, o, v) (((octet *)(p))[o] = U8((v)))

#define LOAD32_B(p) \
(((uint32)GETBYTE((p), 0) << 24) | \
((uint32)GETBYTE((p), 1) << 16) | \
((uint32)GETBYTE((p), 2) << 8) | \
((uint32)GETBYTE((p), 3) << 0))
#define STORE32_B(p, v) \
(PUTBYTE((p), 0, (uint32)(v) >> 24), \
PUTBYTE((p), 1, (uint32)(v) >> 16), \
PUTBYTE((p), 2, (uint32)(v) >> 8), \
PUTBYTE((p), 3, (uint32)(v) >> 0))

(Note: my `octet' and `uint32' types, and so on, may be able to
represent more than 8 or 32 bits.)

* I then wrapped these up in functions which work on buffers, stepping
the pointer along appropriately. On top of these, I built some more
functions for dealing with chunks of memory with lengths. (These
are actually built by hairy macros, due to the combinatorial
explosion of different endiannesses and so on.)

Your serialization job would look something like this.

int serialize(const struct message_t *m, buf *b)
{
if (buf_putu16(b, m->opcode) ||
buf_putmem32(b, m->key.keysize, m->key.key) ||
buf_putmem32(b, m->data.datasize, m->data.data))
return (-1);
return (0);
}

Deserialization is messier, because it involves memory allocation -- and
reading is always harder than writing anyway, because more can go wrong.

Also, your signed types are annoying, and `int' isn't necessarily big
enough to contain a 32-bit value anyway; I'll assume they're fixed to be
unsigned and sufficiently large.

int deserialize(struct message_t *m, buf *b)
{
const octet *p;
size_t n;

m->opcode = buf_getu16(b);

if ((p = buf_getmem32(b, &n)) == 0 ||
(m->key.key = malloc(n)) == 0)
return (-1);
memcpy(m->key.key, p, n);
m->key.keysize = n;

if ((p = buf_getmem32(b, &n)) == 0 ||
(m->data.data = malloc(n)) == 0) {
free(m->key.key);
return (-1);
}
memcpy(m->data.data, p, n);
m->data.datasize = n;

return (0);
}

-- [mdw]
 
K

katmac

the seralization should be something like this: (an array of bytes)

OPCODE KEYSIZE KEY DATASIZE DATA
[2byte] [4byte] [KEYSIZE bytes] [4 byte] [DATASIZE bytes]

allocate a buffer of size 2 + 4 + keysize + 4 + datasize.
put opcode into an int16_t variable using htons(),
then memcpy 2 bytes from the variable to buffer[0].
put keysize into an int32_t variable using htonl(),
then memcpy 4 bytes from the variable to buffer[2].
memcpy keysize bytes from key.key to buffer[6].
handle datasize and data.data in a similar fashion.

for deserializing, you would use memcpy(), ntohs(), and ntohl().
you will probably need separate malloc calls for the message,
key.key, and data.data, and to be robust you should check for
malloc errors and the correct total length.
 
J

JosephKK

There is no universal method of serializing. What method does
the server expect? You have to use *THAT* method if you expect
the server to understand it.

While that may be true, XDR is pretty well established.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top