memcpy confusion

[manya]

Ok, it's been a while since I've done the whole memcpy stuff with C++
and I'm having a hard time remembering everything.

I hope, however, that you can help me with my problem.

I memcpy a struct into a buffer to put it into a database (BerkeleyDB,
to be specific) - what do I do to memcpy the thing back?

Code Example:
//struct I want to work with
struct dbHeader {
MyToken commandToken;
MyId id;
MyDate date;
unsigned long length;
};

where MyToken is an enum and MyId as well as MyDate are classes with a
private member unsigned char value[14]; and just set/get methods ( eg
inline void set( const unsigned char *s ) { strncpy( (char *)value,
(char *)s, 14); }; ) this should be correct so far.

now I want to memcpy the memory into a buffer to store into the
database.

MyToken cCommand = record.getMyToken();
MyId cId = record.getMyId();
MyDate cDate = record.getMyDate();
long cLength = record.getLength();

size_t len;
u_int8_t *p, data_buffer[1024];
p = &data_buffer[0];
len = sizeof(cCommand);
memcpy(p, &len, sizeof(len));
p += sizeof(len);
memcpy(p, &cCommand, len);
p += len;
memcpy(p, cId.get(), sizeof(cId.get()));
p += sizeof(cId.get());
memcpy(p, cDate.get(), sizeof(cDate.get()));
p += sizeof(cDate.get());
memcpy(p, &cLength, sizeof(cLength));
p += sizeof(cLength);

(cId.get() returns a char*)

Ok, p has the struct now, has it?

Now I put it into the db and when I want it back, how do I read it back
to my struct?

I hope you can help me.

 

[mlimber]

Ok, it's been a while since I've done the whole memcpy stuff with C++
and I'm having a hard time remembering everything.

I hope, however, that you can help me with my problem.

I memcpy a struct into a buffer to put it into a database (BerkeleyDB,
to be specific) - what do I do to memcpy the thing back?

Code Example:
//struct I want to work with
struct dbHeader {
MyToken commandToken;
MyId id;
MyDate date;
unsigned long length;
};

where MyToken is an enum and MyId as well as MyDate are classes with a
private member unsigned char value[14]; and just set/get methods ( eg
inline void set( const unsigned char *s ) { strncpy( (char *)value,
(char *)s, 14); }; ) this should be correct so far.

now I want to memcpy the memory into a buffer to store into the
database.

MyToken cCommand = record.getMyToken();
MyId cId = record.getMyId();
MyDate cDate = record.getMyDate();
long cLength = record.getLength();

size_t len;
u_int8_t *p, data_buffer[1024];
p = &data_buffer[0];
len = sizeof(cCommand);
memcpy(p, &len, sizeof(len));
p += sizeof(len);
memcpy(p, &cCommand, len);
p += len;
memcpy(p, cId.get(), sizeof(cId.get()));
p += sizeof(cId.get());
memcpy(p, cDate.get(), sizeof(cDate.get()));
p += sizeof(cDate.get());
memcpy(p, &cLength, sizeof(cLength));
p += sizeof(cLength);

(cId.get() returns a char*)

Ok, p has the struct now, has it?

Now I put it into the db and when I want it back, how do I read it back
to my struct?

I hope you can help me.

This is a good approach using C idioms. However, you should consider
the C++ techniques described in these FAQs:

http://www.parashift.com/c++-faq-lite/serialization.html

The Boost libraries have a serialization library that may be of some
assistance also:

http://boost.org/libs/serialization/doc/index.html

The serialization approach allows each class to serialize and
unserialize itself. Your approach makes extra copies (though they could
potentially be eliminated) and tightly couples the implementation of
the record object to the code that does the serialization. Tight
coupling leads to programs that are harder to understand and maintain.

Cheers! --M

 

[Karl Heinz Buchegger]

now I want to memcpy the memory into a buffer to store into the
database.

MyToken cCommand = record.getMyToken();
MyId cId = record.getMyId();
MyDate cDate = record.getMyDate();
long cLength = record.getLength();

size_t len;
u_int8_t *p, data_buffer[1024];
p = &data_buffer[0];
len = sizeof(cCommand);
memcpy(p, &len, sizeof(len));
p += sizeof(len);
memcpy(p, &cCommand, len);
p += len;
memcpy(p, cId.get(), sizeof(cId.get()));
p += sizeof(cId.get());
memcpy(p, cDate.get(), sizeof(cDate.get()));
p += sizeof(cDate.get());
memcpy(p, &cLength, sizeof(cLength));
p += sizeof(cLength);

(cId.get() returns a char*)

Ok, p has the struct now, has it?

Yes, it has. You successfully eliminated possible struct padding in doing so.
If padding is not an issue, you simply could have done:

dbHeader Tmp;
Tmp.commandToken = record.getMyToken();
Tmp.id = record.getMyId();
Tmp.date = record.getMyDate();
Tmp.length = record.getLength();

u_int8_t data_buffer[1024]; // don't know if 1024 is vital, dimensioning the
// array with sizeof(dbHeader) usually does the trick

memcpy( data_buffer, &Tmp, sizeof( Tmp ) ); // In C++ some casts will be needed

Now I put it into the db and when I want it back, how do I read it back
to my struct?

Reverse everything. That is: reverse source with destination
For my example above (the short version would be)

// come up with the bytes in data_buffer, read it from the DB

dbHeader Tmp;
memcpy( &Tmp, data_buffer, sizeof( Tmp ) ); // again, some casts will be needed
// to feed memcpy with void pointers

... = Tmp.commandToken;
... = Tmp.id;
...

or in your long version

u_int8_t *p;
p = &data_buffer[0];

size_t len;
memcpy( &len, p, sizeof( len ) );

p += sizeof(len);
memcpy( &cCommand, p, len ); // you should assert that sizeof( MyToken ) == len here

....

 

[manya]

thanks, I followed the instructions at Berkeley
(http://www.sleepycat.com/docs/ref/am_misc/struct.html) that are
written in C Style and haven't even thought about Serialization (tah,
once again I should think about thinking outside of the box ;-) )

Thanks so much for the tip and the links! I'll try it out tomorrow -
hopefully it'll work together with what I did with Berkeley.

 

[manya]

thanks, with trial and error I found that out just after I posted it.

 

[mlimber]

thanks, I followed the instructions at Berkeley
(http://www.sleepycat.com/docs/ref/am_misc/struct.html) that are
written in C Style and haven't even thought about Serialization (tah,
once again I should think about thinking outside of the box ;-) )

Thanks so much for the tip and the links! I'll try it out tomorrow -
hopefully it'll work together with what I did with Berkeley.

If you're working in C++, I'd suggest going the serialization route. If
you're working in C (even if you're using a C++ compiler), go ahead and
use their techniques. Mixing C-style code with C++ can sometimes lead
to problems, so I avoid doing it when possible.

BTW, I looked at that link. Their second method is filthy, disgusting,
and just plain vile (anagram of evil) in the C++ world:

------
The second way to solve this problem only works if you have just one
variable length field in the structure. In that case, you can declare
the structure as follows:

struct {
int a, b, c;
u_int8_t buf[1];
} info;

Then, let's say you have a string you want to store in this structure.
When you allocate the structure, you allocate it as:

malloc(sizeof(struct info) + strlen(string));

Since the allocated memory is contiguous, you can the initialize the
structure as:

info.a = 1;
info.b = 2;
info.c = 3;
memcpy(&info.buf[0], string, strlen(string) + 1);

and give it to Berkeley DB to store, with a length of:

sizeof(struct info) + strlen(string);

In this case, the structure can be copied out of the database and used
without any additional work.