C structures for wire protocols

[Frédéric Perrin]

Hello,

I've tried reading some networking code, and I see the following for IP
packets in OpenBSD:

/* Structure of an internet header, naked of options. */
struct ip {
#if _BYTE_ORDER == _LITTLE_ENDIAN
u_int ip_hl:4, /* header length */
ip_v:4; /* version */
#endif
#if _BYTE_ORDER == _BIG_ENDIAN
u_int ip_v:4, /* version */
ip_hl:4; /* header length */
#endif
u_int8_t ip_tos; /* type of service */
u_int16_t ip_len; /* total length */
u_int16_t ip_id; /* identification */
u_int16_t ip_off; /* fragment offset field */
u_int8_t ip_ttl; /* time to live */
u_int8_t ip_p; /* protocol */
u_int16_t ip_sum; /* checksum */
struct in_addr ip_src, ip_dst; /* source and dest address */
};

And in the function that processes incoming IP packets, the structure is
read as (I simplified somewhat):

void ipv4_input(struct mbuf *m)
{
struct ip *ip;

ipstat.ips_total++;
if (m->m_len < sizeof (struct ip)) {
ipstat.ips_toosmall++;
goto bad;
}

#define mtod(m,t) ((t)((m)->m_data))
ip = mtod(m, struct ip *);
if (ip->ip_v != IPVERSION) {
ipstat.ips_badvers++;
goto bad;
}

/* do re-assembly, process the IP fields... */

I am surprised by the lack of a `packed' attribute. I haven't seen a
`-fpack-struct' option in the Makefile's, either (but I'll admit to not
having looked very hard). How can we be sure that some architecture is
not going to require that, say, u_int16_t fields in a structure need to
be aligned on 64-bit boundaries because it is faster for the processor?
Doing so would make the structure not match the wire layout of packets
(and the input function would fail early at the length check).

The files I quoted above (ip.h and ip_input.c) are visible on the web at
<http://www.openbsd.org/cgi-bin/cvsweb/src/sys/netinet/>.

Regards,

 

[Ben Pfaff]

I've tried reading some networking code, and I see the following for IP
packets in OpenBSD: [...]
I am surprised by the lack of a `packed' attribute. I haven't seen a
`-fpack-struct' option in the Makefile's, either (but I'll admit to not
having looked very hard). How can we be sure that some architecture is
not going to require that, say, u_int16_t fields in a structure need to
be aligned on 64-bit boundaries because it is faster for the processor?
Doing so would make the structure not match the wire layout of packets
(and the input function would fail early at the length check).

The code is written with the assumption that the C implementation
only aligns data on "natural" boundaries, that is, an "int" is
aligned on a sizeof(int)-byte boundary and so on for each integer
type. This is commonly found in practice. The BSD and Linux
networking stacks, and some others, make this assumption.

 

[Frédéric Perrin]

I've tried reading some networking code, and I see the following for IP
packets in OpenBSD: [...]
I am surprised by the lack of a `packed' attribute. How can we be
sure that some architecture is not going to require that, say,
u_int16_t fields in a structure need to be aligned on 64-bit
boundaries because it is faster for the processor?

The code is written with the assumption that the C implementation
only aligns data on "natural" boundaries, that is, an "int" is
aligned on a sizeof(int)-byte boundary and so on for each integer
type. This is commonly found in practice. The BSD and Linux
networking stacks, and some others, make this assumption.

So, IIUC, the compiler will not add padding for alignment between the
members because:

1. u_int16 members are located after an even number of u_int8 fields,
and the quantity (2*number of u_int16 members + number of u_int8
members) is divisible by 4 before the 32-bit addresses.

2. we have alignof(u_intX) <= sizeof(u_intX) = X/8.

We have 1. true because Vint Cerf was careful in designing the packet
format, and 2. is a reasonable assumption on all architectures. Is that
correct?

 

[Ben Pfaff]

I've tried reading some networking code, and I see the following for IP
packets in OpenBSD: [...]
I am surprised by the lack of a `packed' attribute. How can we be
sure that some architecture is not going to require that, say,
u_int16_t fields in a structure need to be aligned on 64-bit
boundaries because it is faster for the processor?

The code is written with the assumption that the C implementation
only aligns data on "natural" boundaries, that is, an "int" is
aligned on a sizeof(int)-byte boundary and so on for each integer
type. This is commonly found in practice. The BSD and Linux
networking stacks, and some others, make this assumption.

So, IIUC, the compiler will not add padding for alignment between the
members because:

1. u_int16 members are located after an even number of u_int8 fields,
and the quantity (2*number of u_int16 members + number of u_int8
members) is divisible by 4 before the 32-bit addresses.
Yes.

2. we have alignof(u_intX) <= sizeof(u_intX) = X/8.
Yes.

We have 1. true because Vint Cerf was careful in designing the packet
format, and 2. is a reasonable assumption on all architectures. Is that
correct?

I can't speak for the mindset of the designers, but yes the
packet format is appropriate for this. I'd add more qualifiers
to #2, like "on all architectures that Linux and BSD target", but
with those qualifiers I believe that it is correct.

Some network protocols are not well designed for these
assumptions. The most obvious one is Ethernet itself, which has
a 14-byte frame format, so if you align the Ethernet header on a
4-byte boundary then an IP header directly following it is
misaligned. The customary "fix" is to put the Ethernet header 2
bytes past a 4-byte boundary.

ARP isn't naturally aligned either, so you'll probably find some
kind of "pack" directive or another way of avoiding alignment
problems in use for parsing and outputting ARP.

 

[Frédéric Perrin]

Some network protocols are not well designed for these
assumptions. The most obvious one is Ethernet itself, which has
a 14-byte frame format, so if you align the Ethernet header on a
4-byte boundary then an IP header directly following it is
misaligned. The customary "fix" is to put the Ethernet header 2
bytes past a 4-byte boundary.

Indeed, we have in if_ether.h:

/*
* Mbuf adjust factor to force 32-bit alignment of IP header.
* Drivers should do m_adj(m, ETHER_ALIGN) when setting up a
* receive so the upper layers get the IP header properly aligned
* past the 14-byte Ethernet header.
*/
#define ETHER_ALIGN 2 /* driver adjust for IP hdr alignment */

So the frame Ethernet+IP is not aligned to natural boundaries; but since
the Ethernet header is quickly discarded and the real work is done on
the (properly aligned) IP packet, this is OK.

Thanks for the explanations, Ben!

 

[Jorgen Grahn]

I've tried reading some networking code, and I see the following for IP
packets in OpenBSD: [...]
I am surprised by the lack of a `packed' attribute. I haven't seen a
`-fpack-struct' option in the Makefile's, either (but I'll admit to not
having looked very hard). How can we be sure that some architecture is
not going to require that, say, u_int16_t fields in a structure need to
be aligned on 64-bit boundaries because it is faster for the processor?
Doing so would make the structure not match the wire layout of packets
(and the input function would fail early at the length check).

The code is written with the assumption that the C implementation
only aligns data on "natural" boundaries, that is, an "int" is
aligned on a sizeof(int)-byte boundary and so on for each integer
type. This is commonly found in practice. The BSD and Linux
networking stacks, and some others, make this assumption.

And since those people know exactly /what/ compilers will build their
code /where/, they are fine.

That doesn't mean it's something the rest of us should mimic. Please,
treat I/O as arrays of uint8_t unless you have very good reasons not
to.

/Jorgen