# struct.calcsize problem

Discussion in 'Python' started by Chandu, Nov 7, 2005.

1. ### ChanduGuest

In using the following struct format I get the size as 593. The same C
struct is 590 if packed on byte boundary and 596 when using pragma
pack(4). I am using pack(4) and added 3 spares at the end to get by.
hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
64s L L B B B B B 64s 64s'

Any ideas on what I am doing wrong?

Chandu, Nov 7, 2005

2. ### Larry BatesGuest

Chandu wrote:
> In using the following struct format I get the size as 593. The same C
> struct is 590 if packed on byte boundary and 596 when using pragma
> pack(4). I am using pack(4) and added 3 spares at the end to get by.
> hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
> 64s L L B B B B B 64s 64s'
>
> Any ideas on what I am doing wrong?
>

Maybe this will help:

import struct
hdrFormats = ['16s','32s', '32s','B','8s','H','8s','H','4s','H','H',
'20s','64s','64s','64s','32s','32s','64s','L','L','B',
'B','B','B','B','64s','64s']

sumofcalcsize=0
i=0
for fmt in hdrFormats:
l=struct.calcsize(fmt)
sumofcalcsize+=l
print "fmt='%s', calcsize=%i, sumofcalcsize=%i, calcsize=%i" % \
(fmt, l, sumofcalcsize, struct.calcsize(' '.join(hdrFormats[:i+1])))
i+=1

Outputs:

fmt='16s', calcsize=16, sumofcalcsize=16, calcsize=16
fmt='32s', calcsize=32, sumofcalcsize=48, calcsize=48
fmt='32s', calcsize=32, sumofcalcsize=80, calcsize=80
fmt='B', calcsize=1, sumofcalcsize=81, calcsize=81
fmt='8s', calcsize=8, sumofcalcsize=89, calcsize=89
fmt='H', calcsize=2, sumofcalcsize=91, calcsize=92 <====
fmt='8s', calcsize=8, sumofcalcsize=99, calcsize=100
fmt='H', calcsize=2, sumofcalcsize=101, calcsize=102
fmt='4s', calcsize=4, sumofcalcsize=105, calcsize=106
fmt='H', calcsize=2, sumofcalcsize=107, calcsize=108
fmt='H', calcsize=2, sumofcalcsize=109, calcsize=110
fmt='20s', calcsize=20, sumofcalcsize=129, calcsize=130
fmt='64s', calcsize=64, sumofcalcsize=193, calcsize=194
fmt='64s', calcsize=64, sumofcalcsize=257, calcsize=258
fmt='64s', calcsize=64, sumofcalcsize=321, calcsize=322
fmt='32s', calcsize=32, sumofcalcsize=353, calcsize=354
fmt='32s', calcsize=32, sumofcalcsize=385, calcsize=386
fmt='64s', calcsize=64, sumofcalcsize=449, calcsize=450
fmt='L', calcsize=4, sumofcalcsize=453, calcsize=456 <====
fmt='L', calcsize=4, sumofcalcsize=457, calcsize=460
fmt='B', calcsize=1, sumofcalcsize=458, calcsize=461
fmt='B', calcsize=1, sumofcalcsize=459, calcsize=462
fmt='B', calcsize=1, sumofcalcsize=460, calcsize=463
fmt='B', calcsize=1, sumofcalcsize=461, calcsize=464
fmt='B', calcsize=1, sumofcalcsize=462, calcsize=465
fmt='64s', calcsize=64, sumofcalcsize=526, calcsize=529
fmt='64s', calcsize=64, sumofcalcsize=590, calcsize=593

Larry Bates

Larry Bates, Nov 8, 2005

3. ### Bengt RichterGuest

On 7 Nov 2005 15:27:06 -0800, "Chandu" <> wrote:

>In using the following struct format I get the size as 593. The same C
>struct is 590 if packed on byte boundary and 596 when using pragma
>pack(4). I am using pack(4) and added 3 spares at the end to get by.
> hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
>64s L L B B B B B 64s 64s'
>
>Any ideas on what I am doing wrong?
>

Looks to me like you are getting default native byte order and _alignment_
try prefixing the format string with '='

>>> hdrFormat

'16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s 64s L L B B B B B 64s 64s'

>>> map(struct.calcsize, hdrFormat.split())

[16, 32, 32, 1, 8, 2, 8, 2, 4, 2, 2, 20, 64, 64, 64, 32, 32, 64, 4, 4, 1, 1, 1, 1, 1, 64, 64]
>>> sum(map(struct.calcsize, hdrFormat.split()))

590

But default:
>>> struct.calcsize(hdrFormat)

593
Apparently is native, with native alignment & I get the same as you:
>>> struct.calcsize('@'+hdrFormat)

593
Whereas native order standard (no pad) alignment is:
>>> struct.calcsize('='+hdrFormat)

590
Little endian, standard alignment:
>>> struct.calcsize('<'+hdrFormat)

590
Big endian, standard alignment
>>> struct.calcsize('>'+hdrFormat)

590
Network (big endian), standard alignment:
>>> struct.calcsize('!'+hdrFormat)

590

I guess if you want alignment for anything non-native, you have to specify pad bytes
where you need them (with x format character).

Regards,
Bengt Richter

Bengt Richter, Nov 8, 2005
4. ### ChanduGuest

Thanks for the helpful feedback. I guessed it was the alignment issue,
but could not find the exact format for changing the default. It is not
mystery any more!
Bengt Richter wrote:
> On 7 Nov 2005 15:27:06 -0800, "Chandu" <> wrote:
>
> >In using the following struct format I get the size as 593. The same C
> >struct is 590 if packed on byte boundary and 596 when using pragma
> >pack(4). I am using pack(4) and added 3 spares at the end to get by.
> > hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
> >64s L L B B B B B 64s 64s'
> >
> >Any ideas on what I am doing wrong?
> >

> Looks to me like you are getting default native byte order and _alignment_
> try prefixing the format string with '='
>
>
> >>> hdrFormat

> '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s 64s L L B B B B B 64s 64s'
>
> If you add up the individual sizes, padding doesn't happen, apparently:
>
> >>> map(struct.calcsize, hdrFormat.split())

> [16, 32, 32, 1, 8, 2, 8, 2, 4, 2, 2, 20, 64, 64, 64, 32, 32, 64, 4, 4, 1, 1, 1, 1, 1, 64, 64]
> >>> sum(map(struct.calcsize, hdrFormat.split()))

> 590
>
> But default:
> >>> struct.calcsize(hdrFormat)

> 593
> Apparently is native, with native alignment & I get the same as you:
> >>> struct.calcsize('@'+hdrFormat)

> 593
> Whereas native order standard (no pad) alignment is:
> >>> struct.calcsize('='+hdrFormat)

> 590
> Little endian, standard alignment:
> >>> struct.calcsize('<'+hdrFormat)

> 590
> Big endian, standard alignment
> >>> struct.calcsize('>'+hdrFormat)

> 590
> Network (big endian), standard alignment:
> >>> struct.calcsize('!'+hdrFormat)

> 590
>
> I guess if you want alignment for anything non-native, you have to specify pad bytes
> where you need them (with x format character).
>
> Regards,
> Bengt Richter

Chandu, Nov 9, 2005