struct.calcsize problem

Discussion in 'Python' started by Chandu, Nov 7, 2005.

  1. Chandu

    Chandu Guest

    In using the following struct format I get the size as 593. The same C
    struct is 590 if packed on byte boundary and 596 when using pragma
    pack(4). I am using pack(4) and added 3 spares at the end to get by.
    hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
    64s L L B B B B B 64s 64s'

    Any ideas on what I am doing wrong?
    Chandu, Nov 7, 2005
    #1
    1. Advertising

  2. Chandu

    Larry Bates Guest

    Chandu wrote:
    > In using the following struct format I get the size as 593. The same C
    > struct is 590 if packed on byte boundary and 596 when using pragma
    > pack(4). I am using pack(4) and added 3 spares at the end to get by.
    > hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
    > 64s L L B B B B B 64s 64s'
    >
    > Any ideas on what I am doing wrong?
    >


    Maybe this will help:

    import struct
    hdrFormats = ['16s','32s', '32s','B','8s','H','8s','H','4s','H','H',
    '20s','64s','64s','64s','32s','32s','64s','L','L','B',
    'B','B','B','B','64s','64s']

    sumofcalcsize=0
    i=0
    for fmt in hdrFormats:
    l=struct.calcsize(fmt)
    sumofcalcsize+=l
    print "fmt='%s', calcsize=%i, sumofcalcsize=%i, calcsize=%i" % \
    (fmt, l, sumofcalcsize, struct.calcsize(' '.join(hdrFormats[:i+1])))
    i+=1


    Outputs:

    fmt='16s', calcsize=16, sumofcalcsize=16, calcsize=16
    fmt='32s', calcsize=32, sumofcalcsize=48, calcsize=48
    fmt='32s', calcsize=32, sumofcalcsize=80, calcsize=80
    fmt='B', calcsize=1, sumofcalcsize=81, calcsize=81
    fmt='8s', calcsize=8, sumofcalcsize=89, calcsize=89
    fmt='H', calcsize=2, sumofcalcsize=91, calcsize=92 <====
    fmt='8s', calcsize=8, sumofcalcsize=99, calcsize=100
    fmt='H', calcsize=2, sumofcalcsize=101, calcsize=102
    fmt='4s', calcsize=4, sumofcalcsize=105, calcsize=106
    fmt='H', calcsize=2, sumofcalcsize=107, calcsize=108
    fmt='H', calcsize=2, sumofcalcsize=109, calcsize=110
    fmt='20s', calcsize=20, sumofcalcsize=129, calcsize=130
    fmt='64s', calcsize=64, sumofcalcsize=193, calcsize=194
    fmt='64s', calcsize=64, sumofcalcsize=257, calcsize=258
    fmt='64s', calcsize=64, sumofcalcsize=321, calcsize=322
    fmt='32s', calcsize=32, sumofcalcsize=353, calcsize=354
    fmt='32s', calcsize=32, sumofcalcsize=385, calcsize=386
    fmt='64s', calcsize=64, sumofcalcsize=449, calcsize=450
    fmt='L', calcsize=4, sumofcalcsize=453, calcsize=456 <====
    fmt='L', calcsize=4, sumofcalcsize=457, calcsize=460
    fmt='B', calcsize=1, sumofcalcsize=458, calcsize=461
    fmt='B', calcsize=1, sumofcalcsize=459, calcsize=462
    fmt='B', calcsize=1, sumofcalcsize=460, calcsize=463
    fmt='B', calcsize=1, sumofcalcsize=461, calcsize=464
    fmt='B', calcsize=1, sumofcalcsize=462, calcsize=465
    fmt='64s', calcsize=64, sumofcalcsize=526, calcsize=529
    fmt='64s', calcsize=64, sumofcalcsize=590, calcsize=593

    Larry Bates
    Larry Bates, Nov 8, 2005
    #2
    1. Advertising

  3. On 7 Nov 2005 15:27:06 -0800, "Chandu" <> wrote:

    >In using the following struct format I get the size as 593. The same C
    >struct is 590 if packed on byte boundary and 596 when using pragma
    >pack(4). I am using pack(4) and added 3 spares at the end to get by.
    > hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
    >64s L L B B B B B 64s 64s'
    >
    >Any ideas on what I am doing wrong?
    >

    Looks to me like you are getting default native byte order and _alignment_
    and some pad bytes are getting added in. For native order with no padding,
    try prefixing the format string with '='


    >>> hdrFormat

    '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s 64s L L B B B B B 64s 64s'

    If you add up the individual sizes, padding doesn't happen, apparently:

    >>> map(struct.calcsize, hdrFormat.split())

    [16, 32, 32, 1, 8, 2, 8, 2, 4, 2, 2, 20, 64, 64, 64, 32, 32, 64, 4, 4, 1, 1, 1, 1, 1, 64, 64]
    >>> sum(map(struct.calcsize, hdrFormat.split()))

    590

    But default:
    >>> struct.calcsize(hdrFormat)

    593
    Apparently is native, with native alignment & I get the same as you:
    >>> struct.calcsize('@'+hdrFormat)

    593
    Whereas native order standard (no pad) alignment is:
    >>> struct.calcsize('='+hdrFormat)

    590
    Little endian, standard alignment:
    >>> struct.calcsize('<'+hdrFormat)

    590
    Big endian, standard alignment
    >>> struct.calcsize('>'+hdrFormat)

    590
    Network (big endian), standard alignment:
    >>> struct.calcsize('!'+hdrFormat)

    590

    I guess if you want alignment for anything non-native, you have to specify pad bytes
    where you need them (with x format character).

    Regards,
    Bengt Richter
    Bengt Richter, Nov 8, 2005
    #3
  4. Chandu

    Chandu Guest

    Thanks for the helpful feedback. I guessed it was the alignment issue,
    but could not find the exact format for changing the default. It is not
    mystery any more!
    Bengt Richter wrote:
    > On 7 Nov 2005 15:27:06 -0800, "Chandu" <> wrote:
    >
    > >In using the following struct format I get the size as 593. The same C
    > >struct is 590 if packed on byte boundary and 596 when using pragma
    > >pack(4). I am using pack(4) and added 3 spares at the end to get by.
    > > hdrFormat = '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s
    > >64s L L B B B B B 64s 64s'
    > >
    > >Any ideas on what I am doing wrong?
    > >

    > Looks to me like you are getting default native byte order and _alignment_
    > and some pad bytes are getting added in. For native order with no padding,
    > try prefixing the format string with '='
    >
    >
    > >>> hdrFormat

    > '16s 32s 32s B 8s H 8s H 4s H H 20s 64s 64s 64s 32s 32s 64s L L B B B B B 64s 64s'
    >
    > If you add up the individual sizes, padding doesn't happen, apparently:
    >
    > >>> map(struct.calcsize, hdrFormat.split())

    > [16, 32, 32, 1, 8, 2, 8, 2, 4, 2, 2, 20, 64, 64, 64, 32, 32, 64, 4, 4, 1, 1, 1, 1, 1, 64, 64]
    > >>> sum(map(struct.calcsize, hdrFormat.split()))

    > 590
    >
    > But default:
    > >>> struct.calcsize(hdrFormat)

    > 593
    > Apparently is native, with native alignment & I get the same as you:
    > >>> struct.calcsize('@'+hdrFormat)

    > 593
    > Whereas native order standard (no pad) alignment is:
    > >>> struct.calcsize('='+hdrFormat)

    > 590
    > Little endian, standard alignment:
    > >>> struct.calcsize('<'+hdrFormat)

    > 590
    > Big endian, standard alignment
    > >>> struct.calcsize('>'+hdrFormat)

    > 590
    > Network (big endian), standard alignment:
    > >>> struct.calcsize('!'+hdrFormat)

    > 590
    >
    > I guess if you want alignment for anything non-native, you have to specify pad bytes
    > where you need them (with x format character).
    >
    > Regards,
    > Bengt Richter
    Chandu, Nov 9, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. RA Scheltema
    Replies:
    3
    Views:
    375
    RA Scheltema
    Jan 6, 2004
  2. Gunnar G

    struct in struct

    Gunnar G, May 31, 2004, in forum: C++
    Replies:
    14
    Views:
    779
  3. Chris Fogelklou
    Replies:
    36
    Views:
    1,335
    Chris Fogelklou
    Apr 20, 2004
  4. James Lamanna

    Bug in struct.calcsize() ?

    James Lamanna, Apr 9, 2004, in forum: Python
    Replies:
    2
    Views:
    339
    Andrew Henshaw
    Apr 9, 2004
  5. Glen Rice

    struct calcsize discrepency?

    Glen Rice, Dec 4, 2011, in forum: Python
    Replies:
    9
    Views:
    232
    Nobody
    Dec 6, 2011
Loading...

Share This Page