Unpack Expects the Wrong Number of Bytes

[Timothy Crone]

Hello,

I have noticed that for certain format strings, struct.unpack expects
the wrong number of bytes. For example, this works fine

header = "4si4s4si2h2i3h4s"
data = list(unpack(header,f.read(42)))

however, this

header = "4si4s4si2h2i3h4si"
data = list(unpack(header,f.read(46)))

returns the following error:

struct.error: unpack requires a string argument of length 48

So even though my format string includes an additional 4-byte integer,
unpack is expecting 6 additional bytes. Here's a simpler example:

This works:

header = "s"
data = list(unpack(header,f.read(1)))

however this:

header = "si"
data = list(unpack(header,f.read(5)))

throws

struct.error: unpack requires a string argument of length 8

So unpack expects 7 additional bytes when an integer is added to the
format string. Does anyone know what is going on? I am using Debian
stable, so my Python version is 2.5.2. But I have replicated this with
2.6.2. Here's my proc/version: Linux version 2.6.30-bpo.1-amd64
(Debian 2.6.30-1~bpo50+1) ([email protected]) (gcc version 4.3.2
(Debian 4.3.2-1.1) ) #1 SMP Fri Jun 26 09:41:55 UTC 2009

Any help would be greatly appreciated.

Cheers,
Tim

 

[John Machin]

Hello,

I have noticed that for certain format strings, struct.unpack expects
the wrong number of bytes. [snip]
header = "si"
data = list(unpack(header,f.read(5)))

throws

struct.error: unpack requires a string argument of length 8

So unpack expects 7 additional bytes when an integer is added to the
format string. Does anyone know what is going on?

Alignment. 1 byte for the "s", 3 pad bytes to get to the next "i"
boundary, 4 bytes for the "i".

http://docs.python.org/library/struct.html
Then Ctrl-F search for "align"

Choose a prefix character whose MO matches the struct that you need to
work with.

Cheers,
John

 

[Ross Ridge]

header = "s"
data = list(unpack(header,f.read(1)))

however this:

header = "si"
data = list(unpack(header,f.read(5)))

throws

struct.error: unpack requires a string argument of length 8

So unpack expects 7 additional bytes when an integer is added to the
format string. Does anyone know what is going on?

It's adding pad bytes so that the format matches the equivilent C
structure on your machine. You should use either the "<" little-endian
or the ">" big-endian prefix depending on the byte order used in the file
you're trying to unpack. You can also use the "=" native byte-order
flag if endianness of the file format changes according to the machine
your Python program runs on. If you use any of these flags, no extra
padding will be inserted.

Ross Ridge