Write bits in file

[Monica Leko]

Hi

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

 

[Ken Starks]

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you? is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file, or is the whole file regularly subdivided
at 10-bit intervals?

 

[Monica Leko]

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?
Yes.

is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

 

[Gabriel Genellina]

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

If you really need arbitrary bit sequences that aren't synchonized into bytes, I think there is a BitVector o BitArray package somewhere.
But if you mostly have bytes and sparsely a different size, I think the struct module and some gymnastics involving bitwise operations would be enough.

 

[Ken Starks]

I admit that I was mostly just interested in getting your
question clarified, rather than having any great experise.

But a bit of Googling took me to the 'Bit vector' module,
[I googled: 'python ("bit array" OR "bit vector")']
which might be what you are after. I have no experience
with it, myself:

http://cobweb.ecn.purdue.edu/~kak/dist/BitVector-1.4.1.html

 

[John Nagle]

Hi

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

The "struct" module will let you format Python data as a binary
object of specified format. But it doesn't support arbitrary bit-width
fields; integers can be 1, 2, 4, or 8 bytes.

If you have a space problem, you might use the "struct" module to
convert to a reasonably concise binary representation, then use the
"gzip" module to compress the file down further.

John Nagle

 

[Tim Roberts]

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

 

[pataphor]

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

Here's something to get you started. No guarantees, but I managed to
write four 10 bit numbers to a file producing 5 bytes, saying hello. I
was just looking for an excuse to use send.

P.

def gen(f):
L = []

def flush(L):
L.reverse()
s = ''.join(map(str,L))
j = int(s,2)
f.write(chr(j))

while 1:
x = yield
if x in [0,1]:
L.append(x)
else:
break
if len(L) == 8:
flush(L)
L = []
if L:
while len(L) < 8:
L.append(0)
flush(L)
yield

def tenbits(i):
for j in range(9,-1,-1):
yield i >> j & 1

def charbits(s):
for c in s:
i = ord(c)
for j in range(8):
yield i >> j &1

def byten(L):
while L:
yield L[:10]
L = L[10:]

def test():
f = file('out.dat','w')
g = gen(f)
g.send(None)
for x in [90,611,397,758]:
for bit in tenbits(x):
g.send(bit)
g.send('stop')
f.close()

def test1():
bits = list(charbits('hello'))
L = []
for x in byten(bits):
L.append(int(''.join(map(str,x)),2))
print L

if __name__=='__main__':
test()

 

[castironpi]

You could try

 http://construct.wikispaces.com/

which could well do exactly what you want.

Do you have space to waste? If so, how much? Worst case, you could
be looking at a factor of 32 or higher, if each bit took an entire
system word.

 

[sjdevnull]

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Technically specifying 8-bits isn't quite accurate, as C allows for 9-
bit bytes and other variations depending on the architecture. But
that may be overly pedantic unless you have a PDP-10 laying around
that you're writing C code on or something like that.

 

[Tim Roberts]

Technically specifying 8-bits isn't quite accurate, as C allows for 9-
bit bytes and other variations depending on the architecture. But
that may be overly pedantic unless you have a PDP-10 laying around
that you're writing C code on or something like that.

As long as we are being pedantic, and I don't mind that, I would point out
that I didn't actually say that C worked in 8-bit bytes. I was very
careful to say merely that, assuming you wanted a stream of 8-bit bytes,
you need to use shifting and masking to produce it.

 

[Andrew Lee]

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Hmmmm, bitfields are exactly non-aligned bits in a structure and are
part of ANSI C. Although C gives no guarantee of the ordering of fields
within machine words ... so bitfieds are of limited use in portable
programs unless they are 0-initialized.

IIRC, Huffman code uses arbitrary length bit strings and is the basis of
many compression algorithms.