Getting at the bits of a 32-bit integer

[Jacob H]

Hi there list,

I'm a beginning programmer, so please correct me if any of the
following assumptions are wrong.

Suppose I have the decimal number 255. Since integers in Python are 32
bits, it would look like this in binary:

00000000 00000000 00000000 11111111

There are plenty of unused bits to the left of the number itself. If I
wanted to use some of these bits as true/false flags, how would I go
about it? In Python, how do I write code that gets at the leftmost
byte, or the third bit from the left of the set of 32, or the
rightmost byte plus the bit to its left, etc...

Thanks in advance for any help on this.

Jacob

 

[Phil Frost]

If you really want to do that, you can use the bitwise "and" and "or"
operators to unset/set bits, respectivly. For example, if you want to
set bit 9, that's 0x0100, or 0000 0001 0000 0000 in binary. The integer
'i' with bit 9 unset is 'i & ~0x0100' (~ is the one's complement or
bitwise not operator; it inverts all the bits), and 'i | 0x0100' is 'i'
with bit 9 set. You can also use the bitwise exclusive or "^" to toggle
a bit, like 'i ^ 0x0100'.

However, doing this sort of thing is rather nonpythonic. Python
abstracts the platform's representation of an integer. Integers in
Python arn't always 32 bits; they are actually the size of C's 'long'
type, which is to say you can find the size by looking at sys.maxint,
but it's more trouble than it's worth. You will also encounter problems
if you change the highest bit:
__main__:1: FutureWarning: x<<y losing bits or changing sign will return
a long in Python 2.4 and up
-2147483638

Note that the error message isn't even accurate. This is probably a good
indication that this isn't a common use case!

So, short story: use a bool (True/False)

 

[Paul Rubin]

There are plenty of unused bits to the left of the number itself. If I
wanted to use some of these bits as true/false flags, how would I go
about it? In Python, how do I write code that gets at the leftmost
byte, or the third bit from the left of the set of 32, or the
rightmost byte plus the bit to its left, etc...

It's not really in the Python spirit, but in general you can set the
nth bit (from the right, starting at n=0) of variable x by saying:

x |= (1 << n)

You can clear it with

x &= ~(1 << n)

and you can test it with

if x & (1 << n):
do whatever

 

[Jeremy Bowers]

00000000 00000000 00000000 11111111

There are plenty of unused bits to the left of the number itself. If I
wanted to use some of these bits as true/false flags, how would I go about
it? In Python, how do I write code that gets at the leftmost byte, or the
third bit from the left of the set of 32, or the rightmost byte plus the
bit to its left, etc...

Well, the basic answer is the same as C, with &, >>, and appropriate
constants for what you are doing.
0

Obviously, you need better constants and stuff.

However, I must challenge your need to do this in the first place.
Python and bit twiddling generally don't go together; even if you are
accessing a binary file or protocol you should shuffle out as much to the
struct (?) module as possible. If you want to track every bit and byte,
Python is the wrong language for you... and unless you are tracking many
many millions of bits, this is the wrong decade for it.

Use a class, and set members on it to true or false for your flags.

(The nice thing about such classes is you will often find them smoothly
transition into really *useful* classes, emphasis on the plural "classes" .)

The bit twiddling will most likely be *much* slower than this, and unless
you're dealing with millions of these, the memory savings will be zilch.

If you *are* dealing with millions of these, be sure to look at the
"array" module.

 

[Erik Max Francis]

However, doing this sort of thing is rather nonpythonic. Python
abstracts the platform's representation of an integer. Integers in
Python arn't always 32 bits; they are actually the size of C's 'long'
type, which is to say you can find the size by looking at sys.maxint,
but it's more trouble than it's worth. You will also encounter
problems
if you change the highest bit:

__main__:1: FutureWarning: x<<y losing bits or changing sign will return
a long in Python 2.4 and up
-2147483638

Note that the error message isn't even accurate. This is probably a
good
indication that this isn't a common use case!

So, short story: use a bool (True/False)

So use a Python long instead! That way the size of the Python int is
irrelevant.

 

[Phil Frost]

Then you have paid the cost of a bool, and then some.

 

[Erik Max Francis]

Then you have paid the cost of a bool, and then some.

If someone's interested in using a bitfield to store flags, they're
obviously interested in more than one bool! Now you're talking about
one long vs. a list and n bools.

 

[Grant Edwards]

0

Obviously, you need better constants and stuff.

However, I must challenge your need to do this in the first place.

That's a bit, um, conceited. Somebody's got to twiddle all
those low-level bits out there to provide the infrastructure
that makes the world run.

I'm one of those people, and I need to do bit-twiddling quite
often while implimenting various communications protocols.

Python and bit twiddling generally don't go together;

Says you. I say they go together quite well. Bit-twiddling in
Python works great. It's a hell of a lot easier than doing it
in C, and I've switched to Python for all my non-embedded
bit-twiddling needs.

even if you are accessing a binary file or protocol you should
shuffle out as much to the struct (?) module as possible.

Struct only works at the byte level. For bitfields within
bytes, you've got to use the bitwise and/or/xor/shift operators
just like you do in C.

If you want to track every bit and byte, Python is the wrong
language for you...

So, what language do you think is better? It sure isn't C. I
did bit-twiddling in C for 20 years, and doing it in Python is
much easier.

and unless you are tracking many many millions of bits, this
is the wrong decade for it.

I guess I plain don't understand that last clause. It seems to
imply that if you _are_ tracking many millions of bits, this is
the decade to do it in Python?

I used C for many, many years for low-level stuff like tearing
apart serial data streams, Ethernet frames, IP headers and
suchlike, and I can assure you that it's far, far easier to do
stuff like that in Python than in C.

 

[Jeremy Bowers]

That's a bit, um, conceited.

I said "challenge", not "deny". If you're new to Python and you are
twiddling bits, the odds favor the new user falsely importing old
paradigms from other languages, not needing to do the twiddling. Of
course, in lieu of a problem specification, that's all we can say.

I stand by my message. (And before you jump on this one, please understand
that I use language precisely; unlike a lot of people, when I say "odds
favor", I mean just that. It is not code for "not a chance in hell" or
"absolutely".)

 

[Grant Edwards]

I said "challenge", not "deny".

True, but to me "challenging" somebody's need to do something
implies that you've got some sort of right or authority to
require them to justify what they want to do. I've probably
inferred more than I should. I'm a bit touchy on the subject
now that the Python "int" is going to go away and break some of
my admittedly sloppy code.

If you're new to Python and you are twiddling bits, the odds
favor the new user falsely importing old paradigms from other
languages,

Probably so.

 

[Peter Hansen]

Probably so.

Especially as the OP didn't say he was trying to break apart
Ethernet headers, serial data streams, or suchlike. He just
seems to want to grab a few bits for his own purposes, and
like Jeremy I doubt that the need is real (though also like
he I admit the possibility it could be!) and am quite curious
to hear from the OP again...

-Peter

 

[Jacob H]

Especially as the OP didn't say he was trying to break apart
Ethernet headers, serial data streams, or suchlike. He just
seems to want to grab a few bits for his own purposes, and
like Jeremy I doubt that the need is real (though also like
he I admit the possibility it could be!) and am quite curious
to hear from the OP again...

-Peter

The truth is that I have zero need to fiddle with individual bits. I
just wanted to know how it was done.

And now I do know!

Jacob

 

[Scott David Daniels]

... That's a bit, um, conceited. Somebody's got to twiddle all
those low-level bits out there to provide the infrastructure
that makes the world run.

I'm one of those people, and I need to do bit-twiddling quite
often while implimenting various communications protocols.

If you are interested in such things, take a look at:

http://members.dsl-only.net/~daniels/bits.html

It lets you do things like look at bit-ranges of (int, long, float)
and find things like msb, lsb, particular bits (bit) and
extract bit ranges.

-Scott David Daniels
(e-mail address removed)

 

[Grant Edwards]

If you are interested in such things, take a look at:

http://members.dsl-only.net/~daniels/bits.html

It lets you do things like look at bit-ranges of (int, long, float)
and find things like msb, lsb, particular bits (bit) and
extract bit ranges.

That looks handy.

The other thing that looks very useful is a module posted a few
weeks back that impliments fixed-length numbers. In addition
to manipulating individual bits, one often needs to do
operations on binary, two's compliment numbers of known, fixed
lengths.

 

[Jesper Ribbe]

>

That looks handy.

The other thing that looks very useful is a module posted a few
weeks back that impliments fixed-length numbers. In addition
to manipulating individual bits, one often needs to do
operations on binary, two's compliment numbers of known, fixed
lengths.

I'm very interested in this module which implements fixed-length
integers. Do you perhaps have some reference to it? I tried to google
for it, but couldn't come up with distinct enough keywords.

TIA
Jesper

 

[Grant Edwards]

I'm very interested in this module which implements fixed-length
integers. Do you perhaps have some reference to it? I tried to google
for it, but couldn't come up with distinct enough keywords.

http://groups.google.com/[email protected]&rnum=10

I think that automatic coercion of regular Python integer
values to fixed "Field" objects would make it simpler to use.

 

[Grant Edwards]

http://groups.google.com/[email protected]&rnum=10

I think that automatic coercion of regular Python integer
values to fixed "Field" objects would make it simpler to use.

Oops, I may have been thinking of this one:

http://groups.google.com/[email protected]&rnum=28