# Bit Operations

Discussion in 'Python' started by Gianmaria Iaculo - NVENTA, Nov 28, 2007.

1. ### Gianmaria Iaculo - NVENTAGuest

Hi there,
I'm so new to python (coming from .net so excuse me for the stupid question)
and i'm tring to do a very simple thing,with bytes.

My problem is this:

i've a byte that naturally is composed from 2 nibbles hi&low, and two
chars.. like A nd B. What i wonna do is to write A to the High nibble and B
to the the lower nibble.
Or an other example can be i've 2 numbers.. like 7 and 8 and whant to do the
same as for chars.

I'm really confused on how t do it, maybe cause python is type-less (dynamic
typed)

Any Help?

Cheers,
Gianmaria
ITALY

Gianmaria Iaculo - NVENTA, Nov 28, 2007

2. ### Tim ChaseGuest

> I'm really confused on how t do it, maybe cause python is
> type-less (dynamic typed)

Being duck-typed doesn't really have anything to do with it.
Python supports logical shifting and combining

> i've a byte that naturally is composed from 2 nibbles hi&low,
> and two chars.. like A nd B. What i wonna do is to write A to
> the High nibble and B to the the lower nibble. Or an other
> example can be i've 2 numbers.. like 7 and 8 and whant to do
> the same as for chars.

>>> a = int('1001', 2)
>>> b = int('0110', 2)
>>> a

9
>>> b

6
>>> 0xff & (((0xff & a) << 4) | (0xff & b))

150

or, if you're sloppy,

>>> (a << 4) | b

150

And for verification:

>>> int('10010110', 2)

150

Thus, that can be wrapped up as a function

>>> nibbles2byte = lambda a,b: \

0xff & (((0xff & a) << 4) | (0xff & b))
>>> nibbles2byte(a,b)

150

-tkc

Tim Chase, Nov 28, 2007

3. ### Chris MellonGuest

On Nov 28, 2007 2:07 PM, Gianmaria Iaculo - NVENTA
<> wrote:
> Hi there,
> I'm so new to python (coming from .net so excuse me for the stupid question)
> and i'm tring to do a very simple thing,with bytes.
>
> My problem is this:
>
> i've a byte that naturally is composed from 2 nibbles hi&low, and two
> chars.. like A nd B. What i wonna do is to write A to the High nibble and B
> to the the lower nibble.

A string in python is a sequence of bytes, so what you're describing
here is the string "AB".

> Or an other example can be i've 2 numbers.. like 7 and 8 and whant to do the
> same as for chars.
>

"\x07\x08"

> I'm really confused on how t do it, maybe cause python is type-less (dynamic
> typed)
>

You can use the struct module to convert back and forth between byte
sequences and numerical values. For example, to get an integer with
the value of the nibble you mentioned before:

struct.unpack("h", "AB") -> (16961,)

Exactly what you'll want to use and what format you want will depend
on why you're doing this.

Chris Mellon, Nov 28, 2007
4. ### Chris MellonGuest

On Nov 28, 2007 2:27 PM, Chris Mellon <> wrote:
> On Nov 28, 2007 2:07 PM, Gianmaria Iaculo - NVENTA
> <> wrote:
> > Hi there,
> > I'm so new to python (coming from .net so excuse me for the stupid question)
> > and i'm tring to do a very simple thing,with bytes.
> >
> > My problem is this:
> >
> > i've a byte that naturally is composed from 2 nibbles hi&low, and two
> > chars.. like A nd B. What i wonna do is to write A to the High nibble and B
> > to the the lower nibble.

>
> A string in python is a sequence of bytes, so what you're describing
> here is the string "AB".
>

Ah, I didn't realize until after I'd sent this that you were trying to
merge them into the same byte. This doesn't make a whole lot of sense
- ord("A") is outside the range you can represent in half a byte - but
Python does support the full range of bitwise operations, so you can
do whatever kind of shifting and setting that you'd have done in .NET.

Chris Mellon, Nov 28, 2007
5. ### John MachinGuest

On Nov 29, 7:07 am, "Gianmaria Iaculo - NVENTA"
<> wrote:
> Hi there,
> I'm so new to python (coming from .net so excuse me for the stupid question)
> and i'm tring to do a very simple thing,with bytes.
>
> My problem is this:
>
> i've a byte that naturally is composed from 2 nibbles hi&low, and two
> chars.. like A nd B. What i wonna do is to write A to the High nibble and B
> to the the lower nibble.

But a nibble is 4 bits and a char in general is 8 bits. Please explain
"write A to the high nibble". Let's assume that you mean that 0 <= A
<= 15 ....

The building blocks that you need are the ord() and chr() builtin
functions, and the << (shift-left) operator. The hex() function is
useful for seeing what is happening.

>>> a = '\x07'
>>> b = '\x08'
>>> c = 7
>>> d = 8
>>> ord(a)

7
>>> chr(c)

'\x07'
>>> hex(c)

'0x7'
>>> hex(c << 4)

'0x70'
>>> hex((ord(a) << 4) + ord(b))

'0x78'
>>> hex((c << 4) + d)

'0x78'
>>>

Cheers,
John

John Machin, Nov 28, 2007
6. ### Grant EdwardsGuest

On 2007-11-28, Chris Mellon <> wrote:
> On Nov 28, 2007 2:07 PM, Gianmaria Iaculo - NVENTA
><> wrote:
>> Hi there,
>> I'm so new to python (coming from .net so excuse me for the stupid question)
>> and i'm tring to do a very simple thing,with bytes.
>>
>> My problem is this:
>>
>> i've a byte that naturally is composed from 2 nibbles hi&low, and two
>> chars.. like A nd B. What i wonna do is to write A to the High nibble and B
>> to the the lower nibble.

>
> A string in python is a sequence of bytes, so what you're describing
> here is the string "AB".

No, he's describing something that consists of 2 nibbles (1
byte). The string "AB" is two bytes.

>> Or an other example can be i've 2 numbers.. like 7 and 8 and whant to do the
>> same as for chars.
>>

>
> "\x07\x08"

Again, he wants a single byte and that's two bytes.

>> I'm really confused on how t do it, maybe cause python is
>> type-less (dynamic typed)

>
> You can use the struct module to convert back and forth between byte
> sequences and numerical values. For example, to get an integer with
> the value of the nibble you mentioned before:
>
> struct.unpack("h", "AB") -> (16961,)

No, he wants to do this:

(0x0A<<4) | 0x0B

(7<<4) | 8

> Exactly what you'll want to use and what format you want will depend
> on why you're doing this.

--
Grant Edwards grante Yow! My vaseline is
at RUNNING...
visi.com

Grant Edwards, Nov 28, 2007
7. ### J. Clifford DyerGuest

On Wed, Nov 28, 2007 at 09:07:56PM +0100, Gianmaria Iaculo - NVENTA wrote regarding Bit Operations:
>
> Hi there,
> I'm so new to python (coming from .net so excuse me for the stupid question)
> and i'm tring to do a very simple thing,with bytes.
>
> My problem is this:
>
> i've a byte that naturally is composed from 2 nibbles hi&low, and two
> chars.. like A nd B. What i wonna do is to write A to the High nibble and B
> to the the lower nibble.
> Or an other example can be i've 2 numbers.. like 7 and 8 and whant to do the
> same as for chars.
>
> I'm really confused on how t do it, maybe cause python is type-less (dynamic
> typed)
>

Do you possibly mean that your letters are hexadecimal digits? If so, you can follow the advice given to you by others for numbers, treating your letters as numbers:

A=10
B=11
....
F=15

py>>> hex(15)
'0xf'
>>> int('f', 16)

15
>>> int('0xf', 16)

15

Cheers,
Cliff

J. Clifford Dyer, Nov 28, 2007
8. ### Dan UptonGuest

> >>> 0xff & (((0xff & a) << 4) | (0xff & b))
> 150
>
> or, if you're sloppy,
>
> >>> (a << 4) | b

> 150

Slightly OT, maybe - why exactly is the second alternative 'sloppy?'
I believe you, because I had a problem once (in Java) with bytes not
having the value I expected unless I did the and-magic, but I wasn't
clear on why. Is it an issue with the word otherwise possibly not
being zeroed out?

-dan

Dan Upton, Nov 28, 2007
9. ### Gianmaria Iaculo - NVENTAGuest

Txs all,
i wont to respond to who asked why i needed it:

I'm using python on GSM modules and the informations i have to move goes
along GPRS/UMTS connections so it's beatiful for me to transfer more
informations with less space...
imagine i have to send this simple data....

41.232323,12.345678

i can send it as it's or use the nibble trick and on the receiving station
'unlift" the data and rebuild the original information...

isn'it???

cheers + TXS,
Gianmaria

Firma Gianmaria Iaculo
"Gianmaria Iaculo - NVENTA" <> ha scritto nel messaggio
news:fikhra\$dmc\$...
> Hi there,
> I'm so new to python (coming from .net so excuse me for the stupid
> question) and i'm tring to do a very simple thing,with bytes.
>
> My problem is this:
>
> i've a byte that naturally is composed from 2 nibbles hi&low, and two
> chars.. like A nd B. What i wonna do is to write A to the High nibble and
> B to the the lower nibble.
> Or an other example can be i've 2 numbers.. like 7 and 8 and whant to do
> the same as for chars.
>
> I'm really confused on how t do it, maybe cause python is type-less
> (dynamic typed)
>
>
> Any Help?
>
> Cheers,
> Gianmaria
> ITALY
>
>

Gianmaria Iaculo - NVENTA, Nov 28, 2007
10. ### J. Clifford DyerGuest

On Wed, Nov 28, 2007 at 10:05:40PM +0100, Gianmaria Iaculo - NVENTA wrote regarding Re: Bit Operations:
>
> Txs all,
> i wont to respond to who asked why i needed it:
>
> I'm using python on GSM modules and the informations i have to move goes
> along GPRS/UMTS connections so it's beatiful for me to transfer more
> informations with less space...
> imagine i have to send this simple data....
>
> 41.232323,12.345678
>
> i can send it as it's or use the nibble trick and on the receiving station
> 'unlift" the data and rebuild the original information...
>
> isn'it???
>

Um, no. It isn't. How exactly are you going to pack floating point numbers into a half a byte?

Or are you sending it as strings? Also a waste of space, and unnecessarily complex.

J. Clifford Dyer, Nov 28, 2007
11. ### Chris MellonGuest

On Nov 28, 2007 3:18 PM, J. Clifford Dyer <> wrote:
> On Wed, Nov 28, 2007 at 10:05:40PM +0100, Gianmaria Iaculo - NVENTA wrote regarding Re: Bit Operations:
> >
> > Txs all,
> > i wont to respond to who asked why i needed it:
> >
> > I'm using python on GSM modules and the informations i have to move goes
> > along GPRS/UMTS connections so it's beatiful for me to transfer more
> > informations with less space...
> > imagine i have to send this simple data....
> >
> > 41.232323,12.345678
> >
> > i can send it as it's or use the nibble trick and on the receiving station
> > 'unlift" the data and rebuild the original information...
> >
> > isn'it???
> >

>
> Um, no. It isn't. How exactly are you going to pack floating point numbers into a half a byte?
>
> Or are you sending it as strings? Also a waste of space, and unnecessarily complex.
>

Assuming these are coordinates, not floats, using strings makes sense
but the zlib module is probably a much better choice than a
hand-written compression scheme.

Chris Mellon, Nov 28, 2007
12. ### Gianmaria Iaculo - NVENTAGuest

U are really nice guys... i'm really apreciating (sorry 4 my bad english)

Chriss is right this are coordinates.... and i'm treating as strings
naturally
I dont really have floating points on my module.. it run a 1.5 python
version from Telit.
So i dont have zLib too... just have 1.5 Mb of Ram and 3Mb of Rom... not
realy confortable..isn't it?

I'm tring some experiments on the command line... i've tried this:

My longitude is 42.237897

so as a first step... i created a X and done this job as your examples:

a = 4
b = 2

x = (a<<4)|b
x is 66

so i can do:

and i have the 4 again...( a value) but i've some problems while i decode
the b....

Where i go wrong?

Gianmaria

Firma Gianmaria Iaculo
"Chris Mellon" <> ha scritto nel messaggio
news:...
> On Nov 28, 2007 3:18 PM, J. Clifford Dyer <> wrote:
>> On Wed, Nov 28, 2007 at 10:05:40PM +0100, Gianmaria Iaculo - NVENTA wrote
>> regarding Re: Bit Operations:
>> >
>> > Txs all,
>> > i wont to respond to who asked why i needed it:
>> >
>> > I'm using python on GSM modules and the informations i have to move
>> > goes
>> > along GPRS/UMTS connections so it's beatiful for me to transfer more
>> > informations with less space...
>> > imagine i have to send this simple data....
>> >
>> > 41.232323,12.345678
>> >
>> > i can send it as it's or use the nibble trick and on the receiving
>> > station
>> > 'unlift" the data and rebuild the original information...
>> >
>> > isn'it???
>> >

>>
>> Um, no. It isn't. How exactly are you going to pack floating point
>> numbers into a half a byte?
>>
>> Or are you sending it as strings? Also a waste of space, and
>> unnecessarily complex.
>>

>
> Assuming these are coordinates, not floats, using strings makes sense
> but the zlib module is probably a much better choice than a
> hand-written compression scheme.

Gianmaria Iaculo - NVENTA, Nov 28, 2007
13. ### Tim ChaseGuest

>> >>> 0xff & (((0xff & a) << 4) | (0xff & b))
>> 150
>>
>> or, if you're sloppy,
>>
>> >>> (a << 4) | b

>> 150

>
> Slightly OT, maybe - why exactly is the second alternative 'sloppy?'
> I believe you, because I had a problem once (in Java) with bytes not
> having the value I expected unless I did the and-magic, but I wasn't
> clear on why. Is it an issue with the word otherwise possibly not
> being zeroed out?

Whoops...extra "f"s slipped into my nibble-mask

"Sloppy" lets through things like

>>> a = int('11111', 2) # overflows a nibble
>>> b = int('11111', 2)
>>> (a<<4) | b

511
>>> 0xff & (((0xf & a) << 4) | (0xf & b))

255

It clamps each nibble to a true nibble, and the output to a true
byte. If you validate your nibbles, you could be lazy yet
accurate with

>>> result = ((0xf & a) << 4) | (0xf & b)
>>> result

255

To get the nibbles back out of the resulting byte, one can simply

>>> a = 0xf & (result >> 4)
>>> b = result & 0xf

-tkc

Tim Chase, Nov 28, 2007
14. ### John MachinGuest

On Nov 29, 8:05 am, "Gianmaria Iaculo - NVENTA"
<> wrote:
> Txs all,
> i wont to respond to who asked why i needed it:
>
> I'm using python on GSM modules and the informations i have to move goes
> along GPRS/UMTS connections so it's beatiful for me to transfer more
> informations with less space...
> imagine i have to send this simple data....
>
> 41.232323,12.345678
>
> i can send it as it's or use the nibble trick and on the receiving station
> 'unlift" the data and rebuild the original information...
>
> isn'it???

Sorry, but it's not apparent what you propose to do. If each number
has 8 decimal digits of precision (as in your example), you could
possibly get by with a 32-bit floating point number. If it's always 6
decimal places and 0 <= number < 1000, you could pack (number *
1000000) into a 32-bit integer. For the above two options, check out
the struct module. OTOH, maybe it's "packed decimal" that you mean --
try Googling that phrase and see if it matches your intentions. If it
does, and you are concerned with speed, a 100-element dictionary
mapping each byte-pair to a packed byte might be a good idea instead
of the bit bashing:
convert = {
'78': '\x78',
...
}
See http://mail.python.org/pipermail/python-list/2000-October/056329.html

HTH,
John

John Machin, Nov 28, 2007
15. ### John MachinGuest

On Nov 29, 8:35 am, "Gianmaria Iaculo - NVENTA"
<> wrote:
> U are really nice guys... i'm really apreciating (sorry 4 my bad english)
>
> Chriss is right this are coordinates.... and i'm treating as strings
> naturally
> I dont really have floating points on my module.. it run a 1.5 python
> version from Telit.
> So i dont have zLib too... just have 1.5 Mb of Ram and 3Mb of Rom... not
> realy confortable..isn't it?
>
> I'm tring some experiments on the command line... i've tried this:
>
> My longitude is 42.237897
>
> so as a first step... i created a X and done this job as your examples:
>
> a = 4
> b = 2
>
> x = (a<<4)|b
> x is 66
>
> so i can do:
>
> aDecoded = x >> 4
>
> and i have the 4 again...( a value) but i've some problems while i decode
> the b....
>
> Where i go wrong?

>>> 66 & 0xf

2
>>>

John Machin, Nov 28, 2007
16. ### Gianmaria Iaculo - NVENTAGuest

John can you make an example of this solution? You maen that a more compact
way is possible???

Firma Gianmaria Iaculo
"John Machin" <> ha scritto nel messaggio
news:...
> On Nov 29, 8:05 am, "Gianmaria Iaculo - NVENTA"
> <> wrote:
>> Txs all,
>> i wont to respond to who asked why i needed it:
>>
>> I'm using python on GSM modules and the informations i have to move goes
>> along GPRS/UMTS connections so it's beatiful for me to transfer more
>> informations with less space...
>> imagine i have to send this simple data....
>>
>> 41.232323,12.345678
>>
>> i can send it as it's or use the nibble trick and on the receiving
>> station
>> 'unlift" the data and rebuild the original information...
>>
>> isn'it???

>
> Sorry, but it's not apparent what you propose to do. If each number
> has 8 decimal digits of precision (as in your example), you could
> possibly get by with a 32-bit floating point number. If it's always 6
> decimal places and 0 <= number < 1000, you could pack (number *
> 1000000) into a 32-bit integer. For the above two options, check out
> the struct module. OTOH, maybe it's "packed decimal" that you mean --
> try Googling that phrase and see if it matches your intentions. If it
> does, and you are concerned with speed, a 100-element dictionary
> mapping each byte-pair to a packed byte might be a good idea instead
> of the bit bashing:
> convert = {
> '78': '\x78',
> ...
> }
> See http://mail.python.org/pipermail/python-list/2000-October/056329.html
>
> HTH,
> John
>

Gianmaria Iaculo - NVENTA, Nov 28, 2007
17. ### John MachinGuest

On Nov 29, 9:20 am, "Gianmaria Iaculo - NVENTA"
<> wrote:
> John can you make an example of this solution?

Which possible solution? (a) 32-bit floating point (b) 32-bit integer
(c) packed decimal

> You maen that a more compact
> way is possible???

More compact than what? If your coordinates are all 8-digit decimal
numbers, then each of the above possible solutions will take 32 bits
per number. Which you could use depends on the (unstated) range and
precision of your coordinates. Also, you'll maybe notice that my
comment about using a dictionary for possible solution (c) was a
possible SPEED enhancement, not a compression enhancement.

John Machin, Nov 28, 2007