# Reading hex to int from a binary string

Discussion in 'Python' started by Luc, Oct 8, 2009.

1. ### LucGuest

Hi all,

I read data from a binary stream, so I get hex values as characters
(in a string) with escaped x, like "\x05\x88", instead of 0x05.

I am looking for a clean way to add these two values and turn them
into an integer, knowing that calling int() with base 16 throws an
invalid literal exception.

Any help appreciated, thanks.

Luc, Oct 8, 2009

2. ### Diez B. RoggischGuest

Luc schrieb:
> Hi all,
>
> I read data from a binary stream, so I get hex values as characters
> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
>
> I am looking for a clean way to add these two values and turn them
> into an integer, knowing that calling int() with base 16 throws an
> invalid literal exception.
>
> Any help appreciated, thanks.

Consider this (in the python interpreter):

>>> chr(255)

'\xff'
>>> chr(255) == r"\xff"

False
>>> int(r"ff", 16)

255

In other words: no, you *don't* get hex values. You get bytes from the
stream "as is", with python resorting to printing these out (in the
interpreter!!!) as "\xXX". Python does that so that binary data will
always have a "pretty" output when being inspected on the REPL.

But they are bytes, and to convert them to an integer, you call "ord" on
them.

So assuming your string is read bytewise into two variables a & b, this

>>> a = "\xff"
>>> b = "\xa0"
>>> ord(a) + ord(b)

415

HTH, Diez

Diez B. Roggisch, Oct 8, 2009

3. ### LucGuest

On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
> Luc schrieb:
>
> > Hi all,

>
> > I read data from a binary stream, so I get hex values as characters
> > (in a string) with escaped x, like "\x05\x88", instead of 0x05.

>
> > I am looking for a clean way to add these two values and turn them
> > into an integer, knowing that calling int() with base 16 throws an
> > invalid literal exception.

>
> > Any help appreciated, thanks.

>
> Consider this (in the python interpreter):
>
>  >>> chr(255)
> '\xff'
>  >>> chr(255) == r"\xff"
> False
>  >>> int(r"ff", 16)
> 255
>
> In other words: no, you *don't* get hex values. You get bytes from the
> stream "as is", with python resorting to printing these out (in the
> interpreter!!!) as "\xXX". Python does that so that binary data will
> always have a "pretty" output when being inspected on the REPL.
>
> But they are bytes, and to convert them to an integer, you call "ord" on
> them.
>
> So assuming your string is read bytewise into two variables a & b, this
>
>  >>> a = "\xff"
>  >>> b = "\xa0"
>  >>> ord(a) + ord(b)
> 415
>
> HTH, Diez

Sorry I was not clear enough. When I said "add", I meant concatenate
because I want to read 0x0588 as one value and ord() does not allow
that.

However you pointed me in the right direction and I found that int
(binascii.hexlify(a + b, 16)) does the job.

Thanks.

Luc, Oct 8, 2009
4. ### Dennis Lee BieberGuest

On Thu, 8 Oct 2009 14:52:33 -0700 (PDT), Luc <>
declaimed the following in gmane.comp.python.general:

> On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
> > Luc schrieb:
> >
> > > Hi all,

> >
> > > I read data from a binary stream, so I get hex values as characters
> > > (in a string) with escaped x, like "\x05\x88", instead of 0x05.

> >
> > > I am looking for a clean way to add these two values and turn them
> > > into an integer, knowing that calling int() with base 16 throws an
> > > invalid literal exception.

> >
> > > Any help appreciated, thanks.

> >
> > Consider this (in the python interpreter):
> >
> >  >>> chr(255)
> > '\xff'
> >  >>> chr(255) == r"\xff"
> > False
> >  >>> int(r"ff", 16)
> > 255
> >
> > In other words: no, you *don't* get hex values. You get bytes from the
> > stream "as is", with python resorting to printing these out (in the
> > interpreter!!!) as "\xXX". Python does that so that binary data will
> > always have a "pretty" output when being inspected on the REPL.
> >
> > But they are bytes, and to convert them to an integer, you call "ord" on
> > them.
> >
> > So assuming your string is read bytewise into two variables a & b, this
> > is your desired code:
> >
> >  >>> a = "\xff"
> >  >>> b = "\xa0"
> >  >>> ord(a) + ord(b)
> > 415
> >
> > HTH, Diez

>
> Sorry I was not clear enough. When I said "add", I meant concatenate
> because I want to read 0x0588 as one value and ord() does not allow
> that.
>
> However you pointed me in the right direction and I found that int
> (binascii.hexlify(a + b, 16)) does the job.
>

Yeesh... This is what struct is designed for...

>>> import struct
>>> something = "\x05\x88and more\r\n"
>>> print something

ˆand more

>>>
>>> (h1, st, h2) = struct.unpack("H8sh", something)
>>> h1

34821
>>> st

'and more'
>>> h2

2573
>>>
>>> print "%4x, %4x" % (h1, h2)

8805, a0d

You may need to adjust for expected endian mode...

>>> (h1, st, h2) = struct.unpack(">H8sh", something)
>>> print "%4.4x, %4.4x" % (h1, h2)

0588, 0d0a
>>> h1

1416
>>> h2

3338
>>>

--
Wulfraed Dennis Lee Bieber KD6MOG
HTTP://wlfraed.home.netcom.com/

Dennis Lee Bieber, Oct 9, 2009
5. ### LucGuest

On Oct 9, 3:12Â am, Dennis Lee Bieber <> wrote:
> On Thu, 8 Oct 2009 14:52:33 -0700 (PDT), Luc <>
> declaimed the following in gmane.comp.python.general:
>
>
>
> > On Oct 8, 11:13Â pm, "Diez B. Roggisch" <> wrote:
> > > Luc schrieb:

>
> > > > Hi all,

>
> > > > I read data from a binary stream, so I get hex values as characters
> > > > (in a string) with escaped x, like "\x05\x88", instead of 0x05.

>
> > > > I am looking for a clean way to add these two values and turn them
> > > > into an integer, knowing that calling int() with base 16 throws an
> > > > invalid literal exception.

>
> > > > Any help appreciated, thanks.

>
> > > Consider this (in the python interpreter):

>
> > > Â >>> chr(255)
> > > '\xff'
> > > Â >>> chr(255) == r"\xff"
> > > False
> > > Â >>> int(r"ff", 16)
> > > 255

>
> > > In other words: no, you *don't* get hex values. You get bytes from the
> > > stream "as is", with python resorting to printing these out (in the
> > > interpreter!!!) as "\xXX". Python does that so that binary data will
> > > always have a "pretty" output when being inspected on the REPL.

>
> > > But they are bytes, and to convert them to an integer, you call "ord" on
> > > them.

>
> > > So assuming your string is read bytewise into two variables a & b, this
> > > is your desired code:

>
> > > Â >>> a = "\xff"
> > > Â >>> b = "\xa0"
> > > Â >>> ord(a) + ord(b)
> > > 415

>
> > > HTH, Diez

>
> > Sorry I was not clear enough. When I said "add", I meant concatenate
> > because I want to read 0x0588 as one value and ord() does not allow
> > that.

>
> > However you pointed me in the right direction and I found that int
> > (binascii.hexlify(a + b, 16)) does the job.

>
> Â  Â  Â  Â  Yeesh... This is what Â  struct Â is designed for...
>
> >>> import struct
> >>> something = "\x05\x88and more\r\n"
> >>> print something

>
> Â Ë†and more
>
>
>
> >>> (h1, st, h2) = struct.unpack("H8sh", something)
> >>> h1

> 34821
> >>> st

> 'and more'
> >>> h2

> 2573
>
> >>> print "%4x, %4x" % (h1, h2)

>
> 8805, Â a0d
>
> Â  Â  Â  Â  You may need to adjust for expected endian mode...
>
> >>> (h1, st, h2) = struct.unpack(">H8sh", something)
> >>> print "%4.4x, %4.4x" % (h1, h2)

> 0588, 0d0a
> >>> h1

> 1416
> >>> h2

> 3338
>
> --
> Â  Â  Â  Â  Wulfraed Â  Â  Â  Â  Dennis Lee Bieber Â  Â  Â  Â  Â  Â  Â  KD6MOG
> Â  Â  Â  Â  Â  Â  HTTP://wlfraed.home.netcom.com/

Nice, thanks!

Luc, Oct 9, 2009
6. ### Diez B. RoggischGuest

Luc schrieb:
> On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
>> Luc schrieb:
>>
>>> Hi all,
>>> I read data from a binary stream, so I get hex values as characters
>>> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
>>> I am looking for a clean way to add these two values and turn them
>>> into an integer, knowing that calling int() with base 16 throws an
>>> invalid literal exception.
>>> Any help appreciated, thanks.

>> Consider this (in the python interpreter):
>>
>> >>> chr(255)

>> '\xff'
>> >>> chr(255) == r"\xff"

>> False
>> >>> int(r"ff", 16)

>> 255
>>
>> In other words: no, you *don't* get hex values. You get bytes from the
>> stream "as is", with python resorting to printing these out (in the
>> interpreter!!!) as "\xXX". Python does that so that binary data will
>> always have a "pretty" output when being inspected on the REPL.
>>
>> But they are bytes, and to convert them to an integer, you call "ord" on
>> them.
>>
>> So assuming your string is read bytewise into two variables a & b, this
>>
>> >>> a = "\xff"
>> >>> b = "\xa0"
>> >>> ord(a) + ord(b)

>> 415
>>
>> HTH, Diez

>
> Sorry I was not clear enough. When I said "add", I meant concatenate
> because I want to read 0x0588 as one value and ord() does not allow
> that.

(ord(a) << 8) + ord(b)

Diez

Diez B. Roggisch, Oct 9, 2009
7. ### Jack NortonGuest

Luc wrote:
> Hi all,
>
> I read data from a binary stream, so I get hex values as characters
> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
>
> I am looking for a clean way to add these two values and turn them
> into an integer, knowing that calling int() with base 16 throws an
> invalid literal exception.
>
> Any help appreciated, thanks.
>

Hi,

Check out the ord() function.

Example:
x = '\x34'
print ord(x)

output: 52

Also, if you, lets say read(4), and end up with `x = '\x05\x41\x24\x00'
you can use x to address each character. So if you
print x[1]
output: 'A'

That should be enough to get you started in the right direction.

-Jack

Jack Norton, Oct 9, 2009
8. ### LucGuest

On Oct 9, 10:45 am, "Diez B. Roggisch" <> wrote:
> Luc schrieb:
>
>
>
> > On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
> >> Luc schrieb:

>
> >>> Hi all,
> >>> I read data from a binary stream, so I get hex values as characters
> >>> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
> >>> I am looking for a clean way to add these two values and turn them
> >>> into an integer, knowing that calling int() with base 16 throws an
> >>> invalid literal exception.
> >>> Any help appreciated, thanks.
> >> Consider this (in the python interpreter):

>
> >>  >>> chr(255)
> >> '\xff'
> >>  >>> chr(255) == r"\xff"
> >> False
> >>  >>> int(r"ff", 16)
> >> 255

>
> >> In other words: no, you *don't* get hex values. You get bytes from the
> >> stream "as is", with python resorting to printing these out (in the
> >> interpreter!!!) as "\xXX". Python does that so that binary data will
> >> always have a "pretty" output when being inspected on the REPL.

>
> >> But they are bytes, and to convert them to an integer, you call "ord" on
> >> them.

>
> >> So assuming your string is read bytewise into two variables a & b, this
> >> is your desired code:

>
> >>  >>> a = "\xff"
> >>  >>> b = "\xa0"
> >>  >>> ord(a) + ord(b)
> >> 415

>
> >> HTH, Diez

>
> > Sorry I was not clear enough. When I said "add", I meant concatenate
> > because I want to read 0x0588 as one value and ord() does not allow
> > that.

>
> (ord(a) << 8) + ord(b)
>
> Diez

Yes that too. But I have four bytes fields and single bit fields to
deal with as well so I'll stick with struct.

Thanks.

Luc, Oct 9, 2009
9. ### Diez B. RoggischGuest

Luc schrieb:
> On Oct 9, 10:45 am, "Diez B. Roggisch" <> wrote:
>> Luc schrieb:
>>
>>
>>
>>> On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
>>>> Luc schrieb:
>>>>> Hi all,
>>>>> I read data from a binary stream, so I get hex values as characters
>>>>> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
>>>>> I am looking for a clean way to add these two values and turn them
>>>>> into an integer, knowing that calling int() with base 16 throws an
>>>>> invalid literal exception.
>>>>> Any help appreciated, thanks.
>>>> Consider this (in the python interpreter):
>>>> >>> chr(255)
>>>> '\xff'
>>>> >>> chr(255) == r"\xff"
>>>> False
>>>> >>> int(r"ff", 16)
>>>> 255
>>>> In other words: no, you *don't* get hex values. You get bytes from the
>>>> stream "as is", with python resorting to printing these out (in the
>>>> interpreter!!!) as "\xXX". Python does that so that binary data will
>>>> always have a "pretty" output when being inspected on the REPL.
>>>> But they are bytes, and to convert them to an integer, you call "ord" on
>>>> them.
>>>> So assuming your string is read bytewise into two variables a & b, this
>>>> >>> a = "\xff"
>>>> >>> b = "\xa0"
>>>> >>> ord(a) + ord(b)
>>>> 415
>>>> HTH, Diez
>>> Sorry I was not clear enough. When I said "add", I meant concatenate
>>> because I want to read 0x0588 as one value and ord() does not allow
>>> that.

>> (ord(a) << 8) + ord(b)
>>
>> Diez

>
> Yes that too. But I have four bytes fields and single bit fields to
> deal with as well so I'll stick with struct.

For the future: it helps describing the actual problem, not something
vaguely similar - this will get you better answers, and spare those who
try to help you the effort to come up with solutions that aren't ones.

Diez

Diez B. Roggisch, Oct 10, 2009