Reading hex to int from a binary string

Discussion in 'Python' started by Luc, Oct 8, 2009.

  1. Luc

    Luc Guest

    Hi all,

    I read data from a binary stream, so I get hex values as characters
    (in a string) with escaped x, like "\x05\x88", instead of 0x05.

    I am looking for a clean way to add these two values and turn them
    into an integer, knowing that calling int() with base 16 throws an
    invalid literal exception.

    Any help appreciated, thanks.
     
    Luc, Oct 8, 2009
    #1
    1. Advertising

  2. Luc schrieb:
    > Hi all,
    >
    > I read data from a binary stream, so I get hex values as characters
    > (in a string) with escaped x, like "\x05\x88", instead of 0x05.
    >
    > I am looking for a clean way to add these two values and turn them
    > into an integer, knowing that calling int() with base 16 throws an
    > invalid literal exception.
    >
    > Any help appreciated, thanks.


    Consider this (in the python interpreter):

    >>> chr(255)

    '\xff'
    >>> chr(255) == r"\xff"

    False
    >>> int(r"ff", 16)

    255

    In other words: no, you *don't* get hex values. You get bytes from the
    stream "as is", with python resorting to printing these out (in the
    interpreter!!!) as "\xXX". Python does that so that binary data will
    always have a "pretty" output when being inspected on the REPL.

    But they are bytes, and to convert them to an integer, you call "ord" on
    them.

    So assuming your string is read bytewise into two variables a & b, this
    is your desired code:

    >>> a = "\xff"
    >>> b = "\xa0"
    >>> ord(a) + ord(b)

    415


    HTH, Diez
     
    Diez B. Roggisch, Oct 8, 2009
    #2
    1. Advertising

  3. Luc

    Luc Guest

    On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
    > Luc schrieb:
    >
    > > Hi all,

    >
    > > I read data from a binary stream, so I get hex values as characters
    > > (in a string) with escaped x, like "\x05\x88", instead of 0x05.

    >
    > > I am looking for a clean way to add these two values and turn them
    > > into an integer, knowing that calling int() with base 16 throws an
    > > invalid literal exception.

    >
    > > Any help appreciated, thanks.

    >
    > Consider this (in the python interpreter):
    >
    >  >>> chr(255)
    > '\xff'
    >  >>> chr(255) == r"\xff"
    > False
    >  >>> int(r"ff", 16)
    > 255
    >
    > In other words: no, you *don't* get hex values. You get bytes from the
    > stream "as is", with python resorting to printing these out (in the
    > interpreter!!!) as "\xXX". Python does that so that binary data will
    > always have a "pretty" output when being inspected on the REPL.
    >
    > But they are bytes, and to convert them to an integer, you call "ord" on
    > them.
    >
    > So assuming your string is read bytewise into two variables a & b, this
    > is your desired code:
    >
    >  >>> a = "\xff"
    >  >>> b = "\xa0"
    >  >>> ord(a) + ord(b)
    > 415
    >
    > HTH, Diez


    Sorry I was not clear enough. When I said "add", I meant concatenate
    because I want to read 0x0588 as one value and ord() does not allow
    that.

    However you pointed me in the right direction and I found that int
    (binascii.hexlify(a + b, 16)) does the job.

    Thanks.
     
    Luc, Oct 8, 2009
    #3
  4. On Thu, 8 Oct 2009 14:52:33 -0700 (PDT), Luc <>
    declaimed the following in gmane.comp.python.general:

    > On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
    > > Luc schrieb:
    > >
    > > > Hi all,

    > >
    > > > I read data from a binary stream, so I get hex values as characters
    > > > (in a string) with escaped x, like "\x05\x88", instead of 0x05.

    > >
    > > > I am looking for a clean way to add these two values and turn them
    > > > into an integer, knowing that calling int() with base 16 throws an
    > > > invalid literal exception.

    > >
    > > > Any help appreciated, thanks.

    > >
    > > Consider this (in the python interpreter):
    > >
    > >  >>> chr(255)
    > > '\xff'
    > >  >>> chr(255) == r"\xff"
    > > False
    > >  >>> int(r"ff", 16)
    > > 255
    > >
    > > In other words: no, you *don't* get hex values. You get bytes from the
    > > stream "as is", with python resorting to printing these out (in the
    > > interpreter!!!) as "\xXX". Python does that so that binary data will
    > > always have a "pretty" output when being inspected on the REPL.
    > >
    > > But they are bytes, and to convert them to an integer, you call "ord" on
    > > them.
    > >
    > > So assuming your string is read bytewise into two variables a & b, this
    > > is your desired code:
    > >
    > >  >>> a = "\xff"
    > >  >>> b = "\xa0"
    > >  >>> ord(a) + ord(b)
    > > 415
    > >
    > > HTH, Diez

    >
    > Sorry I was not clear enough. When I said "add", I meant concatenate
    > because I want to read 0x0588 as one value and ord() does not allow
    > that.
    >
    > However you pointed me in the right direction and I found that int
    > (binascii.hexlify(a + b, 16)) does the job.
    >

    Yeesh... This is what struct is designed for...

    >>> import struct
    >>> something = "\x05\x88and more\r\n"
    >>> print something

    ˆand more

    >>>
    >>> (h1, st, h2) = struct.unpack("H8sh", something)
    >>> h1

    34821
    >>> st

    'and more'
    >>> h2

    2573
    >>>
    >>> print "%4x, %4x" % (h1, h2)

    8805, a0d

    You may need to adjust for expected endian mode...

    >>> (h1, st, h2) = struct.unpack(">H8sh", something)
    >>> print "%4.4x, %4.4x" % (h1, h2)

    0588, 0d0a
    >>> h1

    1416
    >>> h2

    3338
    >>>

    --
    Wulfraed Dennis Lee Bieber KD6MOG
    HTTP://wlfraed.home.netcom.com/
     
    Dennis Lee Bieber, Oct 9, 2009
    #4
  5. Luc

    Luc Guest

    On Oct 9, 3:12 am, Dennis Lee Bieber <> wrote:
    > On Thu, 8 Oct 2009 14:52:33 -0700 (PDT), Luc <>
    > declaimed the following in gmane.comp.python.general:
    >
    >
    >
    > > On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
    > > > Luc schrieb:

    >
    > > > > Hi all,

    >
    > > > > I read data from a binary stream, so I get hex values as characters
    > > > > (in a string) with escaped x, like "\x05\x88", instead of 0x05.

    >
    > > > > I am looking for a clean way to add these two values and turn them
    > > > > into an integer, knowing that calling int() with base 16 throws an
    > > > > invalid literal exception.

    >
    > > > > Any help appreciated, thanks.

    >
    > > > Consider this (in the python interpreter):

    >
    > > >  >>> chr(255)
    > > > '\xff'
    > > >  >>> chr(255) == r"\xff"
    > > > False
    > > >  >>> int(r"ff", 16)
    > > > 255

    >
    > > > In other words: no, you *don't* get hex values. You get bytes from the
    > > > stream "as is", with python resorting to printing these out (in the
    > > > interpreter!!!) as "\xXX". Python does that so that binary data will
    > > > always have a "pretty" output when being inspected on the REPL.

    >
    > > > But they are bytes, and to convert them to an integer, you call "ord" on
    > > > them.

    >
    > > > So assuming your string is read bytewise into two variables a & b, this
    > > > is your desired code:

    >
    > > >  >>> a = "\xff"
    > > >  >>> b = "\xa0"
    > > >  >>> ord(a) + ord(b)
    > > > 415

    >
    > > > HTH, Diez

    >
    > > Sorry I was not clear enough. When I said "add", I meant concatenate
    > > because I want to read 0x0588 as one value and ord() does not allow
    > > that.

    >
    > > However you pointed me in the right direction and I found that int
    > > (binascii.hexlify(a + b, 16)) does the job.

    >
    >         Yeesh... This is what   struct  is designed for...
    >
    > >>> import struct
    > >>> something = "\x05\x88and more\r\n"
    > >>> print something

    >
    >  ˆand more
    >
    >
    >
    > >>> (h1, st, h2) = struct.unpack("H8sh", something)
    > >>> h1

    > 34821
    > >>> st

    > 'and more'
    > >>> h2

    > 2573
    >
    > >>> print "%4x, %4x" % (h1, h2)

    >
    > 8805,  a0d
    >
    >         You may need to adjust for expected endian mode...
    >
    > >>> (h1, st, h2) = struct.unpack(">H8sh", something)
    > >>> print "%4.4x, %4.4x" % (h1, h2)

    > 0588, 0d0a
    > >>> h1

    > 1416
    > >>> h2

    > 3338
    >
    > --
    >         Wulfraed         Dennis Lee Bieber               KD6MOG
    >             HTTP://wlfraed.home.netcom.com/


    Nice, thanks!
     
    Luc, Oct 9, 2009
    #5
  6. Luc schrieb:
    > On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
    >> Luc schrieb:
    >>
    >>> Hi all,
    >>> I read data from a binary stream, so I get hex values as characters
    >>> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
    >>> I am looking for a clean way to add these two values and turn them
    >>> into an integer, knowing that calling int() with base 16 throws an
    >>> invalid literal exception.
    >>> Any help appreciated, thanks.

    >> Consider this (in the python interpreter):
    >>
    >> >>> chr(255)

    >> '\xff'
    >> >>> chr(255) == r"\xff"

    >> False
    >> >>> int(r"ff", 16)

    >> 255
    >>
    >> In other words: no, you *don't* get hex values. You get bytes from the
    >> stream "as is", with python resorting to printing these out (in the
    >> interpreter!!!) as "\xXX". Python does that so that binary data will
    >> always have a "pretty" output when being inspected on the REPL.
    >>
    >> But they are bytes, and to convert them to an integer, you call "ord" on
    >> them.
    >>
    >> So assuming your string is read bytewise into two variables a & b, this
    >> is your desired code:
    >>
    >> >>> a = "\xff"
    >> >>> b = "\xa0"
    >> >>> ord(a) + ord(b)

    >> 415
    >>
    >> HTH, Diez

    >
    > Sorry I was not clear enough. When I said "add", I meant concatenate
    > because I want to read 0x0588 as one value and ord() does not allow
    > that.


    (ord(a) << 8) + ord(b)


    Diez
     
    Diez B. Roggisch, Oct 9, 2009
    #6
  7. Luc

    Jack Norton Guest

    Luc wrote:
    > Hi all,
    >
    > I read data from a binary stream, so I get hex values as characters
    > (in a string) with escaped x, like "\x05\x88", instead of 0x05.
    >
    > I am looking for a clean way to add these two values and turn them
    > into an integer, knowing that calling int() with base 16 throws an
    > invalid literal exception.
    >
    > Any help appreciated, thanks.
    >

    Hi,

    Check out the ord() function.

    Example:
    x = '\x34'
    print ord(x)

    output: 52

    Also, if you, lets say read(4), and end up with `x = '\x05\x41\x24\x00'
    you can use x to address each character. So if you
    print x[1]
    output: 'A'

    That should be enough to get you started in the right direction.

    -Jack
     
    Jack Norton, Oct 9, 2009
    #7
  8. Luc

    Luc Guest

    On Oct 9, 10:45 am, "Diez B. Roggisch" <> wrote:
    > Luc schrieb:
    >
    >
    >
    > > On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
    > >> Luc schrieb:

    >
    > >>> Hi all,
    > >>> I read data from a binary stream, so I get hex values as characters
    > >>> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
    > >>> I am looking for a clean way to add these two values and turn them
    > >>> into an integer, knowing that calling int() with base 16 throws an
    > >>> invalid literal exception.
    > >>> Any help appreciated, thanks.
    > >> Consider this (in the python interpreter):

    >
    > >>  >>> chr(255)
    > >> '\xff'
    > >>  >>> chr(255) == r"\xff"
    > >> False
    > >>  >>> int(r"ff", 16)
    > >> 255

    >
    > >> In other words: no, you *don't* get hex values. You get bytes from the
    > >> stream "as is", with python resorting to printing these out (in the
    > >> interpreter!!!) as "\xXX". Python does that so that binary data will
    > >> always have a "pretty" output when being inspected on the REPL.

    >
    > >> But they are bytes, and to convert them to an integer, you call "ord" on
    > >> them.

    >
    > >> So assuming your string is read bytewise into two variables a & b, this
    > >> is your desired code:

    >
    > >>  >>> a = "\xff"
    > >>  >>> b = "\xa0"
    > >>  >>> ord(a) + ord(b)
    > >> 415

    >
    > >> HTH, Diez

    >
    > > Sorry I was not clear enough. When I said "add", I meant concatenate
    > > because I want to read 0x0588 as one value and ord() does not allow
    > > that.

    >
    > (ord(a) << 8) + ord(b)
    >
    > Diez


    Yes that too. But I have four bytes fields and single bit fields to
    deal with as well so I'll stick with struct.

    Thanks.
     
    Luc, Oct 9, 2009
    #8
  9. Luc schrieb:
    > On Oct 9, 10:45 am, "Diez B. Roggisch" <> wrote:
    >> Luc schrieb:
    >>
    >>
    >>
    >>> On Oct 8, 11:13 pm, "Diez B. Roggisch" <> wrote:
    >>>> Luc schrieb:
    >>>>> Hi all,
    >>>>> I read data from a binary stream, so I get hex values as characters
    >>>>> (in a string) with escaped x, like "\x05\x88", instead of 0x05.
    >>>>> I am looking for a clean way to add these two values and turn them
    >>>>> into an integer, knowing that calling int() with base 16 throws an
    >>>>> invalid literal exception.
    >>>>> Any help appreciated, thanks.
    >>>> Consider this (in the python interpreter):
    >>>> >>> chr(255)
    >>>> '\xff'
    >>>> >>> chr(255) == r"\xff"
    >>>> False
    >>>> >>> int(r"ff", 16)
    >>>> 255
    >>>> In other words: no, you *don't* get hex values. You get bytes from the
    >>>> stream "as is", with python resorting to printing these out (in the
    >>>> interpreter!!!) as "\xXX". Python does that so that binary data will
    >>>> always have a "pretty" output when being inspected on the REPL.
    >>>> But they are bytes, and to convert them to an integer, you call "ord" on
    >>>> them.
    >>>> So assuming your string is read bytewise into two variables a & b, this
    >>>> is your desired code:
    >>>> >>> a = "\xff"
    >>>> >>> b = "\xa0"
    >>>> >>> ord(a) + ord(b)
    >>>> 415
    >>>> HTH, Diez
    >>> Sorry I was not clear enough. When I said "add", I meant concatenate
    >>> because I want to read 0x0588 as one value and ord() does not allow
    >>> that.

    >> (ord(a) << 8) + ord(b)
    >>
    >> Diez

    >
    > Yes that too. But I have four bytes fields and single bit fields to
    > deal with as well so I'll stick with struct.


    For the future: it helps describing the actual problem, not something
    vaguely similar - this will get you better answers, and spare those who
    try to help you the effort to come up with solutions that aren't ones.

    Diez
     
    Diez B. Roggisch, Oct 10, 2009
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    10
    Views:
    6,343
    Neredbojias
    Aug 19, 2005
  2. Schnoffos
    Replies:
    2
    Views:
    1,246
    Martien Verbruggen
    Jun 27, 2003
  3. Hal Styli
    Replies:
    14
    Views:
    1,696
    Old Wolf
    Jan 20, 2004
  4. Bengt Richter
    Replies:
    6
    Views:
    504
    Juha Autero
    Aug 19, 2003
  5. jack
    Replies:
    4
    Views:
    602
Loading...

Share This Page