Newbie: implementing lowlevel protocol over socket int32 data handling

Discussion in 'Python' started by Bram, Jul 31, 2003.

  1. Bram

    Bram Guest

    Hi,
    Summ: "What is the best way to handle int8,16 and 32 data in python?"

    Im currently working on a class to implement the gui protocol of
    mldonkey. (http://mldonkey.lemmster.de/wiki/index.php/GuiProtocol)
    However this requires to send int32 and int16 and even int8 integers.

    To create data I'm using the work-around of converting everything to hex
    values, and then converting the hex values to a data string wich I put
    on the socket (e.g. "\x00\xF0").
    I could do the same on the recieving end: chop it up in bytes and
    convert them to numbers (combining the hi and lo bytes of the int16 with
    some calculations)

    However, there must be a better way of doing this.
    Can anyone help me on this problem?

    Bram
     
    Bram, Jul 31, 2003
    #1
    1. Advertising

  2. Bram

    John Roth Guest

    "Bram" <> wrote in message
    news:bgbaj5$kgv$1.nb.home.nl...
    > Hi,
    > Summ: "What is the best way to handle int8,16 and 32 data in python?"
    >
    > Im currently working on a class to implement the gui protocol of
    > mldonkey. (http://mldonkey.lemmster.de/wiki/index.php/GuiProtocol)
    > However this requires to send int32 and int16 and even int8 integers.
    >
    > To create data I'm using the work-around of converting everything to hex
    > values, and then converting the hex values to a data string which I put
    > on the socket (e.g. "\x00\xF0").
    > I could do the same on the recieving end: chop it up in bytes and
    > convert them to numbers (combining the hi and lo bytes of the int16 with
    > some calculations)
    >
    > However, there must be a better way of doing this.
    > Can anyone help me on this problem?


    I took a quick look at the Wiki describing the protocol. For
    something like this, I'd expect that interoperability would be
    an issue, so the actual data format in the packets would be
    part of the protocol specification.

    That being the case, look at the struct module. I think it does
    exactly what you need.

    John Roth
    >
    > Bram
    >
    >
    >
     
    John Roth, Jul 31, 2003
    #2
    1. Advertising

  3. Bram

    Bram Guest

    Re: Newbie: implementing lowlevel protocol over socket int32 datahandling

    Thank you for replying so quickly.
    You thought right, this resolves all my problems :)
    Bram

    John Roth wrote:
    >
    > I took a quick look at the Wiki describing the protocol. For
    > something like this, I'd expect that interoperability would be
    > an issue, so the actual data format in the packets would be
    > part of the protocol specification.
    >
    > That being the case, look at the struct module. I think it does
    > exactly what you need.
    >
    > John Roth
     
    Bram, Jul 31, 2003
    #3
  4. Steve Horsley, Jul 31, 2003
    #4
  5. Bram

    Tero Saarni Guest

    "Rob Renaud" <> wrote in message
    news:...
    > Bram <> wrote in message

    news:<bgbaj5$kgv$1.nb.home.nl>...
    > > Hi,
    > > Summ: "What is the best way to handle int8,16 and 32 data in python?"
    > >
    > > Im currently working on a class to implement the gui protocol of
    > > mldonkey. (http://mldonkey.lemmster.de/wiki/index.php/GuiProtocol)
    > > However this requires to send int32 and int16 and even int8 integers.
    > >
    > > To create data I'm using the work-around of converting everything to hex
    > > values, and then converting the hex values to a data string wich I put
    > > on the socket (e.g. "\x00\xF0").
    > > I could do the same on the recieving end: chop it up in bytes and
    > > convert them to numbers (combining the hi and lo bytes of the int16 with
    > > some calculations)
    > >
    > > However, there must be a better way of doing this.
    > > Can anyone help me on this problem?
    > >
    > > Bram

    >
    > This should make it a lot easier for you.
    >
    > http://www.python.org/doc/current/lib/module-struct.html
    >
    > But this will probably be the 4th post containing such a reference.
    > As undoubtedly 3 other people have posted it, but have not yet shown
    > up on google groups.


    Instead of the "raw" Struct I was experimenting couple of days ago
    with writing something similar to Java ByteBuffer [1]. I use it like this:

    ---------------------------------
    writebuf = ByteBuffer.ByteBufferWriter(file('test.dat', 'wb'))

    writebuf.uint8(0x01)
    writebuf.uint16(0x0203)
    writebuf.string('abc', 4)
    writebuf.uint32(0x04050607)
    writebuf.flush()

    readbuf = ByteBuffer.ByteBufferReader(file('test.dat', 'rb'))

    print "uint8=0x%02x" % readbuf.uint8()
    print "uint16=0x%04x" % readbuf.uint16()
    print "string=%s" % readbuf.string(4)
    print "uint32=0x%08x" % readbuf.uint32()
    ---------------------------------

    This code will output:

    uint8=0x01
    uint16=0x0203
    string=abc
    uint32=0x04050607

    On Intel machine the contents of test.dat will be:
    > od -t x1c test.dat

    0000000 01 03 02 61 62 63 00 07 06 05 04
    001 003 002 a b c \0 \a 006 005 004
    0000013

    The byte order can be changed:
    writebuf.order(ByteBuffer.BIG_ENDIAN)

    My intention is that the classes take a file like object to
    work with so using socket instead of file should be fine.

    See the class ByteBuffer.py in the attachment in this post.

    Please note that the only test I've executed is the code above
    so there is no warranties that the classes will work at all ;)

    BR,
    Tero

    [1] http://java.sun.com/j2se/1.4.1/docs/api/java/nio/ByteBuffer.html



    import struct



    class ByteBuffer:

    NATIVE = '@'
    BIG_ENDIAN = '>'
    LITTLE_ENDIAN = '<'
    NETWORK = '!'

    def __init__(self, stream):
    self._stream = stream
    self._order = ByteBuffer.NATIVE

    def order(self, byteorder):
    self._order = byteorder
    return self



    class ByteBufferReader(ByteBuffer):

    DEFAULT_BUF_SIZE = 512

    def __init__(self, stream, bufsize=DEFAULT_BUF_SIZE):
    self._bufsize = max(4, bufsize)
    self._buf = ""
    self._bufptr = self._bufsize
    ByteBuffer.__init__(self, stream)


    def _readData(self, byteswanted):
    bytesleft = self._bufsize - self._bufptr
    if bytesleft < byteswanted:
    data = self._buf
    self._buf = self._stream.read(self._bufsize)
    self._bufsize = len(self._buf)
    data += self._buf[:byteswanted - bytesleft]
    self._bufptr = byteswanted - bytesleft
    else:
    data = self._buf[self._bufptr:self._bufptr + byteswanted]
    self._bufptr += byteswanted
    return data


    def int8(self):
    return struct.unpack('b', self._readData(1))

    def uint8(self):
    return struct.unpack('B', self._readData(1))

    def int16(self):
    return struct.unpack(self._order+'h', self._readData(2))

    def uint16(self):
    return struct.unpack(self._order+'H', self._readData(2))

    def int32(self):
    return struct.unpack(self._order+'i', self._readData(4))

    def uint32(self):
    return struct.unpack(self._order+'I', self._readData(4))

    def string(self, len):
    return struct.unpack(str(len)+'s', self._readData(len))




    class ByteBufferWriter(ByteBuffer):

    def __init__(self, stream):
    ByteBuffer.__init__(self, stream)


    def int8(self, value):
    self._stream.write(struct.pack('b', value))
    return self

    def uint8(self, value):
    self._stream.write(struct.pack('B', value))
    return self

    def int16(self, value):
    self._stream.write(struct.pack(self._order+'h', value))
    return self

    def uint16(self, value):
    self._stream.write(struct.pack(self._order+'H', value))
    return self

    def int32(self, value):
    self._stream.write(struct.pack(self._order+'i', value))
    return self

    def uint32(self, value):
    self._stream.write(struct.pack(self._order+'I', value))
    return self

    def string(self, value, len):
    self._stream.write(struct.pack(str(len)+'s', value))
    return self


    def flush(self):
    self._stream.flush()
    return self
     
    Tero Saarni, Aug 2, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Robert V. Hanson
    Replies:
    2
    Views:
    506
    Robert Hanson
    Jul 1, 2003
  2. =?Utf-8?B?SGlmbmkgU2hhaHphcmQ=?=

    Invalid cast from System.Int32 to System.Byte[].

    =?Utf-8?B?SGlmbmkgU2hhaHphcmQ=?=, Feb 15, 2005, in forum: ASP .Net
    Replies:
    1
    Views:
    3,803
    William F. Robertson, Jr.
    Feb 15, 2005
  3. Simon Harris
    Replies:
    2
    Views:
    2,624
    Simon Harris
    Apr 27, 2005
  4. Andre Renee
    Replies:
    8
    Views:
    1,106
    OscarM
    Apr 11, 2007
  5. Guillaume Comte
    Replies:
    16
    Views:
    709
    Grant Edwards
    Aug 23, 2012
Loading...

Share This Page