help on packet format for tcp/ip programming

[rattan]

I want a specific packet format for packet exchange between a client
server across the network. For example frame format
as a python class could be:
class Frame:
def __init__(self, buffer=None, count=0, offset=0):
self.buffer = buffer
self.count = count
self.offset = offset
the question is how to convert it to a byte stream so that format of
count and offset also becomes a sequence of bytes. I tried
the pickle module as:
a = Frame()
dat = pickle.dumps(a)
but the size of dat is quite large almost to 4 X the size of a, which
probably is an overkill for the numbers of bytes exchanged. Is there a
simpler solution (similar to C language -- set aside a buffer and fill
it with bytes and network byteorder values for count and offset and
send it out, there is no increase in byte count in the outgoing
packet)?

Any pointers will be appreciated.
-ishwar

 

[Bjoern Schliessmann]

I want a specific packet format for packet exchange between a
client server across the network. For example frame format
as a python class could be:
class Frame:
def __init__(self, buffer=None, count=0, offset=0):
self.buffer = buffer
self.count = count
self.offset = offset
the question is how to convert it to a byte stream so that format
of count and offset also becomes a sequence of bytes.

Try struct.

Regards,


Björn

 

[rattan]

Try struct.

Regards,

Björn

--
BOFH excuse #208:

Your mail is being routed through Germany ... and they're censoring
us.

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

-ishwar

 

[Felipe Almeida Lessa]

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Then send the size of the buffer before the buffer, so the recipient
can expect that many bytes.

 

[Gabriel Genellina]

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Use struct to pack count and offset into a fixed size field (2 or 4 bytes
depending on range); send buffer after that.
buffer must come *after* you send its size, else it's a lot harder to
retrieve at the other end.

 

[Grant Edwards]

struct module pack and unpack will only work for fixed size buffer :
pack('>1024sIL', buffer, count. offset) but the buffer size can vary
from one packet to the next :-(

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

 

[rattan]

Oh for Pete's sake...

struct.pack('>%dsIL' % len(buffer), buffer, count, offset)

that is great but how does one unpack on the other side?

-ishwar

 

[Diez B. Roggisch]

that is great but how does one unpack on the other side?

By peeking into the header, determining the number of bytes necessary?

But why do you need this anyway - if you know that you will have python on
both ends, use Pyro or xmlrpc.

Diez

 

[Grant Edwards]

that is great but how does one unpack on the other side?

struct.unpack('>%dsIL' % buflen ,packet)

 

[Grant Edwards]

By peeking into the header, determining the number of bytes necessary?

Better yet, use a protocol that's not brain-dead: send the
buffer size _before_ you send the buffer.