Socket connection between python and C

[Williamson, Ross X. (Guest)]

Dear All,

I'm trying to implement a server/client system where the server is written in python and the client has to be written in c/c++. I can happily send simple text through the socket. Ideally I would like make say a struct (using python struct library) - and then read that in using C. Is there a better way to package data on the server in python to send down a socket to a C client? XML? Pickle?

Cheers,

Ross

 

[Roy Smith]

Dear All,

I'm trying to implement a server/client system where the server is written in
python and the client has to be written in c/c++. I can happily send simple
text through the socket. Ideally I would like make say a struct (using python
struct library) - and then read that in using C. Is there a better way to
package data on the server in python to send down a socket to a C client?
XML? Pickle?

Depends on what you are trying to accomplish.

If your goal is for the communication to be as efficient as possible,
sending raw packed binary with the struct module on the python side, and
casting the i/o buffer pointer to a struct pointer on the C/C++ side
probably can't be beat. The downside is you need to worry about
low-level things like padding, overflow, and endian-ness yourself.

If you want to give up a little bit of efficiency in return for a huge
amount of convenience, I'd go with JSON. For the kinds of things you
might be thinking about using the struct module for, it's just peachy.
Hugely portable (libraries for every language imaginable), easy to use,
and not grossly inefficient. There's also BSON, which will be a bit
more efficient at the cost of some portability.

Based on your statement that you're thinking of using "struct", my guess
is that XML would be overkill.

 

[Dan Stromberg]

Depends on what you are trying to accomplish.

If your goal is for the communication to be as efficient as possible,
sending raw packed binary with the struct module on the python side, and
casting the i/o buffer pointer to a struct pointer on the C/C++ side
probably can't be beat.  The downside is you need to worry about
low-level things like padding, overflow, and endian-ness yourself.

Yes, this is fast, and yes, this likely won't be a good long-term
option if you envision someday using even slightly exotic (or new)
hardware - even using a different compiler on the same hardware could
lead to troubles with this approach.

However, socket.htons and related functions are a pretty good (and
somewhat similar, though without most of the problems) option.

If you want to give up a little bit of efficiency in return for a huge
amount of convenience, I'd go with JSON.  For the kinds of things you
might be thinking about using the struct module for, it's just peachy.
Hugely portable (libraries for every language imaginable), easy to use,
and not grossly inefficient.  There's also BSON, which will be a bit
more efficient at the cost of some portability.

JSON's cool.

Based on your statement that you're thinking of using "struct", my guess
is that XML would be overkill.

Yes, XML's a bit more heavyweight than JSON.

Also, if your data need not ever become hierarchical (which is another
architectural choice that could paint you into a corner a bit), or you
are prepared to add framing to your protocol if/when the time comes,
you can just use ASCII. This has the benefit of allowing you to test
your server with telnet.

Actually, JSON and XML should preserve telnet-based server testing as
well, at least to some extent - you might have to cut and paste your
tests more though.

 

[Irmen de Jong]

Dear All,

I'm trying to implement a server/client system where the server is written in python and the client has to be written in c/c++. I can happily send simple text through the socket. Ideally I would like make say a struct (using python struct library) - and then read that in using C. Is there a better way to package data on the server in python to send down a socket to a C client? XML? Pickle?

Cheers,

Ross

For a binary protocol I recall once using the marshal module from Python
to send simple data structures over a stream to a C program on the other
end. The marshal protocol is simple and was parsed by the C code on the
other end. Look in marshal.c in Python's source.

Actually now I think of it, it was the other way around: there was a
little bit of minimalistic code in a web server written in C, that
encoded stuff in the marshal protocol, and it was the Python process
that simply used the marshal module to get objects out of it. But I
think it shouldn't be hard to make this 2-way if you make some
restrictions on the data types you use.

Binary protocols have their drawbacks though, and maybe it's best to go
for Json instead for reasons mentioned already in the other replies.

-irmen

 

[Stefan Behnel]

I'm trying to implement a server/client system where the server is written in
python and the client has to be written in c/c++.

Is this a strict requirement? Could you give us a hint why the client needs
to be plain C/C++?

Stefan