C to Java socket input question

R

Richard

(e-mail address removed) wrote...
(e-mail address removed) wrote...
Richard wrote:
My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.
Treat it as a 4 byte array. InetAddress.getByAddress() wants a byte[]
anyway.
Sorry for being brain-dead, but will he be able to do arithmetic on
it if it's in a 4-byte array? As in: unsigned char fbarray[4] ; ?

No, but why do you want to do arithmetic on IP addresses?

The ints aren't exclusively IP addresses. See above; I'm passing 32-
bit integers (the "e.g. the low level form of IP addresses" means
"for example, ... IP addresses").
 
N

Nigel Wade

Thomas Gagné wrote:
If the question had come from a computer I might have suggested it use
something more. But since people are involved text is still preferable.
Will
it be a machine debugging the code?

Of course it will, how would you intend to execute the code?
Will it be a machine translating the
network-ordered bytes into integers?

Of course it will, or did you think they intended to do it by hand?
Will it be a machine snooping
network packets wondering why the message aren't being
assembled/disassembled properly?

I rather think so, how do you intend to snoop the values otherwise?
Are you able to read packets without one?
You're right, machines are capable of more so humans must depend on them
to do more, like translate scalar values into text so we can read them, or
format them into XML so we can both read and understand their format by
reading them.

So use it for that. When you want to look at the values as text, use the
machine to convert them for you. But when the data is being sent
machine-to-machine text is inappropriate unless it's actually text that is
being transferred. The extra network overhead of sending text instead of
binary is totally unnecessary, as is the CPU overhead of converting every
number to and from text at either end.

I presume you don't send much data over networks.
Not using text is a good approach for job security.

Sorry, but that is rubbish.
 
T

Tukla Ratte

Thomas G. Marshall wrote:


For some reason this always leaves a bad taste for me.

Maybe because it doubles your bandwidth requirements?

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.
 
D

David Zimmerman

Tukla said:
Maybe because it doubles your bandwidth requirements?

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.
Why not just pass the bytes? Why complicate your life and increase the
demand on bandwidth?
 
J

Jon A. Cruz

Tukla said:
I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

Network byte order.

explicit sizes.

stdint.h helps now.

floats in IEE754

XML is probably a poor choice for that.

(I've been using XML for quite some time. I even use it to make source
code. Abused, it's bad)
 
X

xarax

Maybe because it doubles your bandwidth requirements?

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

I use a custom program generator that reads an XML specification
file that describes the various message packet types. The program
generator then spits out Java classes and C source files for each
message packet type.

Each message packet has a fixed length header that includes:

a. A 4-byte magic number for verifying that the byte stream
consists of message packets that I expect.

b. A 4-byte transaction ID number.

c. A 4-byte version number (another constant for sanity checking).

d. A 2-byte message type number that includes a "reply flag"
that indicates whether this is an original message, or a reply
to an earlier message (with the same transaction ID).

e. A 2-byte message length that must correlate to the known
message length for the message type (another sanity check).

If the header portion doesn't match expected values (i.e.,
the sanity checks fail), then the TCP/IP socket is closed
(causing a reset condition on the other side).

If the message packet also has payload data fields, then
those fields follow the header.

The generated Java code has a class for each message type.
The class knows how to read/write a message packet using
the java.nio.ByteBuffer class, which has built-in support
for Big Endian vs. Little Endian byte ordering. (My application
uses Big Endian.) The java class has getter/setter methods
for each field in the message packet.

The generated C code has header files that define a C struct
for mapping the message packet, and source files for each
message packet. The source file for a message packet has
functions for getting/setting each field in the buffer.

Neither the Java or C application code refer directly
to the packet fields, but rather use the getter/setter
routines. This extra level of indirection makes the
application more readable and easier to maintain.

I also have GUI front-end for specifying the message
packets, which generates the XML specification file and
invokes the program generator. I then compile the sources
and rebuild the application. Very simple and reliable.

This design removes any knowledge of the "other side of
the TCP/IP pipeline". Each side doesn't know whether it's
talking to a Java or a C application on the other side,
and it doesn't need to know.
 
S

Steve Horsley

I'm also trying to figure out a good way to pass binary data between
Java and C (well, C++). Right now, I'm leaning toward Base64 MIME
encoding, but I need to see if XML could be of any use.

Have you considered TCP/IP? It provides a byte stream. Bytes in one end,
same bytes out the other end - no loss and no re-ordering. This is
probably exactly what you need.

Steve
 
J

Jos A. Horsmeier

Richard said:
Level: Java newbie, C experienced
Platform: Linux and Win32, Intel

Another programmer and I are working on a small project together.
He's writing a server process in Java that accepts input from
processes I've written over a TCP connection. My processes are all
written in C; his are all done in Java. He's new to Java, and I've
never really used it.

My input is basically a stream of 32-bit unsigned integers (e.g., the
low-level form of IP addresses) along with the occasional fixed-
length char array.

I'm hopping in a bit late, but nevertheless. I've implemented something
similar and, the lazy bones that I am, I just wrapped a DataInputStream
and DataOutputStream around the raw SocketStream (this is the Java side
of the story). Those two streams take care of about anything.

If the other side happens to the 'C side' running on some box, all the
other side has to do is use the htonl(), ntohl() macros and their compadres
to send/receive binary integer data. Binary byte arrays can be passed
around without conversion.

Strings can be passed around using a little trick -- Java characters take
up 16 bits (the Unicode stuff) while, normally the C side uses char[]s
to store their strings. (wide characters are a different cup of tea).
This is how Java Strings can be written to the socket:

void write(String s) {
byte[] buf= s.getBytes("ISO-8859-1");
write(buf);
}

where the 'write(buf)' call simply writes a bunch of bytes. The other
side (the C side) simply reads the chars and stores them somewhere.
Sending strings (char[]s) from the C side to the Java side is easy
too. Simply read a bunch of bytes in a Java byte[] buf and create
a new Java String as follows:

String s= new String(buf, "ISO-8859-1");

Double values are easy to send/receive too; simple reverse the byte
order on the C side before sending and after receiving and simply
read/write the double on the Java side.

Of course you still have to design a little protocol, say, sending
the length of the next datum first (using those htonl() thingies
on the C side and using the simple DataOutputStream object on the
Java side), but that is left as an exercide for the reader.

kind regards,

Jos
 
?

=?ISO-8859-1?Q?Thomas_Gagn=E9?=

Jon said:
Network byte order.

explicit sizes.

And what sizes should those be? And what happens when something exceeds the
limits of an integer? Should the interface be rewritten?
stdint.h helps now.

floats in IEE754

What if more precision is required and/or demanded? Why not fractions or some
other expression that can be arbitrarily precise?
XML is probably a poor choice for that.

XML or any parsable text-based information would be more flexible than scalar
values. Admittedly they would use more bandwidth but I haven't detected any
bandwidth requirements--just bandwidth anxieties. Perhaps everyone would like
to omit the century from our dates?

Bandwidth increases every year--more programmers should take advantage of
it--unless of course you're sending spam.
 
J

Jon A. Cruz

Jos said:
I'm hopping in a bit late, but nevertheless. I've implemented something
similar and, the lazy bones that I am, I just wrapped a DataInputStream
and DataOutputStream around the raw SocketStream (this is the Java side
of the story). Those two streams take care of about anything.

Bingo!

That's probably much of why they are as they are.



If the other side happens to the 'C side' running on some box, all the
other side has to do is use the htonl(), ntohl() macros and their compadres
to send/receive binary integer data. Binary byte arrays can be passed
around without conversion.

Generally, I dislike those macros. First, 'cause macros are bad. :)
But more because there are some usage problems. On a little-endian
machine, they flip bytes. If one calls something twice, you undo the fix
and break it again. Also, you have to play magic anyway to get the
initial value.

I prefer clean code that just uses bit math to shift things. No macros
needed, and works on any platform regardless of primitive size. Since
you have to play a little code to get the size issue compensated for,
you can easily fix byte order at the same time.


Strings can be passed around using a little trick -- Java characters take
up 16 bits (the Unicode stuff) while, normally the C side uses char[]s
to store their strings. (wide characters are a different cup of tea).
This is how Java Strings can be written to the socket:

void write(String s) {
byte[] buf= s.getBytes("ISO-8859-1");
write(buf);
}

That's probably not good. Mainly because Latin1 (AKA ISO-8859-1) will
loose lots of characters. On a MS Windows platform you will lose 32
places in the visible character range. On a Mac, you'll use 50%.


It's better to use UTF-8 for your wire format.

String s= new String(buf, "ISO-8859-1");

Here too.

Double values are easy to send/receive too; simple reverse the byte
order on the C side before sending and after receiving and simply
read/write the double on the Java side.

Not quite.

Do that on my Mac G3 and it will be scrambled. :)

Seriously, though, this is a little tricky. So I think it's a good idea
to add some asserts in the C code.
Assert that sizeof(float) == 32 and that sizeof(double) == 64. Then you
might want to add a simple one to be sure that your platform uses
IEE754. If any of these are not true, then you'll need some platform
specific C code in that case (of course, since IEEE754 became standard
and widespread, that's much easier)

And you want to be sure not to just "reverse" the bytes. Instead you
want to "send them in network byte order". IIRC, this is one of the few
places where I cast a pointer and use htnl.
 
T

Tukla Ratte

Why not just pass the bytes? Why complicate your life and increase the
demand on bandwidth?

Yes, you're right.

I looked over my notes this weekend. The Base64 idea was originally
meant to work around endianess issues. For some reason, at that time,
I'd gotten the meaning of "endianess" messed up and thought it meant
how the bits are arranged within a byte, not how the bytes are
arranged in multi-byte value. So that has been nixed.

I was looking into the XML for a way to pass "header" data (filename,
file length, various attributes) along with the binary data itself.
Of course, I don't *need* XML for that; I just thought it might be
more "elegant" and possibly more robust.

My project is a client-server image database. I'm writing it
primarily as a learning experience, and I'm not planning to distribute
it, so it will probably include features that would be bloat in a
"real" project.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top