fast communication with c++

[H Brett Bolen]

I'd like to send messages from c++ to java. lots of messages.
lots of data.

The structs I want to send are complex structs, with many
fields.

The data flows both ways, but the majority of data is from
c++ to java.

We are currently using software that is vendor dependant --
which takes the struct info, and generates java and c++ classes
that knows how to pack, align, endian-swap messages, and
send the messages over sockets.

Is there a better way to do this?

Most people mention xml or corba, but what we want ot do
is send thousands and thousands of big messages. I don't
want something that adds alot of overhead.

I do not have to go to see on any machine, any language. I
need to go to linux gcc compiler ( or maybe windows gcc
compiler [cygwin] in the future). I do not need a
geneneral solution with flexiblity ( and overhead). Just
point to point, c-to-java struct passing ( say over
sockets).

The structs are predefined ( say 30 structs), and consists
of variable length arrays of sub structures.

I would like something that is fast ( minimal overhead
in marshalling and demarshalling the data).

thanks.

b

 

[Chris Smith]

I'd like to send messages from c++ to java. lots of messages.
lots of data.

The structs I want to send are complex structs, with many
fields.

The data flows both ways, but the majority of data is from
c++ to java.

We are currently using software that is vendor dependant --
which takes the struct info, and generates java and c++ classes
that knows how to pack, align, endian-swap messages, and
send the messages over sockets.

Is there a better way to do this?

DataInputStream and DataOutputStream would be the way to go on the Java
side. If your code generation software is generating code using those
classes, then that's probably as good as you're gonna get. You never
actually said what your problem was with the existing software.

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Eugene Mayevski]

I'd like to send messages from c++ to java. lots of messages.
lots of data.
The structs I want to send are complex structs, with many
fields.
The data flows both ways, but the majority of data is from
c++ to java.

Take a look at MsgConnect (http://www.msgconnect.com/). It lets you
easily send messages (packets). To send complex structures you can use
MCDataTree class, included with MsgConnect.

 

[H Brett Bolen]

DataInputStream and DataOutputStream would be the way to go on the Java
side. If your code generation software is generating code using those
classes, then that's probably as good as you're gonna get. You never
actually said what your problem was with the existing software.

yes but what would read them in to c structs?

The existing solution uses some proprietary software that one of
our customers does not like.

It is a little cluncky, but the gist of it is that messages are
defined in xml ( sort of like idl), and it is run through
a compiler that produces both c structs and java classes that
know how to read itself from a byte array. There is no
xml in the transport of the data.

The data is a set of drawing primitives and screen points. a refresh
could consist of 10,000 points, sections of the screen is updated
in real time.

I figure this is a common problem that should have already been
solved. move large structs from c or c++ to java. handle swapping
packing, alignment. needs to be fast.

b

 

[Chris Smith]

yes but what would read them in to c structs?

I suppose you'd do it using the read system call, or some other
equivalent depending on your choice of I/O architecture. You'd then use
something like ntohl or ntohs if needed, depending on the type of data,
to convert the octet stream to larger numerical values in host byte
order, and fill in the struct fields with those values. If the data is
text and you've use Java's writeUTF, then the task becomes more complex
and depends on what you want; in that case, actually, it may be worth
doing the conversion to an appropriate encoding in Java, where the code
is readily accessible, rather than in C where you'd need third-party
code, and then avoid writeUTF altogether.

The existing solution uses some proprietary software that one of
our customers does not like.

It is a little cluncky, but the gist of it is that messages are
defined in xml ( sort of like idl), and it is run through
a compiler that produces both c structs and java classes that
know how to read itself from a byte array. There is no
xml in the transport of the data.

You still haven't said what your customer doesn't like about it --
except that it's "clunky". If you don't know, then you need to go back
to the customer and find out more about their requirements... do that
want less effort in some part of the task? If so, are they willing to
accept more effort in a different part? Are they unhappy with the
performance? If so, what's the limiting factor? And what level of
performance, numerically speaking, would be acceptable to them? Do they
merely want a better API? If so, would it be acceptable to wrap the
existing API? Are there poor licensing terms on the existing product?

Without that information, we're likely to propose a solution with
exactly the same problems as the one you've got.

I figure this is a common problem that should have already been
solved. move large structs from c or c++ to java. handle swapping
packing, alignment. needs to be fast.

I'm sure it's been solved many times. Code generation from a common
description of the data looks like the obvious solution to me, and it's
the one you're using. So, what do you want to change about your
existing system?

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Phil...]

Instead of sending all the primitive data to the
application, how about generating the image
in the C++ application and sending the image instead?
This is analogous to having a "off screen image" in
a java app. Naively this seems to reduce the
amount of data that passes to the java app.

 

[Eugene Mayevski]

It is a little cluncky, but the gist of it is that messages are
defined in xml ( sort of like idl), and it is run through
a compiler that produces both c structs and java classes that
know how to read itself from a byte array. There is no
xml in the transport of the data.

You can switch to some ASN.1 library. This is an international standard
and libraries used to read/write the usually very fast and the data
sequence is compact (the standard was designed with efficiency in mind,
unlike XML and most proprietary things).

 

[Steve Horsley]

The existing solution uses some proprietary software that one of
our customers does not like.

You still haven't said why. What's the problem you're trying to solve?

Dislike of XML?
Dislike of automated code generation?
Dislike of sending "structures" over the wire?
Dislike of your company, but can't say so outright?
Dislike of mixed language solutions?
Fishing for a discount?

Steve

 

[H Brett Bolen]

You still haven't said why. What's the problem you're trying to solve?

Dislike of XML?
Dislike of automated code generation?
Dislike of sending "structures" over the wire?
Dislike of your company, but can't say so outright?
Dislike of mixed language solutions?
Fishing for a discount?

The proprietary software is part if a bigger package that is redundant
for their system. Maybe like swatting a fly with a sledgehammer. In
addition, there may be political reasons as well.

Our customer would like something open that they can have access to
the source code.

'No' to all of these. I'm looking for an open source solution
to what should be a common problem.

problem?

I want to send various structures from c/c++ to/from java over
sockets. Is there a smallish open source solution that accounts
for the various packing/endian/size structure/class layout issues
between the languages?

Java knows how to serialize (marshal) structures, is there a
c/c++ way to read these structures?

Sending the data using xml or text will be too slow for this
real-time application. Corba is too big, and probably
too slow as well.

 

[Gordon Beaton]

Sending the data using xml or text will be too slow for this
real-time application. Corba is too big, and probably
too slow as well.

Java default serialization is notoriously slow compared to the other
methods that you mention here, and AFAIK there are no C or C++
libraries to deal with serialized Java objects anyway.

However since you seem to want a non-proprietary binary format...

- You can use an IDL compiler to generate stubs to marshal and
unmarshal your data in both C and Java, and then communicate over an
ordinary socket connection. You don't need to use Corba to
communicate just because you use IDL.

- There are ASN.1 tools that could generate PER or BER stubs from your
data descriptions. I have no real clue about ASN.1, so this could be
complete nonsense.

- There are also XDR tools available for both environments. Use Google
to find one you like. See also rfcs 1831 and 1832.

/gordon