Dazed and Confused

B

Bill Zimmerman

I am designing a data acquisition, processing and display system that will
use several processing nodes (linux pc) to distribute the load, and are
considering using Java. The system will run stand-alone (not on the www),
will have lots of scientific plots (x-y, color contour, waterfall) and
performance is important. I have a rudimentary knowledge of Java as a
programming language, and am aware of concepts such as EJB and JMS, but I am
overwhelmed with the options available to me (and my team of 4 s/w
engineers). My thoughts are to use Jbuilder and JClass as my development
environment, but before I hurtle head-first into a design and development
effort, I am wondering the following:
* What IDE, (IDEA, JBuilder, Eclipse) if any is appropriate for this type of
system?
* Should I be concerned about MVC and other Patterns?
* What about Struts?
* What is the best way to integrate TCP/IP into Java?
* Are there any good references to using Java for stand-alone apps?
* What Java technologies should I be looking at?

I realize that I am asking a lot. I am looking for guidance as to what and
where to look for information, not necessarily answers. Thanks.
 
T

Tim Ward

Bill Zimmerman said:
I am designing a data acquisition, processing and display system that will
use several processing nodes (linux pc) to distribute the load, and are
considering using Java. The system will run stand-alone (not on the www),
will have lots of scientific plots (x-y, color contour, waterfall) and
performance is important. I have a rudimentary knowledge of Java as a
programming language, and am aware of concepts such as EJB and JMS, but I am
overwhelmed with the options available to me (and my team of 4 s/w
engineers). My thoughts are to use Jbuilder and JClass as my development
environment, but before I hurtle head-first into a design and development
effort, I am wondering the following:

Given that
performance is important.

you might want to write in C++.

Given that
The system will run stand-alone (not on the www),

you can probably avoid all the NAT-box horrors associated with trying to run
it over a WAN and could consider CORBA for the knitting.
 
S

Sudsy

Bill Zimmerman wrote:
* What IDE, (IDEA, JBuilder, Eclipse) if any is appropriate for this type of
system?

Your replies are going to be all over the map on this issue.
I'm quite happy with Eclipse while others will swear by NetBeans,
JBuilder, etc.
* Should I be concerned about MVC and other Patterns?
* What about Struts?

Probably not necessary for your application.
* What is the best way to integrate TCP/IP into Java?

It's already integrated. Check out the Socket and ServerSocket
classes.
* Are there any good references to using Java for stand-alone apps?

Not that I'm aware of. Writing Java apps should be standard fodder
for any text on the language.
* What Java technologies should I be looking at?

Just use the built-in capabilities. Given your stated environment,
I'd probably just go with exchanging serialized objects over sockets.
I wrote a daemon in Java which literally ran for years. No hiccups,
no problems whatsoever.
 
T

Thomas Weidenfeller

Bill said:
My thoughts are to use Jbuilder and JClass as my development
environment,

I think you have it backwards. Why do you think about an IDE and tools,
while you haven't made a decision on the technologies to use? The later
of course might require some learning about the technologies, and maybe
even some prototyping work.

E.g. we here can't decide for you which of the communication
technologies supported by Java are suitable for the communication
between your nodes. Because we don't know the frequency and volume of
your data. We also don't know the number of operations you want or need
to carry out remotely on your data. We don't know your network. We don't
know additional requirements e.g. deployment issues.

Line up your requirements, learn about the technology alternatives, make
technology decisions, and then decide on a development environment.
* What IDE, (IDEA, JBuilder, Eclipse) if any is appropriate for this type of
system?

Undecidable, see above.
* Should I be concerned about MVC and other Patterns?

Undecidable without knowing a lot of details about your system. However,
the question about design patterns makes me suspicious. Design patterns
are not a means by itself. First you identify a problem, then you look
for a design pattern which might fix it. Of course, at this point it is
required that you know your patterns. If your question is "Should I
learn patterns?" then the answer is a definitive YES. If your question
is "Do I need a library with design pattern implementations?" then the
answer is NO.

Applying design patterns left right and center, just because you can is
a sure receipt for ruining your application.
* What about Struts?

A web technology. Didn't you say you want a stand-alone application? You
really should do your homework. Why don't you just google for such
terminology?
* What is the best way to integrate TCP/IP into Java?

Build-in. Didn't you say you have some knowledge of Java? There are
higher-level communication mechanisms in Java, too. But without knowing
your requirements ...
* Are there any good references to using Java for stand-alone apps?

Many have done such applications. And it is not different from
developing such applications in any other language. But may I really
suggest you get a good text book about Java to at least learn the basics?
* What Java technologies should I be looking at?

See above, undecidable without knowing much more of your requirements.

/Thomas

BTW: Someone suggested in this thread to use C++ for performance
reasons. I don't agree. If you have lots of computation ongoing, Java is
on par with C++.
 
B

Bill Zimmerman

Thomas Weidenfeller said:
E.g. we here can't decide for you which of the communication
technologies supported by Java are suitable for the communication
between your nodes. Because we don't know the frequency and volume of
your data. We also don't know the number of operations you want or need
to carry out remotely on your data. We don't know your network. We don't
know additional requirements e.g. deployment issues.
My objective in this post is to get some guidance as to whether Java is the
appropriate language/environment to develop our processing system. As this
is a Java forum, I was not going to go into that, but, we are also
considering:

* C/C++ with X-windows/Motif - We are very familiar with this
approach and realize that the tools for plotting data are dated (no updates
for the last few years), buggy, frustrating to work with, and IMHO likely to
be discontinued at a moments notice
* LabView - Nicely integrated, limited when trying to add features
not originally built-in, limitations in a distributed system
* Java - Little corporate knowledge (our own co.), flexible, many
features (maybe too many). Use of Java will add a skillset to our staff
that has possible future uses.

We will be receiving more than 300 channels of 24-bit data (~18 MB/sec
aggregate rate) comming in a single ATM/SONET spigot. The data can be
blocked in ensembles of at least 0.5 MB. We are considering UDP, TCP, JMS,
Corba, and NDDS for internode comms (Gbit ethernet), but we have doubts as
to whether anything besides UDP will keep up.

We are estimating that we will need between 6-10 dual cpu servers to perform
the tasks (1 for I/O, 1 for GUI and Display, and 4-8 for processing). We
can't be too specific as to the # of GFLOPS needed as the requirements doc
has not been nailed down yet. All of the data must be distributed to
multiple processing nodes, and then collected for display.
Latency is not a big issue, but data skipping is frowned upon.
Line up your requirements, learn about the technology alternatives, make
technology decisions, and then decide on a development environment.
I understand some of the technologies (I have used UDP and TCP, and have
talked to vendors about Corba and NDDS), but really do not have any
experience with most of the Java specific technologies. I know that a
little knowledge is a dangerous thing, but when I start searching web sites,
I am overwhelmed with the alphabet soup (EJB, JSP, XSLT, XMLC, AWT, MVC,
JDBC, CMP, BMP, etc.). I am not looking for definitions, or a solution to
my design problem, but I am hoping to get guidance as to which technologies
are worth looking at.
 
S

Sudsy

Bill Zimmerman wrote:
My objective in this post is to get some guidance as to whether Java is the
appropriate language/environment to develop our processing system.

Fair enough.

I am overwhelmed with the alphabet soup (EJB, JSP, XSLT, XMLC, AWT, MVC,
JDBC, CMP, BMP, etc.). I am not looking for definitions, or a solution to
my design problem, but I am hoping to get guidance as to which technologies
are worth looking at.

Out of that entire mix you'd likely only need AWT for the GUI portion of
the application. You can write daemons in Java almost as easily as in C.
Choose a reliable wire protocol (kind of rules out UDP) and go to town.
Everything else you mentioned is unsuitable for your fairly primitive
application. As I mentioned previously, I wrote a daemon in Java which
ran for years without incident. Nothing fancy, but then you don't need
anything fancy either.
 
C

chris

Bill said:
My objective in this post is to get some guidance as to whether Java is
the
appropriate language/environment to develop our processing system. As
this is a Java forum, I was not going to go into that, but, we are also
considering:

* C/C++ with X-windows/Motif - We are very familiar with this
approach and realize that the tools for plotting data are dated (no
updates for the last few years), buggy, frustrating to work with, and IMHO
likely to be discontinued at a moments notice
* LabView - Nicely integrated, limited when trying to add features
not originally built-in, limitations in a distributed system
* Java - Little corporate knowledge (our own co.), flexible, many
features (maybe too many). Use of Java will add a skillset to our staff
that has possible future uses.

We will be receiving more than 300 channels of 24-bit data (~18 MB/sec
aggregate rate) comming in a single ATM/SONET spigot. The data can be
blocked in ensembles of at least 0.5 MB. We are considering UDP, TCP,
JMS, Corba, and NDDS for internode comms (Gbit ethernet), but we have
doubts as to whether anything besides UDP will keep up.

This sounds like a job for NIO Channels - see
<http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/package-summary.html>.
The good news is: swimming in that alphabet soup somewhere is an API that
fits your application like a glove.
We are estimating that we will need between 6-10 dual cpu servers to
perform
the tasks (1 for I/O, 1 for GUI and Display, and 4-8 for processing). We
can't be too specific as to the # of GFLOPS needed as the requirements doc
has not been nailed down yet. All of the data must be distributed to
multiple processing nodes, and then collected for display.
Latency is not a big issue, but data skipping is frowned upon.

A lot of people use Java for visualisation even when the data collection is
done is C/C++. Take ErgoVU for example (www.ergotech.com): in a classic
installation there is low-level code handling devices and stuffing the
readings into a database, while a Java applet/servlet combo running above
the database handles visualisation. I doubt that that precise technology
will scale up to your application, but it shows you what people are doing
with Java in the SCADA world.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,904
Latest member
HealthyVisionsCBDPrice

Latest Threads

Top