Dazed and Confused

Discussion in 'Java' started by Bill Zimmerman, May 4, 2004.

  1. I am designing a data acquisition, processing and display system that will
    use several processing nodes (linux pc) to distribute the load, and are
    considering using Java. The system will run stand-alone (not on the www),
    will have lots of scientific plots (x-y, color contour, waterfall) and
    performance is important. I have a rudimentary knowledge of Java as a
    programming language, and am aware of concepts such as EJB and JMS, but I am
    overwhelmed with the options available to me (and my team of 4 s/w
    engineers). My thoughts are to use Jbuilder and JClass as my development
    environment, but before I hurtle head-first into a design and development
    effort, I am wondering the following:
    * What IDE, (IDEA, JBuilder, Eclipse) if any is appropriate for this type of
    system?
    * Should I be concerned about MVC and other Patterns?
    * What about Struts?
    * What is the best way to integrate TCP/IP into Java?
    * Are there any good references to using Java for stand-alone apps?
    * What Java technologies should I be looking at?

    I realize that I am asking a lot. I am looking for guidance as to what and
    where to look for information, not necessarily answers. Thanks.
     
    Bill Zimmerman, May 4, 2004
    #1
    1. Advertising

  2. Bill Zimmerman

    Tim Ward Guest

    "Bill Zimmerman" <> wrote in message
    news:dxMlc.4628$...
    > I am designing a data acquisition, processing and display system that will
    > use several processing nodes (linux pc) to distribute the load, and are
    > considering using Java. The system will run stand-alone (not on the www),
    > will have lots of scientific plots (x-y, color contour, waterfall) and
    > performance is important. I have a rudimentary knowledge of Java as a
    > programming language, and am aware of concepts such as EJB and JMS, but I

    am
    > overwhelmed with the options available to me (and my team of 4 s/w
    > engineers). My thoughts are to use Jbuilder and JClass as my development
    > environment, but before I hurtle head-first into a design and development
    > effort, I am wondering the following:


    Given that

    > performance is important.


    you might want to write in C++.

    Given that

    > The system will run stand-alone (not on the www),


    you can probably avoid all the NAT-box horrors associated with trying to run
    it over a WAN and could consider CORBA for the knitting.

    --
    Tim Ward
    Brett Ward Limited - www.brettward.co.uk
     
    Tim Ward, May 4, 2004
    #2
    1. Advertising

  3. Bill Zimmerman

    Sudsy Guest

    Bill Zimmerman wrote:
    <snip>
    > * What IDE, (IDEA, JBuilder, Eclipse) if any is appropriate for this type of
    > system?


    Your replies are going to be all over the map on this issue.
    I'm quite happy with Eclipse while others will swear by NetBeans,
    JBuilder, etc.

    > * Should I be concerned about MVC and other Patterns?
    > * What about Struts?


    Probably not necessary for your application.

    > * What is the best way to integrate TCP/IP into Java?


    It's already integrated. Check out the Socket and ServerSocket
    classes.

    > * Are there any good references to using Java for stand-alone apps?


    Not that I'm aware of. Writing Java apps should be standard fodder
    for any text on the language.

    > * What Java technologies should I be looking at?


    Just use the built-in capabilities. Given your stated environment,
    I'd probably just go with exchanging serialized objects over sockets.
    I wrote a daemon in Java which literally ran for years. No hiccups,
    no problems whatsoever.
     
    Sudsy, May 4, 2004
    #3
  4. Bill Zimmerman wrote:
    > My thoughts are to use Jbuilder and JClass as my development
    > environment,


    I think you have it backwards. Why do you think about an IDE and tools,
    while you haven't made a decision on the technologies to use? The later
    of course might require some learning about the technologies, and maybe
    even some prototyping work.

    E.g. we here can't decide for you which of the communication
    technologies supported by Java are suitable for the communication
    between your nodes. Because we don't know the frequency and volume of
    your data. We also don't know the number of operations you want or need
    to carry out remotely on your data. We don't know your network. We don't
    know additional requirements e.g. deployment issues.

    Line up your requirements, learn about the technology alternatives, make
    technology decisions, and then decide on a development environment.

    > * What IDE, (IDEA, JBuilder, Eclipse) if any is appropriate for this type of
    > system?


    Undecidable, see above.

    > * Should I be concerned about MVC and other Patterns?


    Undecidable without knowing a lot of details about your system. However,
    the question about design patterns makes me suspicious. Design patterns
    are not a means by itself. First you identify a problem, then you look
    for a design pattern which might fix it. Of course, at this point it is
    required that you know your patterns. If your question is "Should I
    learn patterns?" then the answer is a definitive YES. If your question
    is "Do I need a library with design pattern implementations?" then the
    answer is NO.

    Applying design patterns left right and center, just because you can is
    a sure receipt for ruining your application.

    > * What about Struts?


    A web technology. Didn't you say you want a stand-alone application? You
    really should do your homework. Why don't you just google for such
    terminology?

    > * What is the best way to integrate TCP/IP into Java?


    Build-in. Didn't you say you have some knowledge of Java? There are
    higher-level communication mechanisms in Java, too. But without knowing
    your requirements ...

    > * Are there any good references to using Java for stand-alone apps?


    Many have done such applications. And it is not different from
    developing such applications in any other language. But may I really
    suggest you get a good text book about Java to at least learn the basics?

    > * What Java technologies should I be looking at?


    See above, undecidable without knowing much more of your requirements.

    /Thomas

    BTW: Someone suggested in this thread to use C++ for performance
    reasons. I don't agree. If you have lots of computation ongoing, Java is
    on par with C++.
     
    Thomas Weidenfeller, May 4, 2004
    #4
  5. "Thomas Weidenfeller" <> wrote in message >
    > E.g. we here can't decide for you which of the communication
    > technologies supported by Java are suitable for the communication
    > between your nodes. Because we don't know the frequency and volume of
    > your data. We also don't know the number of operations you want or need
    > to carry out remotely on your data. We don't know your network. We don't
    > know additional requirements e.g. deployment issues.

    My objective in this post is to get some guidance as to whether Java is the
    appropriate language/environment to develop our processing system. As this
    is a Java forum, I was not going to go into that, but, we are also
    considering:

    * C/C++ with X-windows/Motif - We are very familiar with this
    approach and realize that the tools for plotting data are dated (no updates
    for the last few years), buggy, frustrating to work with, and IMHO likely to
    be discontinued at a moments notice
    * LabView - Nicely integrated, limited when trying to add features
    not originally built-in, limitations in a distributed system
    * Java - Little corporate knowledge (our own co.), flexible, many
    features (maybe too many). Use of Java will add a skillset to our staff
    that has possible future uses.

    We will be receiving more than 300 channels of 24-bit data (~18 MB/sec
    aggregate rate) comming in a single ATM/SONET spigot. The data can be
    blocked in ensembles of at least 0.5 MB. We are considering UDP, TCP, JMS,
    Corba, and NDDS for internode comms (Gbit ethernet), but we have doubts as
    to whether anything besides UDP will keep up.

    We are estimating that we will need between 6-10 dual cpu servers to perform
    the tasks (1 for I/O, 1 for GUI and Display, and 4-8 for processing). We
    can't be too specific as to the # of GFLOPS needed as the requirements doc
    has not been nailed down yet. All of the data must be distributed to
    multiple processing nodes, and then collected for display.
    Latency is not a big issue, but data skipping is frowned upon.
    >
    > Line up your requirements, learn about the technology alternatives, make
    > technology decisions, and then decide on a development environment.
    >

    I understand some of the technologies (I have used UDP and TCP, and have
    talked to vendors about Corba and NDDS), but really do not have any
    experience with most of the Java specific technologies. I know that a
    little knowledge is a dangerous thing, but when I start searching web sites,
    I am overwhelmed with the alphabet soup (EJB, JSP, XSLT, XMLC, AWT, MVC,
    JDBC, CMP, BMP, etc.). I am not looking for definitions, or a solution to
    my design problem, but I am hoping to get guidance as to which technologies
    are worth looking at.
    >
    > /Thomas
    >
    > BTW: Someone suggested in this thread to use C++ for performance
    > reasons. I don't agree. If you have lots of computation ongoing, Java is
    > on par with C++.
     
    Bill Zimmerman, May 4, 2004
    #5
  6. Bill Zimmerman

    Sudsy Guest

    Bill Zimmerman wrote:
    <snip>
    > My objective in this post is to get some guidance as to whether Java is the
    > appropriate language/environment to develop our processing system.


    Fair enough.

    <snip>
    > I am overwhelmed with the alphabet soup (EJB, JSP, XSLT, XMLC, AWT, MVC,
    > JDBC, CMP, BMP, etc.). I am not looking for definitions, or a solution to
    > my design problem, but I am hoping to get guidance as to which technologies
    > are worth looking at.


    Out of that entire mix you'd likely only need AWT for the GUI portion of
    the application. You can write daemons in Java almost as easily as in C.
    Choose a reliable wire protocol (kind of rules out UDP) and go to town.
    Everything else you mentioned is unsuitable for your fairly primitive
    application. As I mentioned previously, I wrote a daemon in Java which
    ran for years without incident. Nothing fancy, but then you don't need
    anything fancy either.
     
    Sudsy, May 4, 2004
    #6
  7. Bill Zimmerman

    chris Guest

    Bill Zimmerman wrote:

    >
    > "Thomas Weidenfeller" <> wrote in message >
    >> E.g. we here can't decide for you which of the communication
    >> technologies supported by Java are suitable for the communication
    >> between your nodes. Because we don't know the frequency and volume of
    >> your data. We also don't know the number of operations you want or need
    >> to carry out remotely on your data. We don't know your network. We don't
    >> know additional requirements e.g. deployment issues.

    > My objective in this post is to get some guidance as to whether Java is
    > the
    > appropriate language/environment to develop our processing system. As
    > this is a Java forum, I was not going to go into that, but, we are also
    > considering:
    >
    > * C/C++ with X-windows/Motif - We are very familiar with this
    > approach and realize that the tools for plotting data are dated (no
    > updates for the last few years), buggy, frustrating to work with, and IMHO
    > likely to be discontinued at a moments notice
    > * LabView - Nicely integrated, limited when trying to add features
    > not originally built-in, limitations in a distributed system
    > * Java - Little corporate knowledge (our own co.), flexible, many
    > features (maybe too many). Use of Java will add a skillset to our staff
    > that has possible future uses.
    >
    > We will be receiving more than 300 channels of 24-bit data (~18 MB/sec
    > aggregate rate) comming in a single ATM/SONET spigot. The data can be
    > blocked in ensembles of at least 0.5 MB. We are considering UDP, TCP,
    > JMS, Corba, and NDDS for internode comms (Gbit ethernet), but we have
    > doubts as to whether anything besides UDP will keep up.


    This sounds like a job for NIO Channels - see
    <http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/package-summary.html>.
    The good news is: swimming in that alphabet soup somewhere is an API that
    fits your application like a glove.

    > We are estimating that we will need between 6-10 dual cpu servers to
    > perform
    > the tasks (1 for I/O, 1 for GUI and Display, and 4-8 for processing). We
    > can't be too specific as to the # of GFLOPS needed as the requirements doc
    > has not been nailed down yet. All of the data must be distributed to
    > multiple processing nodes, and then collected for display.
    > Latency is not a big issue, but data skipping is frowned upon.


    A lot of people use Java for visualisation even when the data collection is
    done is C/C++. Take ErgoVU for example (www.ergotech.com): in a classic
    installation there is low-level code handling devices and stuffing the
    readings into a database, while a Java applet/servlet combo running above
    the database handles visualisation. I doubt that that precise technology
    will scale up to your application, but it shows you what people are doing
    with Java in the SCADA world.



    --
    Chris Gray
    /k/ Embedded Java Solutions
     
    chris, May 5, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. DJ Miller

    Confused and bewildered.

    DJ Miller, Jul 31, 2003, in forum: ASP .Net
    Replies:
    4
    Views:
    1,083
    DJ Miller
    Sep 4, 2003
  2. John
    Replies:
    0
    Views:
    277
  3. Larry Tate

    getting confused and going crazy

    Larry Tate, Dec 21, 2003, in forum: ASP .Net
    Replies:
    3
    Views:
    502
    Larry Tate
    Dec 21, 2003
  4. Bob H
    Replies:
    4
    Views:
    5,775
    Patrice Scribe
    Mar 3, 2004
  5. arnuld
    Replies:
    19
    Views:
    694
    Jacek Dziedzic
    Mar 30, 2007
Loading...

Share This Page