Directory Structure for java

[Dan Smithers]

Hi,

I have inherited a Java project and would like to introduce a bit more
structure. I am C++ programmer and would like to know what is considered
good practice for Java.

It is compiled on Windows using a batch file that calls

javac -O -g -deprecation *.java

This litters the directories with .class files that are then converted
into a jar with another batch file that contains

zip-r %1.jar . -i \*.class\* \*.gif \*.wav \*jpg \*.png

Currently there is a directory structure

project - contains .java files
com
astra
control - contains .java files
gui - contains .java files
utils - contains .java files
icons - contains .png files
JPEGS - contains .jpg file

All the sub-packages are compiled into one jar.

Is it a good idea to zip sub-packages as separate jars and then link
them together? There are other projects that could use these packages
and it would simplify maintenance.

I would also like to add a build directory to reduce clutter in the
source directory, but when I try it I get a null pointer exception
accessing the icons directories.

I am compiling this as

javac -O -g -deprecation -d build *.java

and running from the project directory using

java -cp build VTASystem

thanks for your help

dan

 

[Robert Klemme]

Hi,

I have inherited a Java project and would like to introduce a bit more
structure. I am C++ programmer and would like to know what is considered
good practice for Java.

It is compiled on Windows using a batch file that calls

javac -O -g -deprecation *.java

This litters the directories with .class files that are then converted
into a jar with another batch file that contains

zip-r %1.jar . -i \*.class\* \*.gif \*.wav \*jpg \*.png

Currently there is a directory structure

project - contains .java files
com
astra
control - contains .java files
gui - contains .java files
utils - contains .java files
icons - contains .png files
JPEGS - contains .jpg file

All the sub-packages are compiled into one jar.

Is it a good idea to zip sub-packages as separate jars and then link
them together? There are other projects that could use these packages
and it would simplify maintenance.

If these are self contained components you could do that. This totally
depends on the contents of those directories. If the code only works as
a whole then there is no point in splitting this up package wise.

I would also like to add a build directory to reduce clutter in the
source directory, but when I try it I get a null pointer exception
accessing the icons directories.

I am compiling this as

javac -O -g -deprecation -d build *.java

and running from the project directory using

java -cp build VTASystem

thanks for your help

The first thing I'd probably do would be to replace the batch based
build with an ant based build - at least for the Java part. With ant
you can fairly easy do all those tasks like compilation, copying of
resources (image files and the like), building jars etc. The nice
benefit is that via ant's task and dependency system you can rebuild
only those parts which must be rebuild to incorporate changes since the
last build.

http://ant.apache.org/

Kind regards

robert

 

[Dan Smithers]

The first thing I'd probably do would be to replace the batch based
build with an ant based build - at least for the Java part. With ant
you can fairly easy do all those tasks like compilation, copying of
resources (image files and the like), building jars etc. The nice
benefit is that via ant's task and dependency system you can rebuild
only those parts which must be rebuild to incorporate changes since the
last build.

http://ant.apache.org/

Thanks, I'll look at ant. I had been thinking of moving to make (and
once that was working automake).

cheers

dan

 

[Mike Schilling]

The first thing I'd probably do would be to replace the batch based
build with an ant based build - at least for the Java part. With
ant
you can fairly easy do all those tasks like compilation, copying of
resources (image files and the like), building jars etc. The nice
benefit is that via ant's task and dependency system you can rebuild
only those parts which must be rebuild to incorporate changes since
the last build.

Most of the time. If the OP is a C++ programmer, used to make's
ability to recompile precisely the files that are stale or that
include a header that's state, he should be warned that ant doesn't
always catch the "needs to be recomnpiled beacuse it depends on a
class that has changed" part of that.

 

[Tom Anderson]

I have inherited a Java project and would like to introduce a bit more
structure. I am C++ programmer and would like to know what is considered
good practice for Java.

Using ant or maven (which is ant plus some additional conventions) is
widely considered best practice. For small projects, it may be more
trouble than it's worth (and even for large projects, i've had maven in
particular cause real pain), but it's worth considering.

Currently there is a directory structure

project - contains .java files
com
astra
control - contains .java files
gui - contains .java files
utils - contains .java files
icons - contains .png files
JPEGS - contains .jpg file

All the sub-packages are compiled into one jar.

Is it a good idea to zip sub-packages as separate jars and then link
them together? There are other projects that could use these packages
and it would simplify maintenance.

That's the plus side. The minus is that it complicates the app - rather
than having one JAR, you have several. Mind you, if your app uses
third-party libraries, those have to be supplied anyway, so this may not
be such a big deal.

You could always build two targets, an all-in-one-JAR app, and the
reusable parts as separate JARs.

I would also like to add a build directory to reduce clutter in the
source directory, but when I try it I get a null pointer exception
accessing the icons directories.

I am compiling this as

javac -O -g -deprecation -d build *.java

and running from the project directory using

java -cp build VTASystem

If the images are being loaded as resources (through the
Class.getResource* methods), then they need to be on the classpath too
(bit weird, but that's how it goes). Do:

java -cp build:icons VTASystem

Assuming that the resource references are written relative to the project
root (/icons/foo.png, /icons/JPEGS/bar.jpg, etc), and that you're running
that command from the project root.

A convention for resources is that they should be organised in a directory
structure that mirrors that of the class files, so resources for your app
might be named /net/foo/vta/icons/foo.png etc. Note that this means that
your classes can then refer to resources via relative names, for instance
a class in net.foo.vta could ask for icons/foo.png (with no leading /) and
get the right thing.

You can then set up a directory hierarchy for resources parallel to the
one for source code, and as part of the build, copy all the resources into
the build directory before jarring that up or running from it. Something
like:

/
src/
net/
foo/
vta/
VTASystem.jar
lib/
lib1.jar
lib2.jar
resource/
net/
foo/
vta/
icons/
foo.png

Then:

rm -rf build
mkdir build
javac -classpath "lib/*" -d build $SOURCE_FILES
cp -r resource/ build
java -classpath "build:lib/*" net.foo.vta.VTASystem

Maven does this automatically, including converting character encodings of
text resources and things like that if needed.

If you're just running, this isn't really easier than using a separate
classpath entry that points to the resource root, but it's convenient when
you make a JAR.

tom

 

[Roedy Green]

I would also like to add a build directory to reduce clutter in the
source directory, but when I try it I get a null pointer exception
accessing the icons directories.

The directory structure in Java is much more important than in other
languages. It MUST reflect the package naming structure. You can't
just go moving source or class files around without changing the
source code to match.

see http://mindprod.com/jgloss/package.html

Generally you want to end up with one big jar file. If you have more
than one jar, you add the complication of linking the master jar to
the sub jars.
see http://mindprod.com/jgloss/jar.html#CLASSPATH

Java Web Start has ability to load multiple jars for you without much
fuss. see http://mindprod.com/jgloss/javawebstart.html
--
Roedy Green Canadian Mind Products
http://mindprod.com

There is a certain cosmic justice in the bankruptcy of General Motors. People at every level of the economic spectrum who for
economic motives helped GM pummel the environment with deliberately inefficient automobiles are being punished both severely and economically.

 

[Robert Klemme]

Most of the time. If the OP is a C++ programmer, used to make's
ability to recompile precisely the files that are stale or that
include a header that's state, he should be warned that ant doesn't
always catch the "needs to be recomnpiled beacuse it depends on a
class that has changed" part of that.

Can you give an example of this situation? I haven't observed this so
far but maybe that's just ignorance. Thank you!

Kind regards

robert

 

[Mike Schilling]

Can you give an example of this situation? I haven't observed this so
far but maybe that's just ignorance. Thank you!

It's very simple; ant recompiles .java files that have changed, period. So:
(please forgive any typos)

A.java:
public class A
{
B b = new B();
}

B.java
public class B
{
}

Ant builds this with no problem. Now I decide that B really should be
created by a factory, so I change it to

B.java
public class B
{
private B()
{
}

public static B create()
{
return new B();
}
}

I run ant again. It recompiles B.java, but not A.java, so the compilation
succeeds. Not until runtime do I see any errors resulting from the fact
that A is trying to call a private constructor.

The usual way around this is to create two ant targets: "compile" compiles
whatever has changed and "clean" deletes all .class and .jar files. Now if
either,

1. You know you've changed or deleted things that should force other classes
to recompile, or
2. You're seeing runtime errors that look like this kind of inconsistency,

simply type "ant clean compile" instead of "ant compile".

The reason for this is that Java lacks the ".h" files that C and C++ use to
import definitions. Instead, the Java compiler gets the defintions of
imported classes from their .class files (or jar entries.) In this case, A
imports definitions from B. But a simple date check of A.class against
B.class is insufficient; some code might have been added to B, or perhaps
methods were added that A.doesn't use. In either case there's no need to
rebuild A. In fact, most of the possible changes to B won't require
recompiling A. So ant, in in fact almost all Java build systems including
the various IDEs, assume that only changed .java files need to be
recompiled, and give the user an option of "recompile everything" in the
cases where that's not true.

 

[Mark Space]

The reason for this is that Java lacks the ".h" files that C and C++ use to
import definitions. Instead, the Java compiler gets the defintions of
imported classes from their .class files (or jar entries.) In this case, A

Nope, it's not the .h files. You can still have dependency problems no
matter what you do with .h files. The real fix is to have the compiler
scan the source files and emit a list of dependency lines for each
source file.

Normally for C programs this goes in the make file like this:

depend:
gcc -E -MM *.c > .depend

which allows you to regenerate the dependency information when you need
to. I'm very surprised javac has no such equivalent feature.

 

[Mark Space]

I'm very surprised javac has no such equivalent feature.

I did however find a tool that purports to list all of the dependencies
for a Java project, among other useful things.

<http://depfind.sourceforge.net/>

 

[Mike Schilling]

Nope, it's not the .h files. You can still have dependency problems
no matter what you do with .h files. The real fix is to have the
compiler scan the source files and emit a list of dependency lines
for each source file.

Yes. The point is that the dependencies in C or C++ can be captured a
simple analysis of source files. That's not true in Java.

 

[Joshua Cranmer]

I'm very surprised javac has no such equivalent feature.

The nature of C and C++ is that you can't have circular dependencies in
the include files. I tried to write my own such statement for a
make-based system and ran into the problem that A can depend on B and B
can depend on A at the same time--any DAG-based make system would fall
flat at such a representation.

 

[Andy Dingley]

Yes.  The point is that the dependencies in C or C++ can be captured a
simple analysis of source files.  That's not true in Java.

Although it is a good start, it's also the full list for many Java
programs (esp. those written by non-Java people) and it's also fairly
easy to determine whether the Java could have dependencies beyond
those that are "obvious" in this way.

 

[Andy Dingley]

I have inherited a Java project and would like to introduce a bit more
structure. I am C++ programmer

Half-decent Java does (IMHE) have considerably better structure than C+
+, produced by a team of comparable experience.

It is compiled on Windows using a batch file

OK, so that sucks. 8-(

Your "build tools" for Java are Ant, Maven, Ivy & Hudson. Forget
CruiseControl (if you're still using it) and switch to Hudson instead.
Your goal is not only to make "build" work, but also to get continuous
integration / continuous testing working as well. This is going to
need a decent source control system, so throw Subversion in there too
and JIRA to track bugs (Bugzilla, like CruiseControl, has been
overtaken). Mercurial might be a better bet than Subversion too (I
admit I'm still weak on that one myself).

The IDE is Eclipse, or else a damn good reason why not. There are such
reasons, but they're rare (and if you think Netbeans had one, that's
surely dead in the water now)

You don't need make. For the trivial little that make offers, just
javac alone can do that much for you.

Step #1 is probably to get your head round Ant. Ant does damn-all, but
does it pretty well. It bites you badly if you write it sloppily, or
if you fail to grasp how limited it is and you try to do things that
it "ought" to do easily. Basic rules for Ant. 1. It won't do that. 2.
If it _can_ do that, you won't believe how awkward and verbose it is
to do that. 3. If someone who does know how to work Ant neatly sets
up your buildfile to do it, some careless muppet with a hammer will
break it shortly afterwards.

So Ant sucks, but it seems to be an essential learning experience. Try
the Steve Loughran book. What Ant does is to manage dependencies
between "targets" which are little scripts of Ant's own "tasks". There
are tasks for making directories, cleaning out old directories,
compiling with javac, making jars and making zip files. You can
probably work out the rest, and how the "depends" list attribute on
each target ties it all together. It's working a step above make: at
the dependencies between these task groupings, not the dependencies
between individual class files. Javac is pretty speedy - we just blast
the whole class tree and re-create it.

One obvious thing you'll discover about Java is how many 3rd party jar
libraries are involved. Now managing these is a right old game and Ivy
can help you out. Again it's declarative metadata attached to each
target, and an engine that knows how to resolve them, Rather than the
usual "bucket" of the lib/ directory, Ivy sets up a local repository
that lists all the components (and a little ivy.xml for each), points
to a remote source for each of them, and then an ivy.xml in your own
project components defines the rest. From your Ant buildfile, the Ivy
task will cache a local copy of the library jars that you need,
including their transitive dependencies. A bit of tweaking and my Ivy
rig even reports on my open source licences for me and does the
redistribution docs.

Maven does both of these things. But not, IMHO, as well as the
combination. It's probably still more popular than Ivy though.

 

[Andy Dingley]

Step #2 is automated unit testing. Each "project component" gets
itself two mirrored source trees called src/ and test/src/ (or
whatever). One is populated with unit tests for the other: simple
little test snippets built on jUnit and other test frameworks. There
are gadgets to mock out database access etc. and any of the stuff
that's otherwise hard to set up test-wise. This is a huge field it has
to be said, with lots of tools in it.

When you run your Ant build, after you've compiled it and before you
package it into that deliverable zip, you run the unit tests. Not when
you feel like it, but on _every_ build. Everything passes. Every time.
If it doesn't, something automatic inflicts pain on a developer, pour
encourager les autres.

If you change code, you check the unit tests still work. Eclipse does
this for you very, very easily. You check this before you commit the
code into Subversion.

If you're hardcore, you read up on Test Driven Development (TDD) and
you use the whole unit test malarkey to drive your development and
coding cycle. It works too.

 

[Roedy Green]

The first thing I'd probably do would be to replace the batch based

build with an ant based build - at least for the Java part. With ant
you can fairly easy do all those tasks like compilation, copying of
resources (image files and the like), building jars etc. The nice
benefit is that via ant's task and dependency system you can rebuild
only those parts which must be rebuild to incorporate changes since the
last build.

see http://mindprod.com/jgloss/ant.html
for some sample scripts
--
Roedy Green Canadian Mind Products
http://mindprod.com

There is a certain cosmic justice in the bankruptcy of General Motors. People at every level of the economic spectrum who for
economic motives helped GM pummel the environment with deliberately inefficient automobiles are being punished both severely and economically.

 

[Andy Dingley]

Step #2 is automated unit testing.

Big step #3 is to set up continuous integration / continuous unit
testing. Now _THIS_ is where reasonable current best practice in the
Java world pisses all over C++, rubs .NET's nose in it and does a
little victory dance around the embers.

Hudson (or CruiseControl a few years back) sits on a "server" (ie any
old crate that isn't your desktop) and watches what happens in the
Subversion (et al.) repository. When your code has changed, Hudson
pulls out the latest and builds / unit tests it. So within less than
an hour of any change, you all ought to know if it was good or bad. No
"weekly builds" or "overnight test cycles", this slaps errant coders
in the face with the wet haddock of reality while there's still time
to fix it (and buy the team's doughnuts). Which means that it always
is good.

Hudson has a few other tricks too...

 

[Mark Space]

The nature of C and C++ is that you can't have circular dependencies in
the include files. I tried to write my own such statement for a
make-based system and ran into the problem that A can depend on B and B
can depend on A at the same time--any DAG-based make system would fall
flat at such a representation.

Ah, I think I understand. Thanks for pointing that out.

 

[Andy Dingley]

When your code has changed, Hudson pulls out the latest and builds / unit tests it.

Step #4 is where it gets clever (like it wasn't before?)

Rather than merely running the unit tests, now make the continuous
test cycle run some system-level acceptance tests too. Then these get
automated into the build, and Hudson schedules them in too. This is
where you realise that Hudson beats CruiseControl, because Hudson can
also:
* Schedule dependencies between build units, i.e. "build & unit test"
is one unit, "acceptance test" is another. Acceptance doesn't fire off
unless build & unit have already passed. This tends to make turn-
around better, especially when you have many products.
* Trivially allocate tasks between farms of Hudson slaves on multiple
servers. Now you can throw hardware at the problem and get sub-hour
acceptance test results too. Which means, in practice, that your
coders don't ever get to break your product (unnoticed) and your build
manager gets their Friday evenings back. Strangely if they don't think
they can get away with it, they stop doing it too.

"Acceptance" starts to get interesting around now. Do you test web
apps with HTTPUnit, or by running Jelly / Watir and poking at a
browser whilst pretending to be human? If you code decent pure-
semantic HTML, then the answer's pretty obvious (and _so_much_ less
coding of tags for test suites!!), but this also redefines the role of
QA. Rather than doing boring testing, and especially rather than
doing boring and routine regression testing, the QA task is now to set
up these tests for the computers to run for us, and also to look at
some overall usability of what the app does and whether this is what
it best could be doing, not just whether it's what we hoped it would
do for us (whether that's a good thing for the user or not).

 

[Lew]

Although it is a good start, it's also the full list for many Java
programs (esp. those written by non-Java people)

That's an oxymoron. Someone writing a Java program perforce is "Java people",
of one experience level or another.

Addressing your main point, that "it" (a simple analysis of source files) is a
good start toward or even yields a "full list" of class dependencies, that is
really not generally true. It is far more common than not for compilation to
depend on JAR files without source on hand, even for "Java people" of limited
experience. Unlike C or C++, Java uses "object" code, actually byte code, to
resolve dependencies in the source code during the compile phase.