How to set up a fast correct java build?

[Arne Vajhøj]

Try maven.

It is like Ant but better

It is OK if you can force your projects to follow maven standards.

Otherwise a nightmare.

In fact you can produce clean builds of your software, as easy as typing
"mvn clean compile" to a commandline.

I don't think the original poster is looking for easy - I think he
is looking for fast.

Arne

 

[Roedy Green]

Java usually compiles so much faster than C/C++ that it is not a
problem that need to be handled.

so long as you don't keep reloading Javac.exe. In the early days of
Java our boss came from a C/C++ and insisted on building our not all
that big project with a sort of C-style make. It took FOREVER. Then
one of the team members wrote a script that loaded javac once and
compiled everything and it was orders of magnitude faster. This is
primarily what gives ANT its fast compile speed.

 

[Arved Sandstrom]

Joshua Maurice wrote:
[ SNIP ]

Speaking of Eclipse, perhaps the way to go might be to just do the
entire build in Eclipse. Write Eclipse build plugins for C++, for our
custom codegen, etc. I'll have to seriously consider that.

I'd say it's worth a try. Leastways rule it out before you start sending
major time on some other approach. I'm currently helping out with some
client J2EE apps of which the largest is perhaps a quarter the size of
the application you describe, and a clean build takes well under a
minute on a decent machine - using Eclipse. We have encountered no
problems deploying to test and prod using the Eclipse builds.

As an aside, perhaps Java Rebel might be able to assist in some respects
as well. I am not connected with the product in any way; I've just used
it some and like it.

AHS

 

[Joshua Maurice]

Well, I just spent a good many hours today whipping this up. I think
it'll work. It's rough though, and it could definitely be cleaned up.

import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.net.URL;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;

import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

import com.sun.source.tree.CompilationUnitTree;
import com.sun.source.tree.IdentifierTree;
import com.sun.source.tree.ImportTree;
import com.sun.source.tree.MemberSelectTree;
import com.sun.source.util.JavacTask;
import com.sun.source.util.TreeScanner;


public class JavaDepends {

private static class ParsedClassFile {
public String sourceFile = null;

public ParsedClassFile(File classfile) throws Exception {
InputStream fin = new FileInputStream(classfile);
try {
DataInputStream in = new DataInputStream(new
BufferedInputStream(fin));
init(in);
} finally {
fin.close();
}
}

private void init(DataInputStream in) throws Exception {
//magic number
byte[] magicNumber = new byte[4];
byte[] expectedMagicNumber = new byte[]
{ (byte)0xCA, (byte)0xFE, (byte)0xBA, (byte)
0xBE };
if (4 != in.read(magicNumber))
throw new Exception("Unexpected end of file");
if ( ! Arrays.equals(magicNumber, expectedMagicNumber))
throw new Exception("Unexpected magic number"
+ " " + magicNumber[0]
+ " " + magicNumber[1]
+ " " + magicNumber[2]
+ " " + magicNumber[3]
);

//minor_version
final short minorVersion = in.readShort();

//major_version
final short majorVersion = in.readShort();

//constant_pool_count
final short constantPoolCount = in.readShort();

//constant_pool
final String cpStringLiterals[] = new String
[constantPoolCount];
final Set<Short> qualifiedClassNameCpIndexes = new
HashSet<Short>();

for (short index = 1; index < constantPoolCount; ++index)
{
final byte tag = in.readByte();
switch (tag)
{
case 1://a literal string
final short byteLen = in.readShort();
final byte[] javaUtf8Str = new byte[byteLen];
in.read(javaUtf8Str);
cpStringLiterals[index] = new String(javaUtf8Str,
"UTF-8");
break;
case 3:
{ final int x = in.readInt();
break;
}
case 4:
{ final float f = in.readFloat();
break;
}
case 5:
{ final long x = in.readLong();
++index;
break;
}
case 6:
{ final double x = in.readDouble();
++index;
break;
}
case 7: //class reference, refers to fully qualified
name
{ final short cpIndex = in.readShort();
qualifiedClassNameCpIndexes.add(cpIndex);
break;
}
case 8: //string object
{ final short cpIndex = in.readShort();
break;
}
case 9: //field
{ final short classCpIndex = in.readShort();
final short nameAndTypeCpIndex = in.readShort
();
break;
}
case 10: //method
{ final short classCpIndex = in.readShort();
final short nameAndTypeCpIndex = in.readShort
();
break;
}
case 11: //interface method
{ final short classCpIndex = in.readShort();
final short nameAndTypeCpIndex = in.readShort
();
break;
}
case 12: //name and type descriptor
{ final short nameCpIndex = in.readShort();
final short typeDescriptorCpIndex =
in.readShort();
break;
}
default: throw new Exception("Unknown tag " + tag + "
at index " + index);
}
}

//access_flags
final short accessFlags = in.readShort();

//this_class
final short thisClass = in.readShort();

//super_class
final short superClass = in.readShort();

//interfaces_count
final short interfacesCount = in.readShort();

//interfaces
final short[] interfaces = new short[interfacesCount];
for (int i=0; i<interfaces.length; ++i)
interfaces = in.readShort();

//fields_count
final short fieldsCount = in.readShort();

//fields
final FieldInfo[] fields = new FieldInfo[fieldsCount];
for (int i=0; i<fields.length; ++i)
fields = readFieldInfo(in);

//methods_count
final short methodsCount = in.readShort();

//methods
final MethodInfo[] methods = new MethodInfo[methodsCount];
for (int i=0; i<methods.length; ++i)
methods = readMethodInfo(in);

//attributes_count
final short attributesCount = in.readShort();

//attributes
final AttributeInfo[] attributes = new AttributeInfo
[attributesCount];
for (int i=0; i<attributes.length; ++i)
attributes = readAttributeInfo(in);

try {
in.readByte();
throw new Exception("Unexpected: not at end of file");
} catch (EOFException e) {
}
// //

for (AttributeInfo attribute : attributes) {
if ("SourceFile".equals(cpStringLiterals
[attribute.attributeNameIndex])) {
if (attribute.attributeLength != 2)
throw new Exception("Malformed class file.
SourceFile should have attribute_length == 2. Current " +
attribute.attributeLength);
short cpIndex = (new DataInputStream(new
ByteArrayInputStream(attribute.info))).readShort();
sourceFile = cpStringLiterals[cpIndex];
break;
}
}
}

private static class FieldInfo {
public short accessFlags;
public short nameIndex;
public short descriptorIndex;
public short attributesCount;
public AttributeInfo[] attributes;
}
private static FieldInfo readFieldInfo(DataInputStream in)
throws Exception {
FieldInfo x = new FieldInfo();
x.accessFlags = in.readShort();
x.nameIndex = in.readShort();
x.descriptorIndex = in.readShort();
x.attributesCount = in.readShort();
x.attributes = new AttributeInfo[x.attributesCount];
for (int i=0; i<x.attributes.length; ++i)
x.attributes = readAttributeInfo(in);
return x;
}

private static class MethodInfo {
public short accessFlags;
public short nameIndex;
public short descriptorIndex;
public short attributesCount;
public AttributeInfo[] attributes;
}
private static MethodInfo readMethodInfo(DataInputStream in)
throws Exception {
MethodInfo x = new MethodInfo();
x.accessFlags = in.readShort();
x.nameIndex = in.readShort();
x.descriptorIndex = in.readShort();
x.attributesCount = in.readShort();
x.attributes = new AttributeInfo[x.attributesCount];
for (int i=0; i<x.attributes.length; ++i)
x.attributes = readAttributeInfo(in);
return x;
}

private static class AttributeInfo {
public short attributeNameIndex;
public int attributeLength;
public byte[] info;
}
private static AttributeInfo readAttributeInfo(DataInputStream
in) throws Exception {
AttributeInfo x = new AttributeInfo();
x.attributeNameIndex = in.readShort();
x.attributeLength = in.readInt();
x.info = new byte[x.attributeLength];
for (int i=0; i<x.info.length; ++i)
x.info = in.readByte();
return x;
}
}

private static class CollectTypesTreeVisitor extends
TreeScanner<Void, Object> {
public String lastImport = null;
public String lastQualifiedClass = null;

public List<String> imports = new ArrayList<String>();
public Set<String> potentialClassIdentifiers = new
TreeSet<String>();

public Void visitIdentifier(IdentifierTree node, Object p) {
if (lastImport != null) {
if (0 != lastImport.length())
lastImport = node.getName() + "." + lastImport;
else
throw new AssertionError();
imports.add(lastImport);
lastImport = null;
} else {
if (null == lastQualifiedClass)
potentialClassIdentifiers.add(node.getName
().toString());
else if (0 == lastQualifiedClass.length())
throw new AssertionError();
else {
potentialClassIdentifiers.add(node.getName
().toString() + "." + lastQualifiedClass);
lastQualifiedClass = null;
}
}
return super.visitIdentifier(node, p);
}

public Void visitImport(ImportTree node, Object p) {
if (null != lastImport)
throw new AssertionError();
lastImport = "";
Void x = super.visitImport(node, p);
if (null != lastImport)
throw new AssertionError();
return x;
}

public Void visitMemberSelect(MemberSelectTree node, Object
p) {
if (null == lastImport) {
if (null == lastQualifiedClass)
lastQualifiedClass = node.getIdentifier().toString
();
else if (0 == lastQualifiedClass.length())
throw new AssertionError();
else
lastQualifiedClass = node.getIdentifier() + "." +
lastQualifiedClass;
}else if (lastImport.equals(""))
lastImport = node.getIdentifier().toString();
else
lastImport = node.getIdentifier() + "." + lastImport;
return super.visitMemberSelect(node, p);
}
}

public static void main(String[] args) throws Exception {
List<String> javaFiles = new ArrayList<String>();
for (String arg : args)
javaFiles.add(arg);

final JavaCompiler compiler =
ToolProvider.getSystemJavaCompiler();
final DiagnosticCollector<JavaFileObject> diagnostics = new
DiagnosticCollector<JavaFileObject>();
final StandardJavaFileManager fileManager =
compiler.getStandardFileManager(diagnostics, null, null);
final Iterable<? extends JavaFileObject> fileObjects =
fileManager.getJavaFileObjectsFromStrings(javaFiles);
final JavaCompiler.CompilationTask task = compiler.getTask
(null, fileManager, diagnostics, null, null, fileObjects);

final JavacTask javacTask = (JavacTask) task;
final Iterable<? extends CompilationUnitTree> ASTs =
javacTask.parse();

final ClassLoader loader = Thread.currentThread
().getContextClassLoader();
for (CompilationUnitTree ast : ASTs) {
CollectTypesTreeVisitor visitor = new
CollectTypesTreeVisitor();
ast.accept(visitor, null);
if (visitor.lastImport != null)
throw new AssertionError();

List<String> imports = new ArrayList<String>();
for (String imp : visitor.imports)
imports.add(imp.replace('.', '/'));

Set<String> resourceFiles = new TreeSet<String>();
ClassIdLoop: for (String id :
visitor.potentialClassIdentifiers) {
while (true) {
//try taking it as is
{
String x = id.replace('.', '/') + ".class";
URL url = loader.getResource(x);
if (null != url) {
resourceFiles.add(url.toString());
continue ClassIdLoop;
}
}

//try adding java.lang.
{
URL url = loader.getResource("java/lang/" + id
+ ".class");
if (null != url) {
resourceFiles.add(url.toString());
continue ClassIdLoop;
}
}

//try imports
for (String imp : imports) {
if ('*' == imp.charAt(imp.length() - 1)) {
String classResourceName = imp.substring
(0, imp.length()-1) + id + ".class";
URL url = loader.getResource
(classResourceName);
if (null != url) {
resourceFiles.add(url.toString());
continue ClassIdLoop;
}
} else {
int index = imp.lastIndexOf('/');
String classNameInImport = imp.substring
(index + 1);
if (classNameInImport.equals(id)) {
//check that it's findable
URL url = loader.getResource(imp +
".class");
if (null != url) {
resourceFiles.add(url.toString());
continue ClassIdLoop;
} else {
System.out.println("Should have
found " + imp + ".class");
}
}
}
}

int index = id.indexOf('.');
if (-1 == index)
break;
id = id.substring(0, index);
}
System.out.println("Unable to find class file for " +
id);
}
System.out.println("-- " + ast.getSourceFile());
for (String resource : resourceFiles) {
if (resource.indexOf("file:/") == 0) {
String file = resource.substring("file:/".length
()).replace("%20", " ");
String sourceFile = (new ParsedClassFile(new File
(file))).sourceFile;
System.out.println("file resource " + file + ",
SourceFile " + sourceFile);
} else if (resource.indexOf("jar:file:/") == 0) {
System.out.println("jar resource " + resource);
} else
throw new Exception("Unknown resource type: " +
resource);
}
}
}
}

 

[Jean-Baptiste Nizet]

Joshua Maurice a écrit :

I'm sorry if this is answered in a FAQ somewhere, but the first
comp.lang.java.programmer FAQ I found referenced Java version 1.1, so
I stopped reading there.

I'm working on a project now with over 20,000 java source files, in
addition to more than 4,000 C++ source files, some forms of custom
code generation, an eclipse build, and probably other things I don't
know offhand.

This is the problem, IMHO.
You should really split this enormous project into smaller ones with
cleanly identified responsibilities and dependencies. For example, if
your project consists of a typical n-tier project, you should identify

- utility classes, without any dependencies except the JDK and external
libraries
- domain object classes, only dependent on utility classes
- data access classes, dependent on the two above sets of classes
- service classes, dependent on the three above sets of classes
- presentation layer classes, dependent on the service layer, domain
objects and utility classes

Add interfaces for the DAO and service classes to decouple all this, and
you have even less dependencies.

This way, you have a set of projects with far less fewer dependencies,
identified dependencies rather than a spaghetti dish, and a much more
easy build.

JB.

 

[Tom Anderson]

My company is currently using Maven. I think we all hate it with a
passion. ~3-4 hour build times without tests from clean. ~1 hour build
times without tests after a complete build and no other changes! It's
not incrementally correct, and it's not parallel, thus it's quite
possibly the worst build system I've ever seen.

I'm so glad i'm not the only person who thinks this! I really don't know
what the Maven guys were thinking when they wrote it. I really, *really*
don't know what Maven users are thinking when they recommend it.

tom

 

[Arved Sandstrom]

I'm so glad i'm not the only person who thinks this! I really don't know
what the Maven guys were thinking when they wrote it. I really, *really*
don't know what Maven users are thinking when they recommend it.

tom

I don't know enough about Maven to put it down vehemently. What I do
know is that I never have encountered anyone who is seriously using it.
I've heard plenty of people say that "we ought to be using it", and many
of them assert (on what grounds I do not know) that it's used a lot, but
I remain unconvinced. Downloaded a lot, no doubt. Experimented with - as
I have experimented with it - a lot, no doubt. Used a fair bit at one
remove (something you need is set to use Maven some), no doubt. But used
intensively by a lot of teams? I am skeptical.

I suspect a lot of people tout Maven, without having done more than read
a bit about it, or dabbled with it on experimental projects, because
Maven talks the good talk:

---------------
"Maven's primary goal is to allow a developer to comprehend the complete
state of a development effort in the shortest period of time. In order
to attain this goal there are several areas of concern that Maven
attempts to deal with:

"Making the build process easy
Providing a uniform build system
Providing quality project information
Providing guidelines for best practices development
Allowing transparent migration to new features"
---------------

And if your typical team lead or manager, struggling with typical
problems, reads all that, it sounds pretty good. Heck, reading through
the introductory parts of the Maven book sounds pretty good.

But based on my own experimentation the reality is different. The vision
is fine, but the implementation lacks. I've already decided that better
process and better documentation is usually what's lacking if you see
something like Maven as a lifesaver, and in fact using Maven is not
going to help you if you don't fix the core problems.

AHS

 

[Tom Anderson]

I agree this is the "best" approach. However, that would require
changing my company's mindset and culture, and doing significant
refactoring of code. The culture here is that they don't believe in
design by contract, in general purpose reusable components, and and
such, we've coded ourselves into a tangled mess.

Fair enough. I would at least considering doing the Change Employer
refactoring on yourself.

I've been reassigned to try and speed up build times, but before I was
working on one of the things at the base of the dependency tree. When I
make a change, I generally had to build everything and run all tests
because the tests for my component were not comprehensive, mostly
because of this culture of no design by contract. Instead, any small
change made at the root may have subtle nuances and break code far far
away, either at compile time or test time.

I disagree with your assessment though that no magic bullet will make
these problems go away. I don't call it a magic bullet, but an
incremental parallel build would work wonders. If I had that, compile
times would go down drastically, like 2+ orders of magnitude.

It will speed up builds. My point was that there will be other huge pains
resulting from this giant syncytial code structure, to do with local
changes having nonlocal effects. Even if a change never outright breaks
something, it means each programmer has to maintain a lot of global
knowledge just to be able to work locally. Still, as you say, them's the
breaks.

I'll bite. How long do you think it would take to recompile 20,000
java files, and just the java files? I'm about to get numbers on that
anyway, and I'll share if / when I get them.

I downloaded HtmlUnit snapshot build 1674 (the most recent). It contains
419 source files, totalling 2.9 MB. I built it with a script that said:

find src -type f -name \*.java | xargs javac -g -classpath 'lib/*' -d build/classes

(where javac is from Sun's JDK 1.6.0_16, running on Ubuntu jaunty)

I ran that, then ran it again under 'time', which reported:

real 0m16.697s
user 0m16.281s
sys 0m0.688s

Assuming compilation time scales linearly (which it probably doesn't - i
imagine it's slightly superlinear), that would suggest that compiling 20
000 files would take 796 seconds, which is 13 minutes and 17 seconds.
That's certainly not something i'd want to do with every file i change,
but it's something i'd be prepared to do several times a day. At work, we
have a build that takes 20-25 minutes (including database setup and so
on), and we run that one to five times a day when developing; it's a pain,
but it's the right amount of time to go and get a fresh cup of coffee,
tend to some minor administrivia, read up on some technical point that
came up while coding, etc.

Oh, and the build time for your case would be lower than my prediction if
your build machine is beefier than mine - an Eee PC 1001HA, with a 1.60
GHz Intel Atom N270 CPU, 1 GB of RAM, and i'm guessing a 5400 rpm disk.
Depends on whether you can afford to spend more than 200 quid on it, i
guess.

I agree with most of your assessment except with one minor
qualification: if your codebase is pure java, then it works out pretty
well. You can just get it all in Eclipse, and you have a wonderfully
good incremental compiler. However, my codebase is not all java. It has
custom inhouse codegen which makes Java.

Assuming you don't rebuild the generator (and if you are, and it changes
slowly, that's a prime candidate for factoring out into a separate build),
and that the input to the generation is a set of files which don't depend
on anything else, then this can be dealt with by some fairly
straightforward make, before any java build.

I assume that once the generation is done, there's nothing else blocking
the compilation of the java - there may be dependencies on C++ via JNI,
but the nature of JNI is that the compile-time dependency is from the java
to the C++, as not vice versa.

It has Java classes implemented in JNI, so some of the C++ compile
depends on Java compile for javah.

So after the java build, you have the javah run, and then the C++ build.

A lot of the tests are reverse: the Java tests depend upon the C++ code,

Again, a runtime rather than a compile-time dependency, i assume.

itself some of which is generated by a Java tool, which itself depends
on more C++ and Java code being built.

I'm guessing (i) that the generated C++ is test code of some description,
and that none of the main C++ depends on it, (ii) that the tool's
dependency on the C++ is again runtime, not compile-time, and that (iii)
the C++ it depends on (what i'm calling the 'main C++') has no further
dependency on generated code except javah output, and via that the java
autogeneration.

If my assumptions are correct, your phases are:

- Autogeneration of the java
- Compilation of all java
- javah
- Compilation of the non-generated C++
- Generation of the test C++
- Compilation of the test C++

That is indeed a lot of work. But it looks to me like the worst of it will
be generating and compiling the C++. BICBW.

That's ignoring entirely the biggest mess in it all: our Eclipse GUI
plugins thingy build.

Yeah, you're hosed.

Maybe in it all it could recompile the java every time. It might work,
as long as I define proper hackery to have the javah step run iff the
java source file has changed, not the class file, to not trigger
rebuilds of C++ stuff. (Would that work?)

Not triggering rebuilds is good, but i'm not sure i'd tie it to the source
file changing. I'd be tempted to always run javah (i think it's pretty
quick), but only to rebuild dependent C++ if the contents of the .h file
have changed. Running javah will make a fresh file, which will have a new
timestamp, even when the contents are the same, so you need a step to weed
this out. I'd set up two locations for javah .h files, one for javah to
write into, and one for make to read from, then add a stage which copies
files from the former to the latter iff they have different contents. And
by 'stage', i mean 'this shell script':

#! /bin/bash

set -eu

SRC_DIR=src
DST_DIR=dst

for header in $SRC_DIR/*
do
header=$(basename $header)
if [[ ! -f $DST_DIR/$header ]]
then
echo Copying new header $header
cp $SRC_DIR/$header $DST_DIR/$header
else
cmp -s $SRC_DIR/$header $DST_DIR/$header || {
echo Copying changed header $header
cp $SRC_DIR/$header $DST_DIR/$header
}
fi
done

Or some suitable improvement thereof.

That said, thinking about it, tying the javah to the .java is probably
both simpler and faster. Oh well.

However, one of my goals is still to get the build system overhead down
to seconds ideally, though I think 1-3 minutes is a more reasonable
goal. Something where I can hit "build" from root after making a change,
instead of the current situation where every developer thinks he knows
more than the build system, and only builds subfolders which he knows
are affected by his change. However, when the developer misses
something, and it gets to the official build machine streaming build, it
causes lots of lost time. I want the developer to no longer feel obliged
to "hack it" and instead let the build system do its job.

How often do developers check in to the trunk from which the main build
gets done? It would be enough to require them to do a successful full
rebuild and test immediately before doing that; being able to do it
frequently while developing would be very nice, but is not essential. If
they're checking in 0.5 - 2 times a day (roughly what we do where i work),
then a 20-30 minute build would be tolerable. If you're integrating more
frequently than that (which is of course a good thing), then it probably
isn't tolerable.

If the working cycle fits my description, you could even enforce this with
checkin hooks - every candidate checkin gets built and tested before it's
accepted.

Speaking of Eclipse, perhaps the way to go might be to just do the
entire build in Eclipse. Write Eclipse build plugins for C++, for our
custom codegen, etc. I'll have to seriously consider that.

Eclipse has some degree of C++ support - it uses an external compiler, and
possibly an external make tool too.

Your only real problem is wiring in the code generation (including javah).
I'm not any kind of Eclipse build expert, but i know there's a way to get
Eclipse to do vaguely make-like rebuilding via arbitrary processes - we
had a project which had some JAXB, and rebuilt the java binding objects if
the schema changed. I suspect this is done via ant, which doesn't have
good make-like incremental rebuilding built in, but can be used to
manually write build scripts which only rebuild when necessary, by looking
at timestamps. If there's a way to get that ant script to trigger further
rebuilding in Eclipse, then this should basically do the job.

tom

 

[Tom Anderson]

I believe my simple example is

// source foo.java
public class foo { foo() { bar x = null; } }
// source bar.java
public class bar { bar() {} }

Ooh, i hadn't thought of that one. Good catch!

I just double checked with javap, (though I didn't run my own hand-
written class file parser),

Since the class file format doesn't compress any strings, 'grep' would do.

tom

 

[Andreas Leitgeb]

This is an outrageous calumny! ...

Let's just agree on that we disagree.

 

[Andreas Leitgeb]

I disagree. My plan was to set up make rules like the following. Note
that these would not be written by hand, but generated from much
simpler input reminiscent of the input to Maven.

I wish you good luck in your endeavor.

 

[Joshua Maurice]

Joshua Maurice a écrit :



This is the problem, IMHO.
You should really split this enormous project into smaller ones with
cleanly identified responsibilities and dependencies. For example, if
your project consists of a typical n-tier project, you should identify

- utility classes, without any dependencies except the JDK and external
libraries
- domain object classes, only dependent on utility classes
- data access classes, dependent on the two above sets of classes
- service classes, dependent on the three above sets of classes
- presentation layer classes, dependent on the service layer, domain
objects and utility classes

Add interfaces for the DAO and service classes to decouple all this, and
you have even less dependencies.

This way, you have a set of projects with far less fewer dependencies,
identified dependencies rather than a spaghetti dish, and a much more
easy build.

I agree, with qualifications. That would require that I change company
culture and procedure to get well defined, stable interfaces: to get
them to program by design by contract. Instead, we were still getting
requirements from PMs like months away from the final release in a
~2-3 year long project.

So yes, that would probably be the "best" approach, but given that I
probably don't have the power to change the entire company's culture,
having a correct fast build is still a good second prize.

 

[John B. Matthews]

Well, I just spent a good many hours today whipping this up. I think
it'll work. It's rough though, and it could definitely be cleaned up.

I enjoyed reading your example. One minor cleanup:

List<String> javaFiles = new ArrayList<String>();
for (String arg : args) {
javaFiles.add(arg);
}

may be replaced by this:

List<String> javaFiles = Arrays.asList(args);

 

[Arne Vajhøj]

I don't know enough about Maven to put it down vehemently. What I do
know is that I never have encountered anyone who is seriously using it.
I've heard plenty of people say that "we ought to be using it", and many
of them assert (on what grounds I do not know) that it's used a lot, but
I remain unconvinced. Downloaded a lot, no doubt. Experimented with - as
I have experimented with it - a lot, no doubt. Used a fair bit at one
remove (something you need is set to use Maven some), no doubt. But used
intensively by a lot of teams? I am skeptical.

I suspect a lot of people tout Maven, without having done more than read
a bit about it, or dabbled with it on experimental projects, because
Maven talks the good talk:

I have seen Maven used at projects.

And I have used several open source projects that require
Maven to build.

---------------
"Maven's primary goal is to allow a developer to comprehend the complete
state of a development effort in the shortest period of time. In order
to attain this goal there are several areas of concern that Maven
attempts to deal with:

"Making the build process easy
Providing a uniform build system
Providing quality project information
Providing guidelines for best practices development
Allowing transparent migration to new features"
---------------

And if your typical team lead or manager, struggling with typical
problems, reads all that, it sounds pretty good. Heck, reading through
the introductory parts of the Maven book sounds pretty good.

But based on my own experimentation the reality is different. The vision
is fine, but the implementation lacks. I've already decided that better
process and better documentation is usually what's lacking if you see
something like Maven as a lifesaver, and in fact using Maven is not
going to help you if you don't fix the core problems.

If you can and are willing to do things the Maven way, then
it works fine.

If you need or want to do things your way, then ant is a
lot better.

Arne

 

[Tom Anderson]

Let's just agree on that we disagree.

I can't argue with that!

tom

 

[Arved Sandstrom]

I agree, with qualifications. That would require that I change company
culture and procedure to get well defined, stable interfaces: to get
them to program by design by contract. Instead, we were still getting
requirements from PMs like months away from the final release in a
~2-3 year long project.

[ SNIP ]

You actually get requirements?

AHS

 

[Arved Sandstrom]

On 09-01-2010 16:49, Arved Sandstrom wrote:

[ SNIP ]

If you can and are willing to do things the Maven way, then
it works fine.

If you need or want to do things your way, then ant is a
lot better.

Arne

That's a good way of putting it.

In any case I'd like to think I haven't been too negatively influenced
by defects and minor technical obscurities in Maven, which things after
all every software system has. Truth be told I am still trying to use it
for some projects. I'm also aware of the more recent Ant + Ivy that's
being extolled as the latest fad, and am checking that out some.

But I'll stand by my observation that none of these systems are the Holy
Grail. I do believe that if people are grasping for any of these straws
that their basic processes probably need more fundamental help than just
choosing a new software project management system.

AHS

 

[Arne Vajhøj]

On 09-01-2010 16:49, Arved Sandstrom wrote:

[ SNIP ]

If you can and are willing to do things the Maven way, then
it works fine.

If you need or want to do things your way, then ant is a
lot better.

That's a good way of putting it.

In any case I'd like to think I haven't been too negatively influenced
by defects and minor technical obscurities in Maven, which things after
all every software system has. Truth be told I am still trying to use it
for some projects. I'm also aware of the more recent Ant + Ivy that's
being extolled as the latest fad, and am checking that out some.

But I'll stand by my observation that none of these systems are the Holy
Grail. I do believe that if people are grasping for any of these straws
that their basic processes probably need more fundamental help than just
choosing a new software project management system.

Absolutely.

It is just a build tool.

It can not by magic make the entire project well structured.

That is like expecting a hopeless programmer to become Knuth
by giving him a 8000 dollar IDE.

Arne

 

[Markus Eberle]

Am 10.01.2010 00:44, schrieb Joshua Maurice:

I agree, with qualifications. That would require that I change company
culture and procedure to get well defined, stable interfaces: to get
them to program by design by contract. Instead, we were still getting
requirements from PMs like months away from the final release in a
~2-3 year long project.

You do not need to create any new responsibilities for each layer, but
the developers are not allowed to put their code where ever they like
but in defined places.

It is just some kind of cleanup in you code. You can still have any
developer change any file (or whatever rules you have)

Cheers,
Markus

 

[Alessio Stalla]

No - because you don't even have enough information to *detect* these
cases. Consider:

// A.java
class A {
        public static final String FOO = "foo";

}

// B.java
class B {
        public static final String FOO = A.FOO;

}

You change A.java. How do you know you have to recompile B?

You don't know. You adopt the rule that whenever you modify a static
final field, you do a full clean and build. My opinion is that it will
happen rarely enough to not slow down development too much.

Alessio