The curse of constant fields

[Arne Vajhøj]

Is it a problem, though? Put the full paths of all referenced source
files in the dependency database. If anything's different on the next
run, you need to recompile.

What about situations where you have multiple files with the
same classname within the project. You will either need a
sourcepath defining a search order or to read and analyze
the build script to understand the structure.

That would not be a problem in 99.9% of projects, but tools
that does not work in 0.1% of cases can be very frustrating.

Arne

 

[Tom Anderson]

That was my point to begin with: in C or C++, date comparisons (which
are much simpler) suffice.

How so? I have a directory full of .c and .h files. And object files if
you like. I edit a .c file, and recompile it. How can the compiler tell
which other files need to be recompiled?

I don't think it can. And it's what the tool i was describing would do. So
date comparisons do not suffice.

tom

 

[Tom Anderson]

I am willing, and offer here publicly, to invite Mr. Bloch for a few beers
and a chat about Java or any other topic that interest him. If he is ever in
the DC or Maryland area and is willing, the first six-pack is on me.

We should send him an email and let him know.

My current favorite beer is "Loose Cannon" from Clipper City Brewing Co.
It's got triple the hops of even Sam Adams brews.

I have heard of this extreme hopping trend in the US. I don't get it. More
hops does not automatically make a better beer. Hops are a vital part of
beer, of course, but there is a question of balance.

tom

 

[Stefan Ram]

I have a directory full of .c and .h files. (...)
How can the compiler tell which other files need to be recompiled?

From the makefile.

 

[Tom Anderson]

What about situations where you have multiple files with the
same classname within the project.

When does that happen?

You will either need a sourcepath defining a search order or to read and
analyze the build script to understand the structure.

In that situation, yes.

That would not be a problem in 99.9% of projects, but tools that does
not work in 0.1% of cases can be very frustrating.

Better than no tool for 99.9% of people!

tom

 

[Mike Schilling]

How so? I have a directory full of .c and .h files. And object files
if you like. I edit a .c file, and recompile it. How can the
compiler
tell which other files need to be recompiled?

If you edit a .c file, only that .c file needs to be recompiled. If
you edit a .h file, and .c file which includes it (directly or
indirectly) needs to be recompiled, and that can be determined by a
straightforward analysis of the source files. The usual solution is to
run a makemake script that does this analysis and creates make files
that represent the dependence of .c files on .h files. It's a solved
problem.

 

[Mike Schilling]

What about situations where you have multiple files with the
same classname within the project. You will either need a
sourcepath defining a search order or to read and analyze
the build script to understand the structure.

Can you explain that situation further? Multiple source files for the
same class sounds like a serious problems to me.

 

[Lew]

I have heard of this extreme hopping trend in the US. I don't get it.
More hops does not automatically make a better beer. Hops are a vital
part of beer, of course, but there is a question of balance.

"Loose Cannon" beer is entirely unbalanced, so it seems to earn its brand
name. There's so much hops in it, it tastes citrusy. It was a major shock
the first time I tasted it, but after I got used to it, I began to appreciate
it, even like it. Now it's my favorite, at least for the time being.

In many ways lewscanon is similar.

 

[Tom Anderson]

The idea of using automated tests for the web GUI has been bruited, but in
our environment it would be unrealistic.

Interesting. Can i ask why?

We have a team of dedicated testers (they've come up through the ranks on
the business side and know exactly what should happen), and quite a few
others that do it part-time as required (again, business-side people who
know the business rules to a T). By the time a production build is to be
produced, a test build which is identical except for some configuration has
been deployed in a similar environment, and tested for weeks (or months).

It might be more accurate to say that I am quite concerned to make sure
that the test builds are correct. Because a lot of time can be wasted if
a tester reports that an error is still there, and then it's a question
of is the error still there because the build is flawed, or because the
developer who "fixed" it only fixed it for a different use case or in
his own development environment.

Yes, i can see that if your testing process is as large-calibre as yours
is, you want to make sure you're not wasting its time on avoidable
mistakes.

We pretty much have the klaxon and flashing lights. If a build
breaks then Hudson emails every developer, and the email tells you what
the base event was (whose commit), and what broke (if JUnit tests then
you follow the emailed link and drill down). Hudson itself is quite easy
to configure, and jobs (say one for trunk, one for each feature branch
etc) are also very easy to configure. What's nice too is that the Ant
build.xml (if such you choose to use) that you point Hudson at is
exactly the same thing that you'd use in your dev environment, or test
or production.

That all sounds good. I have no idea what we're using for nightly builds -
knowing the guy who's our semi-official buildmaster, a bunch of perl
scripts. Not necessarily a bad thing, mind!

In fact, i fantasise about having a build and test running on a
pre-commit hook, so that if you try to check in code that doesn't build
and run, it gets rejected! [...] Obviously you'd also need a fairly
fast set of unit tests - you'd need to flag anything slow as not to be
run on checkin.

This is one thing we don't enforce. It has not been an issue to date.

With the exception of slow tests (slow by their nature), I haven't seen
that large test suites take so long to run that a developer couldn't
execute them on each commit. On the specific app I refer to we have
probably close to 3000 JUnit tests, and it's on the order of a minute on
my local box to run them all. They're typically not trivial tests
either.

I've had mixed experiences. Mostly, the same as you - tests are fast.
Three projects i've worked on have been quite different, though. Two are
web apps where we do heavy testing through the web interface, so every
test is making multiple roundtrips up and down the whole stack, from web
server down to RDBMS via a zillion layers in between. The third is
HtmlUnit, which has about 3000 tests (IIRC), and which take quite a while
to run, because a lot of them involve running gigantic amounts of
javascript (the point being to test HtmlUnit's javascript handling). All
of those test suites ran in <5 minutes, though, which would still be
doable on a checkin hook, just about. It might be kinder to only run a
subset, though.

Indeed, we do run the complete test set before a commit - but it's up to
the developers to remember to do it. Enforcing it by machine would be
preferable!

Speaking of tests and Hudson, one handy other thing to hook in is test
coverage, like Emma. It just gets added to the script that Hudson is
provided...you end up with nice graphs of coverage at various levels.
IMO this is indispensable (even in a TDD environrment) for staying on
top of whether your tests are sufficiently blanketing the codebase.

Yes, we haven't been using any coverage tools. It'd be something i'd like
to try. I'd really like to have ways of knowing whether our tests are any
good. Quis custodiet ipsos custodes?

tom

 

[Tom Anderson]

I think there's an even easier way which could easily be built into ant.

Instead of merely compiling every class which is older than the
corresponding source file, set a target timestamp to match the most
recent class file and then recompile all classes whose sources have been
amended more recently than the target. That will sometimes do more than
the minimum amount of work, but I don't think it will ever fail to
recompile everything that needs to be recompiled.

Put these three classes in their own files:

public class Foo {
public int m() {
return 1 ;
}
}

public class Bar
public int n() {
return new Foo().m() ;
}
}

Compile them both.

Now edit Foo to read:

public class Foo {
public String m() {
return "one" ;
}
}

At this point, your algorithm would only recompile Foo, even though Bar is
now wrong, and needs to be recompiled to detect that fact.

tom

 

[Martin Gregorie]

Put these three classes in their own files:

public class Foo {
public int m() {
return 1 ;
}
}

public class Bar
public int n() {
return new Foo().m() ;
}
}

Compile them both.

Now edit Foo to read:

public class Foo {
public String m() {
return "one" ;
}
}

At this point, your algorithm would only recompile Foo, even though Bar
is now wrong, and needs to be recompiled to detect that fact.

OK, so a dependency tree is needed after all.

Has a Java dependency scanner ever been made? Presumably it would be
somewhat more complex than makemake.

 

[Juha Laiho]

...

You have my sympathy for your frustration, but I would call this story
"The curse of manual builds".

It is that as well - but at the moment I haven't had the chance of
fully validating that the source under VC matches with what is deployed,
so for a minor change I'm really not ready to do a full build. Once
I've convinced myself that the two sides truly match, I'm happy to
switch to ant. But, alas, such are the priorities; that was something
I was working with last week, this week it's another piece of software
that keeps me busy. Looks like eventually I will have the time to go
through that application and judge it safe for full builds, but that
time is not now.

 

[Juha Laiho]

Really?

I was thinking about this a while ago, and it seemed to me that the
dependencies a class had were moderately simple - every other class it
references directly, and every class which is a superclass of a class it
references. Am i missing something?

Then there's also the deliberate shoot-myself-in-the-foot introspection
(i.e. use of methods forName(), getConstructor(), getMethod() and the
like from class Class).

 

[Arne Vajhøj]

OK, so a dependency tree is needed after all.

Has a Java dependency scanner ever been made? Presumably it would be
somewhat more complex than makemake.

I believe that no links have been posted in this thread.

So either it does not exist or it is a well kept secret.

Arne

 

[Mike Schilling]

I believe that no links have been posted in this thread.

So either it does not exist or it is a well kept secret.

Jikes did something with dependency processing, but I've never used it
myself.

 

[Martin Gregorie]

I believe that no links have been posted in this thread.

So either it does not exist or it is a well kept secret.

So it would appear.

Hint: writing a dependency analyser would be a nice project for anybody
who wants to get to grips with the Coco/R lexer.

 

[Volker Borchert]

Jikes did something with dependency processing, but I've never used it
myself.

Is Jikes still being maintained?

 

[Arved Sandstrom]

Interesting. Can i ask why?

The original writers of the app - they converted a mainframe legacy app over
to a J2EE one - never saw fit to write web tests, only a reasonably
comprehensive set of JUnit tests. Also, the pages have been more of a moving
target. Now that there are completely different people adding to, and
fixing, the application, none of whom had a hand in writing it originally,
and there aren't many of us, we have no time to write a sufficient body of
web tests (and there would have to be hundreds).

They'd be nice to have...nobody denies that. We were also left with no
documentation...what saves us is the fact that we're surrounded by people
who know what the app is supposed to do in the business sense, since it's
not like the rules for motor vehicles changed all that much in the
transition from legacy to J2EE. But web tests would help document those
business rules, and also assist maintenance programmers who have a hard time
reproducing errors in what is quite a complex front-end.

[ SNIP ]

Yes, we haven't been using any coverage tools. It'd be something i'd like
to try. I'd really like to have ways of knowing whether our tests are any
good. Quis custodiet ipsos custodes?

tom

Having had to put some thought into my replies, I'm getting more convinced
that we really need to have a good set of web tests. Our human testers are
fantastic, but it takes many weeks to completely regression test the
application. The JUnit tests miss most of the business logic, quite frankly,
since that is linked into the interplay of JSF managed beans and session
beans. The fact that all of our thousands of JUnit tests pass has to be held
up alongside the fact that there are hundreds of listed defects.

So code coverage linked with a good set of *web* tests is, I believe, the
way to go for a J2EE app. The JUnit tests are still necessary, but by
themselves they don't help much.

AHS

 

[Mike Schilling]

Is Jikes still being maintained?

No, and it never got as far as full 1.5 support. That's why I say "did"
instead of "does".

 

[Volker Borchert]

No, and it never got as far as full 1.5 support. That's why I say "did"
instead of "does".

Are there any other _fast_ (3rd party) compilers? Target system is
a 10/71 running 5.7. Is the compiler from gc* useable? (I'm not keen
on bootstrapping that monster though.)