D
Danno
I just want to know you're experience. Do you use both ant and
scripting? Pure script builds? Do you use Groovy or Ruby?
scripting? Pure script builds? Do you use Groovy or Ruby?
A
Andy Dingley
Danno said:
Hmmm... Two quite different questions there, but they're damn good
ones.
I have build problems like you wouldn't believe. My full-time job title
is "Build Manager", so you can imagine how big a problem it has to be
to grow and keep a full-time role involved in it.
Currently we're using both Ant and Python, with a bunch of XSLT and
still some legacy shell. Bugzilla and Subversion are tightly integrated
into this. The Ant build.xml is over 1000 lines and there's another
couple of thousand in Python.
My attempt to impose sanity on this is focussed round the ideas of
"Internal" and "External" builds. The Internal build is Ant, and it
begins by assuming an accurate source tree that's all in place locally
and ends with a couple of well-behaved JARs and ZIPs sitting locally.
Nothing changes outside this local directory tree, and especially
nothing changes in the repositories.
Everything else, both before and after this, is handled in the
"External" build, which is in Python. This starts with a build request
in Bugzilla and ends with big single ZIPs on an ftp server, including
generated docs, and will also have updated Subversion, Bugzilla and
sent a few emails.
The idea really is that the Internal part is idempotent, whilst the
External bit generates paperwork. Developers use the same Internal part
over and over, the External bit only gets fired up for real Builds that
are going out to QA or customers.
In fact my developers _don't_ use the Internal build. They tend to use
Eclipse and let Eclipse handle things its own way. This sucks when they
then hand over a completed project with its unused and ignored
buildfile in tatters. If another one does it at 5pm after telling the
customer "it's ready now", then there'll be blood on the carpet 8-(
We're also heavily into CurseControl and Fitnesse. Those have a
dependency on Ant we could work round, but haven't felt any urge to.
Ant sucks for its lack of string handling. This is so poor that I don't
even like using Ant with a command line, as it's impractical to
validate command line parameters. Ant is good in that it understands
dependencies, which is slightly more use than a chocolate teapot. In
all this mess though, I only have one place where I actually use
dependencies.
Ant is probably more use for deployment, where a simple initial choice
(particularly which application server is in use) implies a whole chain
of other tasks. I'm also using it as a SQL client tool during
installation, because I know it's available, I know JDBC is already set
up, it doesn't care whether it's connected to Oracle or SQL Server and
it even does dependencies between successive patch scripts.
I'd probably use Ant again, if I started from scratch. -- But I'm
having a hard time to really think _why_.
I'm impressed with Python though. Six months ago I wrote bad Perl, then
I inherited this bundle mostly as shell scripts. I've gone from nowhere
on Python to devotee in just that time.
S
Simon Brooke
Danno said:
Most of us have ditched scripting (i.e. makefiles) for ant; ant is a very,
very good solution. Ant is, of course, a scripting approach with a
XML-syntax scripting language, but its flexibility and extensibility make
it extremely effective.
S
Scott Harper
Andy,
This is probably not the exact best forum for this discussion, but since I
didn't start the thread, I'll tag on anyway... Perhaps there are others out
there who can add to it as well.
We've been trying to get our build system in place. The majority of our
developers use Eclipse, but we want to be able to build from the command line
(automated) as well. I *think* I can handle this by exporting the Eclipse
build files and by some judicious use of environment variables/properties.
But what we are having problems with is how to best organize our "projects" in
Eclipse and CVS.
We have a bunch of Java packages that we want to mix and match in various
application projects -- think libraries of utilities. How do people generally
orgainize these in CVS and Eclipse? I'd like to keep the library package
projects separate from the application projects, so someone doesn't
accidentally change some "library" code -- not completely realizing that it's
really global to their app -- and breaking a bunch of other apps.
Is it a good idea to keep each package as it's own directory in the CVS tree?
We've looked at trying to use CVS modules to rearrage and combine source code
into more modular units, but it really isn't well suited for that.
Then, do people import these projects individually into the Eclipse workspace,
then create inter-project dependencies? Or is there a better way to
incorporate subordinate projects into a bigger project? Seems like if you
"import" code from another project it makes a local copy, which kind of
defeats the purpose of source code control...
It sounds complicated, but surely this isn't an uncommon way to approach
software development...
scott
G
Gordon Beaton
When did these become equivalent?
/gordon
S
Simon Brooke
Gordon Beaton said:
Make is a scripting language interpreter in itself, and uses conventional
shell scripting languages as the basis both for its macros and for its
actions.
A
Andy Dingley
Gordon said:
They clearly didn't!
There are (at least) three levels here: Make, Ant and procedural
scripts.
Make is a good solution to a problem of 30 years ago, that of reducing
compilation to the minimum number of affected compilation units. These
days compilation is so fast that we just don't care about this - we can
re-compile from scratch in negligible time (certainly compared to
running the unit tests) and so the "blunderbuss" approach to
compilation is adequate. I don't need to spend time (and lots of it)
managing the file-level dependencies I used to do with Make.
Ant is powerful, but I'm increasingly coming to the opinion that it's
power I don't need. It does "task level" dependencies such that I can
re-run the compile target before the JAR-maker, but this is of little
_real_ benefit. I have a big Ant file with lots of separate targets,
but the practicality is that I only ever use two execution paths
through this: "Changed some source, so re-compile it and do the rest"
or "This one is going out to a punter, so run the whole lot from
scratch in clean directories". So there's the great virtue of Ant, and
I'm not even using it. I don't need target dependencies here, just a
few lines of rigid procedural script would be adequate.
I also hate Ant's lack of string-slicing. IMHO (to paraphrase emacs and
email), every scripting tool should expand until it has the ability to
use regexes. It's not much to ask, but I feed my buildfile a build tag
parameter and need to make deliverable zipfiles with names based on it.
This is _painful_ in Ant. One of my Ants has about 800 lines of 1500
total devoted just to parameter-handling logic. I could probably
replace this with 20 lines of Python / Visual Basic.
When I started this project, I went through an aberrant phase of
writing custom Ant tasks. These were supposedly great - I could do
anything, my Ant was simple and elegant afterwards, and we're a Java
shop so the language was familiar. I no longer do this anywhere, and
would be most reluctant to ever do it again. Custom tasks are a
terrible way to work - just don't go there!
The problem is that custom tasks split your build in two, quite
probably even in two different editors. If you have "business logic" in
the build, this is an obvious maintenance problem.
_Extending_ Ant by writing new tasks is OK, provided that the
requirements for the task are sufficiently generic. A task that
understands regexes is one thing, because it's stable within itself. A
task that understands your local build tag format and maps it in and
out of SVN paths is quite another problem -- day-to-day maintenance and
process development means working across two platforms, with an
unstable interface.
I'm also reluctant to use Ant tasks that are outside the basic Ant
package (so JUnit and FOP are just about OK) because I have to assume
an Ant environment in many locations, even customer sites. It's no use
using Ant for an easy product deployment if I first have to work
through a non-standard Ant deployment.
Ant might still have something to offer for multi-platform deployment.
Problem of the week is deploying what began as a pure Tomcat
development onto Weblogic / Websphere and JBoss as well. This is messy
and Ant (or at least the Ant community and collected best practices)
has things to offer for cleaning it up.
So if I don't (very often) need task-level dependencies and I don't
need file level dependencies in compilation, then what's left for Ant
to do ? And of this remainder, is Any any good at doing it?
As I mentioned before, I'm moving to more and more scripting (Python)
but keeping this outside a somewhat arbiitrary boundary of the
Internal/External build. With hindsight though (and some process dev
time before the build-request-batphone next rings) I'd certainly be
thinking about a pure procedural script approach.
T
Thomas Weidenfeller
Simon said:
Make is a rule-based engine - of course interpreting its own language.
However, that language is fare from being a scripting language.
No. Make interprets its macros, and does not enlist the help of a shell
or similar to do so.
It forwards the actions to an interpreter of your choice. That one is
typically a shell, but can be anything you like it to be.
/Thomas
A
Andy Dingley
Simon said:
The fact (debatable) that Make is a scripting language certainly
doesn't mean that scripting languages are make! "Equivalence" would
require both of these to be true.
While it's arguable that "make does some scripting" the key capability
of make is the ability to take a graph of dependencies and linearise it
to a procedural sequence. Scripting languages (for many and various
values of "scripting language") don't have this ability.
D
Danno
Andy said:
Wow, you can have a full time job out of it?
So you have people request a product build in Bugzilla, and you use
that to initiate all the work to get the product build started?
Since CruiseControl requires that the external build works, wouldn't
you know then who is suspect when someone commits a tattered project?
Exactly, this is my problem which led me to put up this post.
I am going to move towards Groovy instead of Python. Groovy has some
documentation (http://groovy.codehaus.org/Groovy+Ant+Task) that can
integrate with ant as far as building. Since I know Ruby, I think
Groovy would be a nice fit. I appreciate you taking time to tell me
about your background and your building techniques.
D
Danno
Simon said:
I've used ant for years, I just have more complicated scenarios that I
need to use a scripting language (e.g. Groovy, Ruby, JavaScript, Perl).
That being said, I have to heartily disagree that make, ant, and XML
are scripting languages. Make and ant are tools, that use scripts, but
that doesn't mean those scripts are scripting languages. XML is a
markup language (hence the ML in XML) , so XML is not a scripting
language. I do agree that make is much better than ant though.
C
Chris Stiles
Danno said:
Ant is a batch file interpreter that understands XML and has flow control via
implicit gotos (task dependancies).
A
Andy Dingley
Danno said:
Ant is a scripting language. Make one target, stick a sequence of tasks
in it. Use <antcall> as a subroutine mechanism. You've got yourself a
scripting language.
It's not a _good_ scripting language, but it's still better than
Windows offers.
The question is whether Ant's advantage of _also_ (and separately)
being a dependency-flattening langauge is sufficient to excuse the poor
scripting features, when our application of it is generally more
dependent on scripting than anything else.
A
Andy Dingley
Danno said:
1/3rd getting builds out
1/3rd improving the build process, writing scripts to reduce #1,
telling developers where to stick their code.
1/3rd chasing the developers who didn't follow #2 with my clueiron
http://codesmiths.com/shed/things/clueiron/
1/3rd generally being sworn at over the phone
Yes. You also use Bugzilla to request your PC to be fixed, or to
recruit a new developer.
Usually, but not always. The usual culprit is brand new work, where it
hasn't yet been added to our Fitnesse tests. Even when it has been
"added", it needs an active inspection by me to ensure that everything
is as it should be. The process isn't watertight, particularly when
timescales are pressured.
The "Agile" malarkey is great, but it's not very agile. Works a _lot_
better when you're doing incremental work on an established project,
rather than creating new projects.
I considered that too, but it seemed too much like my previous
experience of embedding Java snippets inside my Ant, when what I really
needed was a wrapper around it.
D
Danno
Andy said:
Whew, I'd have to strain to see that.
T
Thomas Weidenfeller
Andy said:
And another 1/3rd to finish your math degree along the way, right?
/Thomas
P
pkriens
Agree with this question! Only bad makefiles looked like scripts. GoodGordon said:
makefiles used the dependency facility which is declarative. However,
most makefiles I have seen became highly procedureal because the
maintainers did not understand the declarative approach of make.
Kind regards,
Peter Kriens
A
Andy Dingley
Thomas said:And another 1/3rd to finish your math degree along the way, right?
If three thirds add up to 1, that's arithmetic.
If four thirds add up to 1, that's maths.
If an uncertain number of thirds definitely add up to 1, but you can't
tell how many, that's crypto.
D
David Zimmerman
Sounds like the 90/10 rule of software
The first 90% of the job takes 90% of the time.
The remaining 10% of the job takes the other 90% of the time.
The first 90% of the job takes 90% of the time.
The remaining 10% of the job takes the other 90% of the time.
C
Chris Uppal
David said:
Nice...
Up there with "n+1 trivial tasks will take the same time as n trivial tasks".
-- chris