Has anyone here ditched ant in favor of scripting for builds?

Discussion in 'Java' started by Danno, Oct 3, 2006.

  1. Danno

    Danno Guest

    I just want to know you're experience. Do you use both ant and
    scripting? Pure script builds? Do you use Groovy or Ruby?
    Danno, Oct 3, 2006
    #1
    1. Advertising

  2. Danno

    Andy Dingley Guest

    Danno wrote:

    > Has anyone here ditched ant in favor of scripting for builds?
    > Do you use both ant and scripting?


    Hmmm... Two quite different questions there, but they're damn good
    ones.

    I have build problems like you wouldn't believe. My full-time job title
    is "Build Manager", so you can imagine how big a problem it has to be
    to grow and keep a full-time role involved in it.

    Currently we're using both Ant and Python, with a bunch of XSLT and
    still some legacy shell. Bugzilla and Subversion are tightly integrated
    into this. The Ant build.xml is over 1000 lines and there's another
    couple of thousand in Python.

    My attempt to impose sanity on this is focussed round the ideas of
    "Internal" and "External" builds. The Internal build is Ant, and it
    begins by assuming an accurate source tree that's all in place locally
    and ends with a couple of well-behaved JARs and ZIPs sitting locally.
    Nothing changes outside this local directory tree, and especially
    nothing changes in the repositories.

    Everything else, both before and after this, is handled in the
    "External" build, which is in Python. This starts with a build request
    in Bugzilla and ends with big single ZIPs on an ftp server, including
    generated docs, and will also have updated Subversion, Bugzilla and
    sent a few emails.

    The idea really is that the Internal part is idempotent, whilst the
    External bit generates paperwork. Developers use the same Internal part
    over and over, the External bit only gets fired up for real Builds that
    are going out to QA or customers.

    In fact my developers _don't_ use the Internal build. They tend to use
    Eclipse and let Eclipse handle things its own way. This sucks when they
    then hand over a completed project with its unused and ignored
    buildfile in tatters. If another one does it at 5pm after telling the
    customer "it's ready now", then there'll be blood on the carpet 8-(

    We're also heavily into CurseControl and Fitnesse. Those have a
    dependency on Ant we could work round, but haven't felt any urge to.

    Ant sucks for its lack of string handling. This is so poor that I don't
    even like using Ant with a command line, as it's impractical to
    validate command line parameters. Ant is good in that it understands
    dependencies, which is slightly more use than a chocolate teapot. In
    all this mess though, I only have one place where I actually use
    dependencies.

    Ant is probably more use for deployment, where a simple initial choice
    (particularly which application server is in use) implies a whole chain
    of other tasks. I'm also using it as a SQL client tool during
    installation, because I know it's available, I know JDBC is already set
    up, it doesn't care whether it's connected to Oracle or SQL Server and
    it even does dependencies between successive patch scripts.

    I'd probably use Ant again, if I started from scratch. -- But I'm
    having a hard time to really think _why_.

    I'm impressed with Python though. Six months ago I wrote bad Perl, then
    I inherited this bundle mostly as shell scripts. I've gone from nowhere
    on Python to devotee in just that time.
    Andy Dingley, Oct 3, 2006
    #2
    1. Advertising

  3. Danno

    Simon Brooke Guest

    in message <>, Danno
    ('') wrote:

    > I just want to know you're experience. Do you use both ant and
    > scripting? Pure script builds? Do you use Groovy or Ruby?


    Most of us have ditched scripting (i.e. makefiles) for ant; ant is a very,
    very good solution. Ant is, of course, a scripting approach with a
    XML-syntax scripting language, but its flexibility and extensibility make
    it extremely effective.

    --
    (Simon Brooke) http://www.jasmine.org.uk/~simon/

    <p>Schroedinger's cat is <blink><strong>NOT</strong></blink> dead.</p>
    Simon Brooke, Oct 3, 2006
    #3
  4. Danno

    Scott Harper Guest

    In article <>, "Andy Dingley" <> wrote:
    >I have build problems like you wouldn't believe. My full-time job title
    >is "Build Manager", so you can imagine how big a problem it has to be
    >to grow and keep a full-time role involved in it.


    [...]

    >In fact my developers _don't_ use the Internal build. They tend to use
    >Eclipse and let Eclipse handle things its own way. This sucks when they
    >then hand over a completed project with its unused and ignored
    >buildfile in tatters. If another one does it at 5pm after telling the
    >customer "it's ready now", then there'll be blood on the carpet 8-(


    Andy,

    This is probably not the exact best forum for this discussion, but since I
    didn't start the thread, I'll tag on anyway... Perhaps there are others out
    there who can add to it as well.

    We've been trying to get our build system in place. The majority of our
    developers use Eclipse, but we want to be able to build from the command line
    (automated) as well. I *think* I can handle this by exporting the Eclipse
    build files and by some judicious use of environment variables/properties.

    But what we are having problems with is how to best organize our "projects" in
    Eclipse and CVS.

    We have a bunch of Java packages that we want to mix and match in various
    application projects -- think libraries of utilities. How do people generally
    orgainize these in CVS and Eclipse? I'd like to keep the library package
    projects separate from the application projects, so someone doesn't
    accidentally change some "library" code -- not completely realizing that it's
    really global to their app -- and breaking a bunch of other apps.

    Is it a good idea to keep each package as it's own directory in the CVS tree?
    We've looked at trying to use CVS modules to rearrage and combine source code
    into more modular units, but it really isn't well suited for that.

    Then, do people import these projects individually into the Eclipse workspace,
    then create inter-project dependencies? Or is there a better way to
    incorporate subordinate projects into a bigger project? Seems like if you
    "import" code from another project it makes a local copy, which kind of
    defeats the purpose of source code control...

    It sounds complicated, but surely this isn't an uncommon way to approach
    software development...


    scott
    Scott Harper, Oct 4, 2006
    #4
  5. On Tue, 03 Oct 2006 22:36:17 +0100, Simon Brooke wrote:
    > scripting (i.e. makefiles)


    When did these become equivalent?

    /gordon

    --
    [ don't email me support questions or followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
    Gordon Beaton, Oct 4, 2006
    #5
  6. Danno

    Simon Brooke Guest

    in message <45237bc3$0$1134$>, Gordon Beaton
    ('') wrote:

    > On Tue, 03 Oct 2006 22:36:17 +0100, Simon Brooke wrote:
    >> scripting (i.e. makefiles)

    >
    > When did these become equivalent?


    Make is a scripting language interpreter in itself, and uses conventional
    shell scripting languages as the basis both for its macros and for its
    actions.

    --
    (Simon Brooke) http://www.jasmine.org.uk/~simon/

    'graveyards are full of indispensable people'
    Simon Brooke, Oct 4, 2006
    #6
  7. Danno

    Andy Dingley Guest

    Gordon Beaton wrote:

    > > scripting (i.e. makefiles)

    >
    > When did these become equivalent?


    They clearly didn't!

    There are (at least) three levels here: Make, Ant and procedural
    scripts.

    Make is a good solution to a problem of 30 years ago, that of reducing
    compilation to the minimum number of affected compilation units. These
    days compilation is so fast that we just don't care about this - we can
    re-compile from scratch in negligible time (certainly compared to
    running the unit tests) and so the "blunderbuss" approach to
    compilation is adequate. I don't need to spend time (and lots of it)
    managing the file-level dependencies I used to do with Make.

    Ant is powerful, but I'm increasingly coming to the opinion that it's
    power I don't need. It does "task level" dependencies such that I can
    re-run the compile target before the JAR-maker, but this is of little
    _real_ benefit. I have a big Ant file with lots of separate targets,
    but the practicality is that I only ever use two execution paths
    through this: "Changed some source, so re-compile it and do the rest"
    or "This one is going out to a punter, so run the whole lot from
    scratch in clean directories". So there's the great virtue of Ant, and
    I'm not even using it. I don't need target dependencies here, just a
    few lines of rigid procedural script would be adequate.

    I also hate Ant's lack of string-slicing. IMHO (to paraphrase emacs and
    email), every scripting tool should expand until it has the ability to
    use regexes. It's not much to ask, but I feed my buildfile a build tag
    parameter and need to make deliverable zipfiles with names based on it.
    This is _painful_ in Ant. One of my Ants has about 800 lines of 1500
    total devoted just to parameter-handling logic. I could probably
    replace this with 20 lines of Python / Visual Basic.

    When I started this project, I went through an aberrant phase of
    writing custom Ant tasks. These were supposedly great - I could do
    anything, my Ant was simple and elegant afterwards, and we're a Java
    shop so the language was familiar. I no longer do this anywhere, and
    would be most reluctant to ever do it again. Custom tasks are a
    terrible way to work - just don't go there!

    The problem is that custom tasks split your build in two, quite
    probably even in two different editors. If you have "business logic" in
    the build, this is an obvious maintenance problem.

    _Extending_ Ant by writing new tasks is OK, provided that the
    requirements for the task are sufficiently generic. A task that
    understands regexes is one thing, because it's stable within itself. A
    task that understands your local build tag format and maps it in and
    out of SVN paths is quite another problem -- day-to-day maintenance and
    process development means working across two platforms, with an
    unstable interface.

    I'm also reluctant to use Ant tasks that are outside the basic Ant
    package (so JUnit and FOP are just about OK) because I have to assume
    an Ant environment in many locations, even customer sites. It's no use
    using Ant for an easy product deployment if I first have to work
    through a non-standard Ant deployment.

    Ant might still have something to offer for multi-platform deployment.
    Problem of the week is deploying what began as a pure Tomcat
    development onto Weblogic / Websphere and JBoss as well. This is messy
    and Ant (or at least the Ant community and collected best practices)
    has things to offer for cleaning it up.

    So if I don't (very often) need task-level dependencies and I don't
    need file level dependencies in compilation, then what's left for Ant
    to do ? And of this remainder, is Any any good at doing it?

    As I mentioned before, I'm moving to more and more scripting (Python)
    but keeping this outside a somewhat arbiitrary boundary of the
    Internal/External build. With hindsight though (and some process dev
    time before the build-request-batphone next rings) I'd certainly be
    thinking about a pure procedural script approach.
    Andy Dingley, Oct 4, 2006
    #7
  8. Simon Brooke wrote:
    > Make is a scripting language interpreter in itself,


    Make is a rule-based engine - of course interpreting its own language.
    However, that language is fare from being a scripting language.

    > and uses conventional
    > shell scripting languages as the basis both for its macros


    No. Make interprets its macros, and does not enlist the help of a shell
    or similar to do so.

    > and for its
    > actions.


    It forwards the actions to an interpreter of your choice. That one is
    typically a shell, but can be anything you like it to be.

    /Thomas
    --
    The comp.lang.java.gui FAQ:
    http://gd.tuwien.ac.at/faqs/faqs-hierarchy/comp/comp.lang.java.gui/
    ftp://ftp.cs.uu.nl/pub/NEWS.ANSWERS/computer-lang/java/gui/faq
    Thomas Weidenfeller, Oct 4, 2006
    #8
  9. Danno

    Andy Dingley Guest

    Simon Brooke wrote:

    > in message <45237bc3$0$1134$>, Gordon Beaton
    > ('') wrote:
    >
    > > On Tue, 03 Oct 2006 22:36:17 +0100, Simon Brooke wrote:
    > >> scripting (i.e. makefiles)

    > >
    > > When did these become equivalent?

    >
    > Make is a scripting language interpreter in itself,


    The fact (debatable) that Make is a scripting language certainly
    doesn't mean that scripting languages are make! "Equivalence" would
    require both of these to be true.

    While it's arguable that "make does some scripting" the key capability
    of make is the ability to take a graph of dependencies and linearise it
    to a procedural sequence. Scripting languages (for many and various
    values of "scripting language") don't have this ability.
    Andy Dingley, Oct 4, 2006
    #9
  10. Danno

    Danno Guest

    Andy Dingley wrote:
    > Danno wrote:
    >
    > > Has anyone here ditched ant in favor of scripting for builds?
    > > Do you use both ant and scripting?

    >
    > Hmmm... Two quite different questions there, but they're damn good
    > ones.
    >
    > I have build problems like you wouldn't believe. My full-time job title
    > is "Build Manager", so you can imagine how big a problem it has to be
    > to grow and keep a full-time role involved in it.


    Wow, you can have a full time job out of it?

    >
    > Currently we're using both Ant and Python, with a bunch of XSLT and
    > still some legacy shell. Bugzilla and Subversion are tightly integrated
    > into this. The Ant build.xml is over 1000 lines and there's another
    > couple of thousand in Python.
    >
    > My attempt to impose sanity on this is focussed round the ideas of
    > "Internal" and "External" builds. The Internal build is Ant, and it
    > begins by assuming an accurate source tree that's all in place locally
    > and ends with a couple of well-behaved JARs and ZIPs sitting locally.
    > Nothing changes outside this local directory tree, and especially
    > nothing changes in the repositories.
    >
    > Everything else, both before and after this, is handled in the
    > "External" build, which is in Python. This starts with a build request
    > in Bugzilla and ends with big single ZIPs on an ftp server, including
    > generated docs, and will also have updated Subversion, Bugzilla and
    > sent a few emails.


    So you have people request a product build in Bugzilla, and you use
    that to initiate all the work to get the product build started?

    >
    > The idea really is that the Internal part is idempotent, whilst the
    > External bit generates paperwork. Developers use the same Internal part
    > over and over, the External bit only gets fired up for real Builds that
    > are going out to QA or customers.
    >
    > In fact my developers _don't_ use the Internal build. They tend to use
    > Eclipse and let Eclipse handle things its own way. This sucks when they
    > then hand over a completed project with its unused and ignored
    > buildfile in tatters. If another one does it at 5pm after telling the
    > customer "it's ready now", then there'll be blood on the carpet 8-(


    Since CruiseControl requires that the external build works, wouldn't
    you know then who is suspect when someone commits a tattered project?

    >
    > We're also heavily into CurseControl and Fitnesse. Those have a
    > dependency on Ant we could work round, but haven't felt any urge to.
    >
    > Ant sucks for its lack of string handling. This is so poor that I don't
    > even like using Ant with a command line, as it's impractical to
    > validate command line parameters. Ant is good in that it understands
    > dependencies, which is slightly more use than a chocolate teapot. In
    > all this mess though, I only have one place where I actually use
    > dependencies.


    Exactly, this is my problem which led me to put up this post.
    >
    > Ant is probably more use for deployment, where a simple initial choice
    > (particularly which application server is in use) implies a whole chain
    > of other tasks. I'm also using it as a SQL client tool during
    > installation, because I know it's available, I know JDBC is already set
    > up, it doesn't care whether it's connected to Oracle or SQL Server and
    > it even does dependencies between successive patch scripts.
    >
    > I'd probably use Ant again, if I started from scratch. -- But I'm
    > having a hard time to really think _why_.
    >
    > I'm impressed with Python though. Six months ago I wrote bad Perl, then
    > I inherited this bundle mostly as shell scripts. I've gone from nowhere
    > on Python to devotee in just that time.


    I am going to move towards Groovy instead of Python. Groovy has some
    documentation (http://groovy.codehaus.org/Groovy Ant Task) that can
    integrate with ant as far as building. Since I know Ruby, I think
    Groovy would be a nice fit. I appreciate you taking time to tell me
    about your background and your building techniques.
    Danno, Oct 4, 2006
    #10
  11. Danno

    Danno Guest

    Simon Brooke wrote:
    > in message <>, Danno
    > ('') wrote:
    >
    > > I just want to know you're experience. Do you use both ant and
    > > scripting? Pure script builds? Do you use Groovy or Ruby?

    >
    > Most of us have ditched scripting (i.e. makefiles) for ant; ant is a very,
    > very good solution. Ant is, of course, a scripting approach with a
    > XML-syntax scripting language, but its flexibility and extensibility make
    > it extremely effective.
    >
    > --
    > (Simon Brooke) http://www.jasmine.org.uk/~simon/
    >
    > <p>Schroedinger's cat is <blink><strong>NOT</strong></blink> dead.</p>


    I've used ant for years, I just have more complicated scenarios that I
    need to use a scripting language (e.g. Groovy, Ruby, JavaScript, Perl).
    That being said, I have to heartily disagree that make, ant, and XML
    are scripting languages. Make and ant are tools, that use scripts, but
    that doesn't mean those scripts are scripting languages. XML is a
    markup language (hence the ML in XML) , so XML is not a scripting
    language. I do agree that make is much better than ant though.
    Danno, Oct 4, 2006
    #11
  12. Danno

    Chris Stiles Guest

    "Danno" <> writes:
    > are scripting languages. Make and ant are tools, that use scripts, but
    > that doesn't mean those scripts are scripting languages.


    Ant is a batch file interpreter that understands XML and has flow control via
    implicit gotos (task dependancies).

    --
    regards, chris
    Chris Stiles, Oct 4, 2006
    #12
  13. Danno

    Andy Dingley Guest

    Danno wrote:

    > Make and ant are tools, that use scripts, but
    > that doesn't mean those scripts are scripting languages.


    Ant is a scripting language. Make one target, stick a sequence of tasks
    in it. Use <antcall> as a subroutine mechanism. You've got yourself a
    scripting language.

    It's not a _good_ scripting language, but it's still better than
    Windows offers.

    The question is whether Ant's advantage of _also_ (and separately)
    being a dependency-flattening langauge is sufficient to excuse the poor
    scripting features, when our application of it is generally more
    dependent on scripting than anything else.
    Andy Dingley, Oct 4, 2006
    #13
  14. Danno

    Andy Dingley Guest

    Danno wrote:

    > Wow, you can have a full time job out of it?


    1/3rd getting builds out

    1/3rd improving the build process, writing scripts to reduce #1,
    telling developers where to stick their code.

    1/3rd chasing the developers who didn't follow #2 with my clueiron
    http://codesmiths.com/shed/things/clueiron/

    1/3rd generally being sworn at over the phone


    > So you have people request a product build in Bugzilla, and you use
    > that to initiate all the work to get the product build started?


    Yes. You also use Bugzilla to request your PC to be fixed, or to
    recruit a new developer.


    > Since CruiseControl requires that the external build works, wouldn't
    > you know then who is suspect when someone commits a tattered project?


    Usually, but not always. The usual culprit is brand new work, where it
    hasn't yet been added to our Fitnesse tests. Even when it has been
    "added", it needs an active inspection by me to ensure that everything
    is as it should be. The process isn't watertight, particularly when
    timescales are pressured.

    The "Agile" malarkey is great, but it's not very agile. Works a _lot_
    better when you're doing incremental work on an established project,
    rather than creating new projects.


    > I am going to move towards Groovy instead of Python.


    I considered that too, but it seemed too much like my previous
    experience of embedding Java snippets inside my Ant, when what I really
    needed was a wrapper around it.
    Andy Dingley, Oct 4, 2006
    #14
  15. Danno

    Danno Guest

    Andy Dingley wrote:
    > Danno wrote:
    >
    > > Make and ant are tools, that use scripts, but
    > > that doesn't mean those scripts are scripting languages.

    >
    > Ant is a scripting language. Make one target, stick a sequence of tasks
    > in it. Use <antcall> as a subroutine mechanism. You've got yourself a
    > scripting language.


    Whew, I'd have to strain to see that.

    >
    > It's not a _good_ scripting language, but it's still better than
    > Windows offers.
    >
    > The question is whether Ant's advantage of _also_ (and separately)
    > being a dependency-flattening langauge is sufficient to excuse the poor
    > scripting features, when our application of it is generally more
    > dependent on scripting than anything else.
    Danno, Oct 4, 2006
    #15
  16. Andy Dingley wrote:
    > Danno wrote:
    >
    >> Wow, you can have a full time job out of it?

    >
    > 1/3rd getting builds out
    >
    > 1/3rd improving the build process, writing scripts to reduce #1,
    > telling developers where to stick their code.
    >
    > 1/3rd chasing the developers who didn't follow #2 with my clueiron
    > http://codesmiths.com/shed/things/clueiron/
    >
    > 1/3rd generally being sworn at over the phone


    And another 1/3rd to finish your math degree along the way, right? :)

    /Thomas

    --
    The comp.lang.java.gui FAQ:
    http://gd.tuwien.ac.at/faqs/faqs-hierarchy/comp/comp.lang.java.gui/
    ftp://ftp.cs.uu.nl/pub/NEWS.ANSWERS/computer-lang/java/gui/faq
    Thomas Weidenfeller, Oct 5, 2006
    #16
  17. Danno

    Guest

    Gordon Beaton wrote:
    > On Tue, 03 Oct 2006 22:36:17 +0100, Simon Brooke wrote:
    > > scripting (i.e. makefiles)

    > When did these become equivalent?

    Agree with this question! Only bad makefiles looked like scripts. Good
    makefiles used the dependency facility which is declarative. However,
    most makefiles I have seen became highly procedureal because the
    maintainers did not understand the declarative approach of make.

    Kind regards,

    Peter Kriens
    , Oct 5, 2006
    #17
  18. Danno

    Andy Dingley Guest

    Thomas Weidenfeller wrote:

    > And another 1/3rd to finish your math degree along the way, right? :)


    If three thirds add up to 1, that's arithmetic.

    If four thirds add up to 1, that's maths.

    If an uncertain number of thirds definitely add up to 1, but you can't
    tell how many, that's crypto.
    Andy Dingley, Oct 5, 2006
    #18
  19. Sounds like the 90/10 rule of software

    The first 90% of the job takes 90% of the time.
    The remaining 10% of the job takes the other 90% of the time.

    Andy Dingley wrote:
    > Thomas Weidenfeller wrote:
    >
    >> And another 1/3rd to finish your math degree along the way, right? :)

    >
    > If three thirds add up to 1, that's arithmetic.
    >
    > If four thirds add up to 1, that's maths.
    >
    > If an uncertain number of thirds definitely add up to 1, but you can't
    > tell how many, that's crypto.
    >
    David Zimmerman, Oct 5, 2006
    #19
  20. Danno

    Chris Uppal Guest

    David Zimmerman wrote:

    > The first 90% of the job takes 90% of the time.
    > The remaining 10% of the job takes the other 90% of the time.


    Nice...

    Up there with "n+1 trivial tasks will take the same time as n trivial tasks".

    -- chris
    Chris Uppal, Oct 6, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nick
    Replies:
    0
    Views:
    458
  2. Evan
    Replies:
    6
    Views:
    344
    bjarne
    Dec 23, 2006
  3. Damon Getsman
    Replies:
    2
    Views:
    403
    Chris Riesbeck
    Sep 17, 2008
  4. Gianni Mariani
    Replies:
    1
    Views:
    315
    Ian Collins
    Oct 28, 2008
  5. Gregory Brown
    Replies:
    0
    Views:
    108
    Gregory Brown
    Apr 15, 2010
Loading...

Share This Page