SCons build tool speed

Discussion in 'Python' started by ted, Feb 12, 2005.

  1. ted

    ted Guest

    How does the speed of the Scons build tool compare with Ant? Right now with
    out Ant builds take around an hour. Hoping to speed that up.

    TIA,
    Ted
    ted, Feb 12, 2005
    #1
    1. Advertising

  2. On Sat, Feb 12, 2005 at 07:16:02PM +0000, ted wrote:
    > How does the speed of the Scons build tool compare with Ant?


    I would recommend asking this question on , but my
    impressions is that most of the time is probably spent in the compiler. If you
    are working on a java project you could try switching from javac to jikes and
    that might improve your time, though it has been a while since I used jikes.

    Chris
    Christopher De Vries, Feb 13, 2005
    #2
    1. Advertising

  3. ted

    Peter Hansen Guest

    ted wrote:
    > How does the speed of the Scons build tool compare with Ant? Right now with
    > out Ant builds take around an hour. Hoping to speed that up.


    Don't tools like Scons, Ant, and for that matter "make" just
    execute other programs? So that 99% of the time is consumed
    external to the Scons, Ant, or make process itself? Why
    do you think the speed of the build tool is of any significance
    compared to the time for things like compilers and linkers
    to execute?

    -Peter
    Peter Hansen, Feb 13, 2005
    #3
  4. ted

    Mike Meyer Guest

    Peter Hansen <> writes:

    > ted wrote:
    >> How does the speed of the Scons build tool compare with Ant? Right
    >> now with out Ant builds take around an hour. Hoping to speed that up.

    >
    > Don't tools like Scons, Ant, and for that matter "make" just
    > execute other programs? So that 99% of the time is consumed
    > external to the Scons, Ant, or make process itself? Why
    > do you think the speed of the build tool is of any significance
    > compared to the time for things like compilers and linkers
    > to execute?


    Actually, these tools do a number of things oter than run other
    programs. For instance, they parse input files to construct a build
    tree, walk the build tree figuring out what needs to be rebuilt, and
    only *then* do they get around to executing those other
    programs. Assuming that the work prior to running those other programs
    is "neglible" can lead to some truly atrocious build-time behavior. In
    fact, one major advantage that ant/scons/jam/etc. have over Make is
    that they avoid the defacto standard "recursive make" build
    system. See <URL:
    http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html > for
    details on that.

    <mike
    --
    Mike Meyer <> http://www.mired.org/home/mwm/
    Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
    Mike Meyer, Feb 13, 2005
    #4
  5. ted

    Neil Hodgson Guest

    ted:

    > How does the speed of the Scons build tool compare with Ant?
    > Right now with out Ant builds take around an hour. Hoping to
    > speed that up.


    Scons emphasises accuracy over speed and is normally a little slower than
    other build tools although still fast enough for most purposes. One cause of
    slowness is that it reads the source files, traces include files and
    checksums them all rather than relying on file times. There are some things
    you can do to speed it up:
    http://www.scons.org/cgi-bin/wiki/GoFastButton

    Neil
    Neil Hodgson, Feb 13, 2005
    #5
  6. ted

    PA Guest

    PA, Feb 13, 2005
    #6
  7. ted

    Guest

    Hi Neil, ted, et al.--

    > > How does the speed of the Scons build tool compare with Ant?
    > > Right now with out Ant builds take around an hour. Hoping to
    > > speed that up.

    >
    > Scons emphasises accuracy over speed and is normally a little
    > slower than other build tools although still fast enough for most
    > purposes. One cause of slowness is that it reads the source files,
    > traces include files and checksums them all rather than relying
    > on file times. There are some things you can do to speed it up:
    > http://www.scons.org/cgi-bin/wiki/GoFastButton


    In anticipation of the next release of SCons, we've been doing
    a *lot* of work on profiling the performance and eliminating
    bottlenecks.

    It turns out that scanning the source files and performing the MD5
    calculation is not the dominant factor that we've all assumed it is.
    There were some other inefficiencies in some of our algorithms that
    were much more significant, including some unnecessary disk scans,
    recomputing the same dependencies for every target in a list of
    targets generated by one command, and repeated just-in-time creation
    of some internal objects that could be created just once and cached.

    That said, SCons will never be as fast as Make, because it *is*
    doing more for you under the covers. Although the next version
    won't necessarily speed up every configuration (the bottlenecks
    are extremely configuration dependent), it should be a significant
    improvement for many configurations out there.

    --SK
    , Feb 13, 2005
    #7
  8. ted

    Guest

    Hi Peter--

    > > How does the speed of the Scons build tool compare with
    > > Ant? Right now with out Ant builds take around an hour. Hoping
    > > to speed that up.

    >
    > Don't tools like Scons, Ant, and for that matter "make" just
    > execute other programs? So that 99% of the time is consumed
    > external to the Scons, Ant, or make process itself? Why
    > do you think the speed of the build tool is of any significance
    > compared to the time for things like compilers and linkers
    > to execute?


    You're right that build times are dominated by the external commands
    when rebuilds occur. If nothing needs to be rebuilt, though, the
    wall-clock time is obviously dominated by how long it takes for
    the build tool to decide that. A build tool that doesn't decide
    whether or not targets are up-to-date *and* dispatch the commnds
    to rebuild them reasonably quickly and efficiently doesn't do a
    good job of serving its users.

    --SK
    , Feb 13, 2005
    #8
  9. ted

    Peter Hansen Guest

    wrote:
    > You're right that build times are dominated by the external commands
    > when rebuilds occur. If nothing needs to be rebuilt, though, the
    > wall-clock time is obviously dominated by how long it takes for
    > the build tool to decide that. A build tool that doesn't decide
    > whether or not targets are up-to-date *and* dispatch the commnds
    > to rebuild them reasonably quickly and efficiently doesn't do a
    > good job of serving its users.


    That answer, combined with Mike's response pointing out
    that tools more sophisticated than basic "make" actually
    can delve into the source and identify the dependencies,
    removes my confusion. Thanks. :)

    -Peter
    Peter Hansen, Feb 14, 2005
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. efiedler
    Replies:
    1
    Views:
    2,015
    Tim Ward
    Oct 9, 2003
  2. Lothar Werzinger
    Replies:
    1
    Views:
    501
    fuzzylollipop
    Dec 27, 2004
  3. Steven Knight
    Replies:
    0
    Views:
    361
    Steven Knight
    Oct 23, 2003
  4. Replies:
    3
    Views:
    315
    Steve Christensen
    Apr 19, 2005
  5. Ruby Script
    Replies:
    0
    Views:
    127
    Ruby Script
    Aug 21, 2004
Loading...

Share This Page