The curse of constant fields

Discussion in 'Java' started by Juha Laiho, Dec 12, 2008.

  1. Juha Laiho

    Juha Laiho Guest

    I got recently bitten by something I consider to be a rather nasty
    feature in the Java language, and decided to write up about it.

    I had to do a couple of simple changes to a relatively shaky and
    not-so-testable codebase, so I chose to do the absolute minimum
    required: change initialization values for a couple of String fields
    in a class, compile the changed class, and run the code manually to
    observe that I had achieved the desired change.

    What I didn't pay attention to initially was that the strings were
    declared final. So, I changed the values, compiled the changed
    class and replaced old classfile with new. Time for the test run,
    and what I see is that the program still shows the original values.

    A quick search through the source code shows that there is no other
    component in the program containing a copy of these values except
    the one I already changed - so, I'm starting to worry whether even
    the version control for the code is out of synch with the actual
    compiled code. This feeling did strenghten quite a lot when I found
    out that another class (which directly uses the values of the fields
    I changed) contains the old values of the fields. So, has there been
    some other version of the _other_ class that at some point did
    itself contain the fields that now need to be changed? I rummage
    around, but do not find any such version of the source code for
    the offending class. At a whim, I recompile the offending class
    from source code fresh out of the version control, to observe any
    possible differences in the compilation results. The results seem
    pretty much similar, except that now the newly compiled class file
    contains the changed value for the fields - which were declared
    and initialized IN A JAVA SOURCE FILE FOR ANOTHER CLASS! (apologies
    for raising my voice)

    So, problem solved and programmer confused. Off I go to take a look
    at the JLS, and find the explanation there in chapter 13.4.9:
    : If a field is a constant variable (§4.12.4), then deleting the
    : keyword final or changing its value will not break compatibility
    : with pre-existing binaries by causing them not to run, but they
    : will not see any new value for the usage of the field unless they
    : are recompiled. This is true even if the usage itself is not
    : a compile-time constant expression (§15.28)

    (with other things hinting to this same behaviour in a number of
    places, among others in ch. 3.10.5)

    In a word, ouch. A lesson learned.
    --
    Wolf a.k.a. Juha Laiho Espoo, Finland
    (GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V
    PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
    "...cancel my subscription to the resurrection!" (Jim Morrison)
     
    Juha Laiho, Dec 12, 2008
    #1
    1. Advertising

  2. Juha Laiho

    Arne Vajhøj Guest

    Patricia Shanahan wrote:
    > Juha Laiho wrote:
    >> I got recently bitten by something I consider to be a rather nasty
    >> feature in the Java language, and decided to write up about it.
    >>
    >> I had to do a couple of simple changes to a relatively shaky and
    >> not-so-testable codebase, so I chose to do the absolute minimum
    >> required: change initialization values for a couple of String fields
    >> in a class, compile the changed class, and run the code manually to
    >> observe that I had achieved the desired change.

    > ...
    >
    > You have my sympathy for your frustration, but I would call this story
    > "The curse of manual builds".
    >
    > Back in the 1970's, long before Java was invented, I worked on a large
    > project that avoided reassembling all the code by having a team of
    > programmer techs who constructed the list of modules that needed to be
    > assembled due to each change. The programmer techs did their best, but
    > it was a *very* error prone procedure.
    >
    > Since then, I have seen complaints, for several languages, of the form
    > "I did a manual build, and got into trouble due to [some aspect of the
    > programming language or environment]".
    >
    > I am convinced that the only viable choices are rebuilding everything
    > for every change or using language-appropriate automatic dependency
    > analysis.


    And tools like ant make it easy.

    Arne
     
    Arne Vajhøj, Dec 12, 2008
    #2
    1. Advertising

  3. Juha Laiho

    Roedy Green Guest

    On Fri, 12 Dec 2008 21:32:04 GMT, Juha Laiho <>
    wrote, quoted or indirectly quoted someone who said :

    >
    >In a word, ouch. A lesson learned.


    Rule of thumb, any time you change the values of any non-private
    constants, do a clean compile.

    I got nailed with that myself the other day, with build numbers that
    failed to increment.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    PM Steven Harper is fixated on the costs of implementing Kyoto, estimated as high as 1% of GDP.
    However, he refuses to consider the costs of not implementing Kyoto which the
    famous economist Nicholas Stern estimated at 5 to 20% of GDP
     
    Roedy Green, Dec 13, 2008
    #3
  4. Arne Vajhøj wrote:
    > Patricia Shanahan wrote:
    >> Juha Laiho wrote:
    >>> I got recently bitten by something I consider to be a rather nasty
    >>> feature in the Java language, and decided to write up about it.
    >>>
    >>> I had to do a couple of simple changes to a relatively shaky and
    >>> not-so-testable codebase, so I chose to do the absolute minimum
    >>> required: change initialization values for a couple of String
    >>> fields
    >>> in a class, compile the changed class, and run the code manually
    >>> to
    >>> observe that I had achieved the desired change.

    >> ...
    >>
    >> You have my sympathy for your frustration, but I would call this
    >> story "The curse of manual builds".
    >>
    >> Back in the 1970's, long before Java was invented, I worked on a
    >> large project that avoided reassembling all the code by having a
    >> team of programmer techs who constructed the list of modules that
    >> needed to be assembled due to each change. The programmer techs did
    >> their best, but it was a *very* error prone procedure.
    >>
    >> Since then, I have seen complaints, for several languages, of the
    >> form "I did a manual build, and got into trouble due to [some
    >> aspect
    >> of the programming language or environment]".
    >>
    >> I am convinced that the only viable choices are rebuilding
    >> everything
    >> for every change or using language-appropriate automatic dependency
    >> analysis.

    >
    > And tools like ant make it easy.


    Easy to get it wrong, I think you mean. Ant's dependency analysis
    amounts to:

    Compile A.java if A.class is either absent or older than A.java.

    Ant will not recompile A.java on the grounds that it uses contants
    from class B and B.java is newer than A.class. When accepting a set
    of changes from elsewhere (say, when updating from an SCM system), the
    only safe procedure is to do a full, clean rebuild.
     
    Mike Schilling, Dec 13, 2008
    #4
  5. Mike Schilling wrote:
    > Ant will not recompile A.java on the grounds that it uses contants
    > from class B and B.java is newer than A.class. When accepting a set
    > of changes from elsewhere (say, when updating from an SCM system), the
    > only safe procedure is to do a full, clean rebuild.


    Shouldn't running the ant build twice suffice?

    First time, all the .java files that define constants that have changed
    will get recompiled.

    Second time, all the .java files that use constants that have changed,
    and weren't recompiled in the first pass, will get recompiled because
    they now have a dependency whose .class is newer.

    Ex:

    A.java uses constants in B.java last mod Dec 12 A.class last mod Dec 12
    B.java last mod Dec 13 B.class last mod Dec 12
    C.java uses constants in B.java last mod Dec 12 C.class last mod Dec 12

    Pass 1:

    A.class younger than A.java and dependency B.class, leave A.java alone.
    B.class older than B.java. Recompile B.java. B.class now modified Dec 13
    C.class younger than C.java but older than B.class. Recompile C.java.

    A.java uses constants in B.java last mod Dec 12 A.class last mod Dec 12
    B.java last mod Dec 13 B.class last mod Dec 13
    C.java uses constants in B.java last mod Dec 12 C.class last mod Dec 13

    Pass 2:

    A.class younger than A.java but older than B.class. Recompile A.java.
    B.class younger than B.java. Leave B.java alone.
    C.class younger than C.java and dependency B.class. Leave C.java alone.

    A.java uses constants in B.java last mod Dec 12 A.class last mod Dec 13
    B.java last mod Dec 13 B.class last mod Dec 13
    C.java uses constants in B.java last mod Dec 12 C.class last mod Dec 13

    A and C should both be up to date after the second pass, although A
    wasn't after the first pass.

    Or does ant get things wrong in a more complicated way than you
    originally stated?
     
    Wesley MacIntosh, Dec 13, 2008
    #5
  6. Wesley MacIntosh wrote:
    > Mike Schilling wrote:
    >> Ant will not recompile A.java on the grounds that it uses contants
    >> from class B and B.java is newer than A.class. When accepting a set
    >> of changes from elsewhere (say, when updating from an SCM system), the
    >> only safe procedure is to do a full, clean rebuild.

    >
    > Shouldn't running the ant build twice suffice?
    >


    No because class files that logically depend on a constant defined
    elsewhere do not include any reference to that dependency. To do this
    dependency analysis reliably you have to analyse the source yourself
    with a tool other than javac. You can not do it just from the .class files.

    Mark Thornton
     
    Mark Thornton, Dec 13, 2008
    #6
  7. Juha Laiho

    Arne Vajhøj Guest

    Mike Schilling wrote:
    > Arne Vajhøj wrote:
    >> Patricia Shanahan wrote:
    >>> Juha Laiho wrote:
    >>>> I got recently bitten by something I consider to be a rather nasty
    >>>> feature in the Java language, and decided to write up about it.
    >>>>
    >>>> I had to do a couple of simple changes to a relatively shaky and
    >>>> not-so-testable codebase, so I chose to do the absolute minimum
    >>>> required: change initialization values for a couple of String
    >>>> fields
    >>>> in a class, compile the changed class, and run the code manually
    >>>> to
    >>>> observe that I had achieved the desired change.
    >>> ...
    >>>
    >>> You have my sympathy for your frustration, but I would call this
    >>> story "The curse of manual builds".
    >>>
    >>> Back in the 1970's, long before Java was invented, I worked on a
    >>> large project that avoided reassembling all the code by having a
    >>> team of programmer techs who constructed the list of modules that
    >>> needed to be assembled due to each change. The programmer techs did
    >>> their best, but it was a *very* error prone procedure.
    >>>
    >>> Since then, I have seen complaints, for several languages, of the
    >>> form "I did a manual build, and got into trouble due to [some
    >>> aspect
    >>> of the programming language or environment]".
    >>>
    >>> I am convinced that the only viable choices are rebuilding
    >>> everything
    >>> for every change or using language-appropriate automatic dependency
    >>> analysis.

    >> And tools like ant make it easy.

    >
    > Easy to get it wrong, I think you mean. Ant's dependency analysis
    > amounts to:
    >
    > Compile A.java if A.class is either absent or older than A.java.
    >
    > Ant will not recompile A.java on the grounds that it uses contants
    > from class B and B.java is newer than A.class. When accepting a set
    > of changes from elsewhere (say, when updating from an SCM system), the
    > only safe procedure is to do a full, clean rebuild.


    I meant that ant make rebuilding everything easy.

    Clean all dirs, extract from VCS, compile src->build, pack jars.

    Arne
     
    Arne Vajhøj, Dec 13, 2008
    #7
  8. Juha Laiho

    Arne Vajhøj Guest

    Wesley MacIntosh wrote:
    > Mike Schilling wrote:
    >> Ant will not recompile A.java on the grounds that it uses contants
    >> from class B and B.java is newer than A.class. When accepting a set
    >> of changes from elsewhere (say, when updating from an SCM system), the
    >> only safe procedure is to do a full, clean rebuild.

    >
    > Shouldn't running the ant build twice suffice?
    >
    > First time, all the .java files that define constants that have changed
    > will get recompiled.
    >
    > Second time, all the .java files that use constants that have changed,
    > and weren't recompiled in the first pass, will get recompiled because
    > they now have a dependency whose .class is newer.
    >
    > Ex:
    >
    > A.java uses constants in B.java last mod Dec 12 A.class last mod Dec 12
    > B.java last mod Dec 13 B.class last mod Dec 12
    > C.java uses constants in B.java last mod Dec 12 C.class last mod Dec 12
    >
    > Pass 1:
    >
    > A.class younger than A.java and dependency B.class, leave A.java alone.
    > B.class older than B.java. Recompile B.java. B.class now modified Dec 13
    > C.class younger than C.java but older than B.class. Recompile C.java.
    >
    > A.java uses constants in B.java last mod Dec 12 A.class last mod Dec 12
    > B.java last mod Dec 13 B.class last mod Dec 13
    > C.java uses constants in B.java last mod Dec 12 C.class last mod Dec 13
    >
    > Pass 2:
    >
    > A.class younger than A.java but older than B.class. Recompile A.java.
    > B.class younger than B.java. Leave B.java alone.
    > C.class younger than C.java and dependency B.class. Leave C.java alone.
    >
    > A.java uses constants in B.java last mod Dec 12 A.class last mod Dec 13
    > B.java last mod Dec 13 B.class last mod Dec 13
    > C.java uses constants in B.java last mod Dec 12 C.class last mod Dec 13
    >
    > A and C should both be up to date after the second pass, although A
    > wasn't after the first pass.
    >
    > Or does ant get things wrong in a more complicated way than you
    > originally stated?


    His statement:

    >> Ant will not recompile A.java on the grounds that it uses contants
    >> from class B and B.java is newer than A.class.


    and yours:

    > A.class younger than A.java but older than B.class. Recompile A.java.


    are different.

    Arne
     
    Arne Vajhøj, Dec 13, 2008
    #8
  9. Arne Vajhøj wrote:
    > Mike Schilling wrote:
    >> Arne Vajhøj wrote:
    >>> Patricia Shanahan wrote:
    >>>> Juha Laiho wrote:
    >>>>> I got recently bitten by something I consider to be a rather
    >>>>> nasty
    >>>>> feature in the Java language, and decided to write up about it.
    >>>>>
    >>>>> I had to do a couple of simple changes to a relatively shaky and
    >>>>> not-so-testable codebase, so I chose to do the absolute minimum
    >>>>> required: change initialization values for a couple of String
    >>>>> fields
    >>>>> in a class, compile the changed class, and run the code manually
    >>>>> to
    >>>>> observe that I had achieved the desired change.
    >>>> ...
    >>>>
    >>>> You have my sympathy for your frustration, but I would call this
    >>>> story "The curse of manual builds".
    >>>>
    >>>> Back in the 1970's, long before Java was invented, I worked on a
    >>>> large project that avoided reassembling all the code by having a
    >>>> team of programmer techs who constructed the list of modules that
    >>>> needed to be assembled due to each change. The programmer techs
    >>>> did
    >>>> their best, but it was a *very* error prone procedure.
    >>>>
    >>>> Since then, I have seen complaints, for several languages, of the
    >>>> form "I did a manual build, and got into trouble due to [some
    >>>> aspect
    >>>> of the programming language or environment]".
    >>>>
    >>>> I am convinced that the only viable choices are rebuilding
    >>>> everything
    >>>> for every change or using language-appropriate automatic
    >>>> dependency
    >>>> analysis.
    >>> And tools like ant make it easy.

    >>
    >> Easy to get it wrong, I think you mean. Ant's dependency analysis
    >> amounts to:
    >>
    >> Compile A.java if A.class is either absent or older than A.java.
    >>
    >> Ant will not recompile A.java on the grounds that it uses contants
    >> from class B and B.java is newer than A.class. When accepting a
    >> set
    >> of changes from elsewhere (say, when updating from an SCM system),
    >> the only safe procedure is to do a full, clean rebuild.

    >
    > I meant that ant make rebuilding everything easy.
    >
    > Clean all dirs, extract from VCS, compile src->build, pack jars.


    That's quite true, though of course it would be true of any scripting
    language. Back in C and C++ days, there were fairly simple tools to
    optimize builds: makemake scripts that analyzed source code to build
    dependency lists and generate make scripts, so you really could do a
    build that was both safe and minimal. Java (in particular, the fact
    that a .java file is the equivalent of both .c and .h files) makes
    this astonishingly difficult.
     
    Mike Schilling, Dec 13, 2008
    #9
  10. Juha Laiho

    Arne Vajhøj Guest

    Mike Schilling wrote:
    > Arne Vajhøj wrote:
    >> Mike Schilling wrote:
    >>> Arne Vajhøj wrote:
    >>>> Patricia Shanahan wrote:
    >>>>> Juha Laiho wrote:
    >>>>>> I got recently bitten by something I consider to be a rather
    >>>>>> nasty
    >>>>>> feature in the Java language, and decided to write up about it.
    >>>>>>
    >>>>>> I had to do a couple of simple changes to a relatively shaky and
    >>>>>> not-so-testable codebase, so I chose to do the absolute minimum
    >>>>>> required: change initialization values for a couple of String
    >>>>>> fields
    >>>>>> in a class, compile the changed class, and run the code manually
    >>>>>> to
    >>>>>> observe that I had achieved the desired change.
    >>>>> ...
    >>>>>
    >>>>> You have my sympathy for your frustration, but I would call this
    >>>>> story "The curse of manual builds".
    >>>>>
    >>>>> Back in the 1970's, long before Java was invented, I worked on a
    >>>>> large project that avoided reassembling all the code by having a
    >>>>> team of programmer techs who constructed the list of modules that
    >>>>> needed to be assembled due to each change. The programmer techs
    >>>>> did
    >>>>> their best, but it was a *very* error prone procedure.
    >>>>>
    >>>>> Since then, I have seen complaints, for several languages, of the
    >>>>> form "I did a manual build, and got into trouble due to [some
    >>>>> aspect
    >>>>> of the programming language or environment]".
    >>>>>
    >>>>> I am convinced that the only viable choices are rebuilding
    >>>>> everything
    >>>>> for every change or using language-appropriate automatic
    >>>>> dependency
    >>>>> analysis.
    >>>> And tools like ant make it easy.
    >>> Easy to get it wrong, I think you mean. Ant's dependency analysis
    >>> amounts to:
    >>>
    >>> Compile A.java if A.class is either absent or older than A.java.
    >>>
    >>> Ant will not recompile A.java on the grounds that it uses contants
    >>> from class B and B.java is newer than A.class. When accepting a
    >>> set
    >>> of changes from elsewhere (say, when updating from an SCM system),
    >>> the only safe procedure is to do a full, clean rebuild.

    >> I meant that ant make rebuilding everything easy.
    >>
    >> Clean all dirs, extract from VCS, compile src->build, pack jars.

    >
    > That's quite true, though of course it would be true of any scripting
    > language. Back in C and C++ days, there were fairly simple tools to
    > optimize builds: makemake scripts that analyzed source code to build
    > dependency lists and generate make scripts, so you really could do a
    > build that was both safe and minimal. Java (in particular, the fact
    > that a .java file is the equivalent of both .c and .h files) makes
    > this astonishingly difficult.


    Huge C/C++ build often took a long time, because of the include thing,
    optimizing by the compiler and because systems were slower back then.

    Java doesn't use include, javac doesn't optimize and the systems are
    fast.

    I don't see full rebuild as a problem.

    Arne
     
    Arne Vajhøj, Dec 13, 2008
    #10
  11. On Sat, 13 Dec 2008 13:38:02 -0500, Arne Vajhøj wrote:

    > Mike Schilling wrote:
    >> Arne Vajhøj wrote:
    >>> Mike Schilling wrote:
    >>>> Arne Vajhøj wrote:
    >>>>> Patricia Shanahan wrote:
    >>>>>> Juha Laiho wrote:
    >>>>>>> I got recently bitten by something I consider to be a rather nasty
    >>>>>>> feature in the Java language, and decided to write up about it.
    >>>>>>>
    >>>>>>> I had to do a couple of simple changes to a relatively shaky and
    >>>>>>> not-so-testable codebase, so I chose to do the absolute minimum
    >>>>>>> required: change initialization values for a couple of String
    >>>>>>> fields
    >>>>>>> in a class, compile the changed class, and run the code manually
    >>>>>>> to
    >>>>>>> observe that I had achieved the desired change.
    >>>>>> ...
    >>>>>>
    >>>>>> You have my sympathy for your frustration, but I would call this
    >>>>>> story "The curse of manual builds".
    >>>>>>
    >>>>>> Back in the 1970's, long before Java was invented, I worked on a
    >>>>>> large project that avoided reassembling all the code by having a
    >>>>>> team of programmer techs who constructed the list of modules that
    >>>>>> needed to be assembled due to each change. The programmer techs did
    >>>>>> their best, but it was a *very* error prone procedure.
    >>>>>>
    >>>>>> Since then, I have seen complaints, for several languages, of the
    >>>>>> form "I did a manual build, and got into trouble due to [some
    >>>>>> aspect
    >>>>>> of the programming language or environment]".
    >>>>>>
    >>>>>> I am convinced that the only viable choices are rebuilding
    >>>>>> everything
    >>>>>> for every change or using language-appropriate automatic dependency
    >>>>>> analysis.
    >>>>> And tools like ant make it easy.
    >>>> Easy to get it wrong, I think you mean. Ant's dependency analysis
    >>>> amounts to:
    >>>>
    >>>> Compile A.java if A.class is either absent or older than A.java.
    >>>>
    >>>> Ant will not recompile A.java on the grounds that it uses contants
    >>>> from class B and B.java is newer than A.class. When accepting a set
    >>>> of changes from elsewhere (say, when updating from an SCM system),
    >>>> the only safe procedure is to do a full, clean rebuild.
    >>> I meant that ant make rebuilding everything easy.
    >>>
    >>> Clean all dirs, extract from VCS, compile src->build, pack jars.

    >>
    >> That's quite true, though of course it would be true of any scripting
    >> language. Back in C and C++ days, there were fairly simple tools to
    >> optimize builds: makemake scripts that analyzed source code to build
    >> dependency lists and generate make scripts, so you really could do a
    >> build that was both safe and minimal. Java (in particular, the fact
    >> that a .java file is the equivalent of both .c and .h files) makes this
    >> astonishingly difficult.

    >
    > Huge C/C++ build often took a long time, because of the include thing,
    > optimizing by the compiler and because systems were slower back then.
    >
    > Java doesn't use include, javac doesn't optimize and the systems are
    > fast.
    >

    'make' doesn't take kindly to circular references, while Java often seems
    to end up with something remarkably similar - is this why javac seems to
    prefer simultaneous compiles of all source files in a package?

    I've tried to use 'make' for Java but it doesn't take kindly to Java. I
    ended up leaving dependencies out of the makefile to stop 'make'
    complaining about circular references. This in turn meant that the only
    way I could guarantee a complete, clean build was to force a full build
    by deleting the class files. Once I hit this situation I stopped trying
    to use 'make' for Java and switched to ant: in any case even a full ant
    build is faster than a 'make' build thanks to ant taking advantage of
    javac's parallel compile capability, but I still periodically find I have
    to force a complete build.

    'make' only works well for C compilation and other tasks where its
    possible to define a straight dependency tree with no crosslinks.

    > I don't see full rebuild as a problem.
    >

    Agreed, but its a bit irritating. Do you preempt it by writing the ant
    script to delete all class files before it runs javac?


    --
    martin@ | Martin Gregorie
    gregorie. | Essex, UK
    org |
     
    Martin Gregorie, Dec 13, 2008
    #11
  12. Martin Gregorie wrote:
    > On Sat, 13 Dec 2008 13:38:02 -0500, Arne Vajhøj wrote:
    >> Mike Schilling wrote:
    >>> That's quite true, though of course it would be true of any scripting
    >>> language. Back in C and C++ days, there were fairly simple tools to
    >>> optimize builds: makemake scripts that analyzed source code to build
    >>> dependency lists and generate make scripts, so you really could do a
    >>> build that was both safe and minimal. Java (in particular, the fact
    >>> that a .java file is the equivalent of both .c and .h files) makes this
    >>> astonishingly difficult.

    >> Huge C/C++ build often took a long time, because of the include thing,
    >> optimizing by the compiler and because systems were slower back then.
    >>
    >> Java doesn't use include, javac doesn't optimize and the systems are
    >> fast.
    >>

    > 'make' doesn't take kindly to circular references, while Java often seems
    > to end up with something remarkably similar - is this why javac seems to
    > prefer simultaneous compiles of all source files in a package?


    I think so.

    > I've tried to use 'make' for Java but it doesn't take kindly to Java. I
    > ended up leaving dependencies out of the makefile to stop 'make'
    > complaining about circular references. This in turn meant that the only
    > way I could guarantee a complete, clean build was to force a full build
    > by deleting the class files. Once I hit this situation I stopped trying
    > to use 'make' for Java and switched to ant: in any case even a full ant
    > build is faster than a 'make' build thanks to ant taking advantage of
    > javac's parallel compile capability, but I still periodically find I have
    > to force a complete build.


    It is usually best to use the build tool matching the technology. Ant or
    maven for Java. Make for C/C++. Nant or MSBuild for .NET.

    >> I don't see full rebuild as a problem.
    >>

    > Agreed, but its a bit irritating. Do you preempt it by writing the ant
    > script to delete all class files before it runs javac?


    I believe it is very common to have a clean target.

    Arne
     
    Arne Vajhøj, Dec 13, 2008
    #12
  13. Juha Laiho

    Tom Anderson Guest

    On Sat, 13 Dec 2008, Mike Schilling wrote:

    > Back in C and C++ days, there were fairly simple tools to optimize
    > builds: makemake scripts that analyzed source code to build dependency
    > lists and generate make scripts, so you really could do a build that was
    > both safe and minimal. Java (in particular, the fact that a .java file
    > is the equivalent of both .c and .h files) makes this astonishingly
    > difficult.


    Really?

    I was thinking about this a while ago, and it seemed to me that the
    dependencies a class had were moderately simple - every other class it
    references directly, and every class which is a superclass of a class it
    references. Am i missing something?

    Further (and i'm rambling now), in all cases (i *think*), a class only
    depends on the interface of the other class, so if you can work out if a
    class's interface has changed (by comparing the fresh class file with the
    old one, for instance), you can work out if a change is one that means
    dependents have to be recompiled. If you also captured information about
    whether the dependency was on public, package-access or protected
    features, you could possibly also prune your list of recompilations
    further. In the limit, you could track dependencies on individual
    features, and be very selective about what you have to recompile. Although
    there are gotchas around inheritance - eg adding a new protected final
    method to a class breaks subclasses which define a method of the same
    name.

    Eclipse has an incremental compiler, which has been released as a
    standalone tool. I wonder if this might be a solution, or the foundation
    for a solution, to this problem.

    tom

    --
    For me, thats just logic. OTOH, Spock went bananas several times using
    logic. -- Pete, mfw
     
    Tom Anderson, Dec 13, 2008
    #13
  14. On Sat, 13 Dec 2008 16:01:43 -0500, Arne Vajhøj wrote:

    >
    > I believe it is very common to have a clean target.
    >

    Yes, and I do have one. Of course.

    What I was wondering was if anybody has ever found it beneficial to chain
    a step to delete all class files before the compile step. This could be
    because the structure of the class hierarchy, due to externally visible
    constants or whatever, was complex and/or tangled enough to need a full
    compile more often than not.


    --
    martin@ | Martin Gregorie
    gregorie. | Essex, UK
    org |
     
    Martin Gregorie, Dec 13, 2008
    #14
  15. Juha Laiho

    Arne Vajhøj Guest

    Tom Anderson wrote:
    > On Sat, 13 Dec 2008, Mike Schilling wrote:
    >
    >> Back in C and C++ days, there were fairly simple tools to optimize
    >> builds: makemake scripts that analyzed source code to build dependency
    >> lists and generate make scripts, so you really could do a build that
    >> was both safe and minimal. Java (in particular, the fact that a .java
    >> file is the equivalent of both .c and .h files) makes this
    >> astonishingly difficult.

    >
    > Really?
    >
    > I was thinking about this a while ago, and it seemed to me that the
    > dependencies a class had were moderately simple - every other class it
    > references directly, and every class which is a superclass of a class it
    > references. Am i missing something?
    >
    > Further (and i'm rambling now), in all cases (i *think*), a class only
    > depends on the interface of the other class, so if you can work out if a
    > class's interface has changed (by comparing the fresh class file with
    > the old one, for instance), you can work out if a change is one that
    > means dependents have to be recompiled. If you also captured information
    > about whether the dependency was on public, package-access or protected
    > features, you could possibly also prune your list of recompilations
    > further. In the limit, you could track dependencies on individual
    > features, and be very selective about what you have to recompile.
    > Although there are gotchas around inheritance - eg adding a new
    > protected final method to a class breaks subclasses which define a
    > method of the same name.


    I don't think it is simple.

    As far as I can see, then:
    * it is not possible only using class files - source code analysis is
    needed
    * it is not possible only with current code - both current and previous
    code is necessary

    Classpath for source code is not a well defined concept.

    And a database with info for different versions adds work.

    Arne
     
    Arne Vajhøj, Dec 13, 2008
    #15
  16. Martin Gregorie wrote:
    > On Sat, 13 Dec 2008 16:01:43 -0500, Arne Vajhøj wrote:
    >> I believe it is very common to have a clean target.
    >>

    > Yes, and I do have one. Of course.
    >
    > What I was wondering was if anybody has ever found it beneficial to chain
    > a step to delete all class files before the compile step. This could be
    > because the structure of the class hierarchy, due to externally visible
    > constants or whatever, was complex and/or tangled enough to need a full
    > compile more often than not.


    I would expect almost everyone to do clean before an official
    build.

    It is only with developer builds that safe versus quick
    is an issue.

    I must admit that in development I don't always clean as
    part of build.

    Arne
     
    Arne Vajhøj, Dec 13, 2008
    #16
  17. Arne Vajhøj wrote:
    > Martin Gregorie wrote:
    >> On Sat, 13 Dec 2008 16:01:43 -0500, Arne Vajhøj wrote:
    >>> I believe it is very common to have a clean target.
    >>>

    >> Yes, and I do have one. Of course.
    >>
    >> What I was wondering was if anybody has ever found it beneficial to
    >> chain a step to delete all class files before the compile step.
    >> This
    >> could be because the structure of the class hierarchy, due to
    >> externally visible constants or whatever, was complex and/or
    >> tangled
    >> enough to need a full compile more often than not.

    >
    > I would expect almost everyone to do clean before an official
    > build.


    Me too. We run build/test cycles nightly, and they always starts
    with:

    Clean the world
    Update all the source from the SCM system
    Build (etc.)

    The clean build eats up only machine resources. Figuring out whether
    a failure is a real failure or an anomaly that would be fixed by a
    clean build uses far more valuable human time.

    >
    > It is only with developer builds that safe versus quick
    > is an issue.
    >
    > I must admit that in development I don't always clean as
    > part of build.


    Me too, though usually if I don't clean, it's because I made all the
    changes myself and know that a non-clean build is safe. (Or think I
    know it, anyway).
     
    Mike Schilling, Dec 14, 2008
    #17
  18. Tom Anderson wrote:
    > On Sat, 13 Dec 2008, Mike Schilling wrote:
    >
    >> Back in C and C++ days, there were fairly simple tools to optimize
    >> builds: makemake scripts that analyzed source code to build
    >> dependency lists and generate make scripts, so you really could do
    >> a
    >> build that was both safe and minimal. Java (in particular, the
    >> fact
    >> that a .java file is the equivalent of both .c and .h files) makes
    >> this astonishingly difficult.

    >
    > Really?



    >
    > I was thinking about this a while ago, and it seemed to me that the
    > dependencies a class had were moderately simple - every other class
    > it
    > references directly, and every class which is a superclass of a
    > class
    > it references. Am i missing something?
    >
    > Further (and i'm rambling now), in all cases (i *think*), a class
    > only
    > depends on the interface of the other class,


    And the values of their constants, which is where we came in.

    > so if you can work out
    > if a class's interface has changed (by comparing the fresh class
    > file
    > with the old one, for instance), you can work out if a change is one
    > that means dependents have to be recompiled.


    And if there were any changes to that class's ancestor classes, or any
    interfaces it implements (directly or indirectly.)

    > If you also captured
    > information about whether the dependency was on public,
    > package-access or protected features, you could possibly also prune
    > your list of recompilations further. In the limit, you could track
    > dependencies on individual features, and be very selective about
    > what
    > you have to recompile. Although there are gotchas around inheritance
    > - eg adding a new protected final method to a class breaks
    > subclasses
    > which define a method of the same name.


    All of this is feasible, but it's complicated. Once you've figured
    out in detail all of the possible sorts of changes you'd like to track
    (and that's complex in itself), you've now got to make the right
    tradeoff between. And you'd like to build this all into the compiler,
    rather than having to run successive compilation passes, as each one
    invalidates a new set of classes.

    1. Keeping very detailed dependencies, which minimizes the number of
    recompilations, but explodes the size of your dependency database, and
    2. Keeping coarser dependencies, which minimized the size of your
    dependency database, but increases the number of recompilations

    Doing comparisons between file dates is a lot simpler.
     
    Mike Schilling, Dec 14, 2008
    #18
  19. Arne Vajhøj wrote:
    >
    > I don't think it is simple.
    >
    > As far as I can see, then:
    > * it is not possible only using class files - source code analysis
    > is
    > needed


    I think everything you need is in the class file. What's missing?

    > * it is not possible only with current code - both current and
    > previous code is necessary


    Yes, you'd need a database of "stuff I saw last time I compiled
    X.java". (And, of course, if that's missing, you'd compile X.java
    regardless.)
     
    Mike Schilling, Dec 14, 2008
    #19
  20. Juha Laiho

    Arne Vajhøj Guest

    Mike Schilling wrote:
    > Arne Vajhøj wrote:
    >> I don't think it is simple.
    >>
    >> As far as I can see, then:
    >> * it is not possible only using class files - source code analysis
    >> is
    >> needed

    >
    > I think everything you need is in the class file. What's missing?


    What this thread started with.

    If the class file contains 123, then you don't know that it really
    was mypack.MyClass.ConstX ...

    >> * it is not possible only with current code - both current and
    >> previous code is necessary

    >
    > Yes, you'd need a database of "stuff I saw last time I compiled
    > X.java". (And, of course, if that's missing, you'd compile X.java
    > regardless.)


    Arne
     
    Arne Vajhøj, Dec 14, 2008
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Brian Alexander

    update of curse panel in background?

    Brian Alexander, Jul 21, 2003, in forum: Python
    Replies:
    0
    Views:
    288
    Brian Alexander
    Jul 21, 2003
  2. Adrian Smith

    CGI python use "under a curse"

    Adrian Smith, May 7, 2007, in forum: Python
    Replies:
    3
    Views:
    399
    Steve Holden
    May 7, 2007
  3. XLiIV
    Replies:
    4
    Views:
    366
    Terry Reedy
    May 9, 2008
  4. Pascal Chambon
    Replies:
    8
    Views:
    503
    yam850
    Jul 3, 2009
  5. Paul Battley
    Replies:
    3
    Views:
    122
    Paul Battley
    Aug 2, 2006
Loading...

Share This Page