javac vs Makefile problem ...

Discussion in 'Java' started by for.fun@laposte.net, Nov 30, 2005.

  1. Guest

    Hi everybody,


    I actually have the following problem :

    I have a Java/JNI application which mixes ".h", ".cpp", ".java" (native
    and regular files)
    In order to compile everything easily, I did a Makefile.

    I know that "javac" resolves the dependencies but because I have to
    generate ".h" files from ".class", I had to include the Java
    compilation in my Makefile.

    To be clearer, here is my compilation chain :

    javac javah CC
    X.java => X.class => X.h => X.o
    X.cpp


    All this chain is achieved thanks to my Makefile.
    It works but not as well as I expect it to.


    My problem is :

    1/ Consider that "Y.java" depends on "X.java"

    2/ If "Y.java" is compiled later, it involves "X.java" to be compiled
    again.

    3/ Consequently, the modify time of "X.java" is changed so all the long
    job starting from "X.class" is done again and my Makefile takes a while
    ....


    Is there a way to disable the "javac" automatic dependency resolution
    in order to completely manage it in a Makefile ?
    Do you have another issue which could help me ?


    Thanks in advance.
     
    , Nov 30, 2005
    #1
    1. Advertising

  2. Adam Maass Guest

    <> wrote:
    > Hi everybody,
    >
    >
    > I actually have the following problem :
    >
    > I have a Java/JNI application which mixes ".h", ".cpp", ".java" (native
    > and regular files)
    > In order to compile everything easily, I did a Makefile.
    >
    > I know that "javac" resolves the dependencies but because I have to
    > generate ".h" files from ".class", I had to include the Java
    > compilation in my Makefile.
    >
    > To be clearer, here is my compilation chain :
    >
    > javac javah CC
    > X.java => X.class => X.h => X.o
    > X.cpp
    >
    >
    > All this chain is achieved thanks to my Makefile.
    > It works but not as well as I expect it to.
    >
    >
    > My problem is :
    >
    > 1/ Consider that "Y.java" depends on "X.java"
    >
    > 2/ If "Y.java" is compiled later, it involves "X.java" to be compiled
    > again.
    >
    > 3/ Consequently, the modify time of "X.java" is changed so all the long
    > job starting from "X.class" is done again and my Makefile takes a while
    > ...
    >
    >
    > Is there a way to disable the "javac" automatic dependency resolution
    > in order to completely manage it in a Makefile ?
    > Do you have another issue which could help me ?
    >


    There is no way to disable the javac automatic dependency resolution.

    However, there is a way to fix your build script so that it ends up doing
    what you want:

    You don't want 1 execution of javac for each .java file in your project;
    instead, what you want to do is invoke javac once on *all* .java files in
    your project -- all at one go. Do this first, then do the javah and
    subsequent steps.

    If all of your .java files live in one directory (or are contained in
    subdirectories of one directory), your life is much simplified. Examine
    the -sourcepath option to javac.
     
    Adam Maass, Nov 30, 2005
    #2
    1. Advertising

  3. Roedy Green Guest

    On 30 Nov 2005 10:32:12 -0800, wrote, quoted or
    indirectly quoted someone who said :

    >Is there a way to disable the "javac" automatic dependency resolution
    >in order to completely manage it in a Makefile ?
    >Do you have another issue which could help me ?


    Almost no one use a makefile anymore. They use ANT to handle this,
    which solves the problem in a platform-independent way.

    You periodically do a clean compile (recompile the universe). Other
    than than that, A depending on B won't recompile B. If B has changed
    then A might get recompiled, but there are situations where A should
    be recompiled, but is not. I don't know of any situation where you
    get spurious recompiles.

    The more common problem is failing to rebuild all jars that contain A
    when A changes.

    see http://mindprod.com/jgloss/ant.html
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
     
    Roedy Green, Nov 30, 2005
    #3
  4. Roedy Green Guest

    On Wed, 30 Nov 2005 13:39:47 -0800, "Adam Maass"
    <> wrote, quoted or indirectly quoted
    someone who said :

    >You don't want 1 execution of javac for each .java file in your project;
    >instead, what you want to do is invoke javac once on *all* .java files in
    >your project -- all at one go. Do this first, then do the javah and
    >subsequent steps.


    ANT is clever enough to invoke JavaC only once no matter how
    complicated your build script. It is at least a order of magnitude
    faster than invoking Javac for each *.java file.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
     
    Roedy Green, Nov 30, 2005
    #4
  5. Adam Maass wrote:
    > <> wrote:
    >
    >>Hi everybody,
    >>
    >>
    >>I actually have the following problem :
    >>
    >>I have a Java/JNI application which mixes ".h", ".cpp", ".java" (native
    >>and regular files)
    >>In order to compile everything easily, I did a Makefile.
    >>
    >>I know that "javac" resolves the dependencies but because I have to
    >>generate ".h" files from ".class", I had to include the Java
    >>compilation in my Makefile.
    >>
    >>To be clearer, here is my compilation chain :
    >>
    >> javac javah CC
    >>X.java => X.class => X.h => X.o
    >> X.cpp
    >>
    >>
    >>All this chain is achieved thanks to my Makefile.
    >>It works but not as well as I expect it to.
    >>
    >>
    >>My problem is :
    >>
    >>1/ Consider that "Y.java" depends on "X.java"
    >>
    >>2/ If "Y.java" is compiled later, it involves "X.java" to be compiled
    >>again.
    >>
    >>3/ Consequently, the modify time of "X.java" is changed so all the long
    >>job starting from "X.class" is done again and my Makefile takes a while
    >>...
    >>
    >>
    >>Is there a way to disable the "javac" automatic dependency resolution
    >>in order to completely manage it in a Makefile ?
    >>Do you have another issue which could help me ?
    >>

    >
    >
    > There is no way to disable the javac automatic dependency resolution.
    >
    > However, there is a way to fix your build script so that it ends up doing
    > what you want:
    >
    > You don't want 1 execution of javac for each .java file in your project;
    > instead, what you want to do is invoke javac once on *all* .java files in
    > your project -- all at one go. Do this first, then do the javah and
    > subsequent steps.
    >
    > If all of your .java files live in one directory (or are contained in
    > subdirectories of one directory), your life is much simplified. Examine
    > the -sourcepath option to javac.


    OP: Of course Adam's solution is probably optimal. Just out of
    curiousity, why is the modification time of X.java changing? Compiling
    a source file should not change the source's modification time.
     
    Jeffrey Schwab, Nov 30, 2005
    #5
  6. Guest

    Jeffrey Schwab a écrit :

    > >>3/ Consequently, the modify time of "X.java" is changed so all the long
    > >>job starting from "X.class" is done again and my Makefile takes a while


    > OP: Of course Adam's solution is probably optimal. Just out of
    > curiousity, why is the modification time of X.java changing? Compiling
    > a source file should not change the source's modification time.


    Of course, you are right : the source Java modify time will not change.
    I did a write mistake.
    In facts, I meant this : 'the modify time of "X.class" is changed' and
    so on ...

    I had a quick look to ANT.
    I understand your point of view but I am not going to use it because I
    think that the Unix shell commands are powerfull and I can not work
    without them.
    Moreover, it is no problem installing Win32 ports of the Unix shell
    commands (direct port or using Cygwin) on any system so you can run
    make everywhere.
    I just did it did and it works fine on Solaris and Windows XP.

    The Adam Maass will work but I will not gain any performance : each
    time I will touch a Java source file, everything is recompiled. I work
    on a very big project and recompiling anything each time will take
    hours ...
     
    , Dec 1, 2005
    #6
  7. Guest

    Today, I spent some time looking for some other Java tools and I
    discovered the Jikes Java compiler which is said to be compatible with
    the JVM.
    Moreover, Jikes allows to manually manage dependencies by generating
    dependencies Makefile.

    I am going to try this ...


    Adam Maass a écrit :

    > <> wrote:
    > > Hi everybody,
    > >
    > >
    > > I actually have the following problem :
    > >
    > > I have a Java/JNI application which mixes ".h", ".cpp", ".java" (native
    > > and regular files)
    > > In order to compile everything easily, I did a Makefile.
    > >
    > > I know that "javac" resolves the dependencies but because I have to
    > > generate ".h" files from ".class", I had to include the Java
    > > compilation in my Makefile.
    > >
    > > To be clearer, here is my compilation chain :
    > >
    > > javac javah CC
    > > X.java => X.class => X.h => X.o
    > > X.cpp
    > >
    > >
    > > All this chain is achieved thanks to my Makefile.
    > > It works but not as well as I expect it to.
    > >
    > >
    > > My problem is :
    > >
    > > 1/ Consider that "Y.java" depends on "X.java"
    > >
    > > 2/ If "Y.java" is compiled later, it involves "X.java" to be compiled
    > > again.
    > >
    > > 3/ Consequently, the modify time of "X.java" is changed so all the long
    > > job starting from "X.class" is done again and my Makefile takes a while
    > > ...
    > >
    > >
    > > Is there a way to disable the "javac" automatic dependency resolution
    > > in order to completely manage it in a Makefile ?
    > > Do you have another issue which could help me ?
    > >

    >
    > There is no way to disable the javac automatic dependency resolution.
    >
    > However, there is a way to fix your build script so that it ends up doing
    > what you want:
    >
    > You don't want 1 execution of javac for each .java file in your project;
    > instead, what you want to do is invoke javac once on *all* .java files in
    > your project -- all at one go. Do this first, then do the javah and
    > subsequent steps.
    >
    > If all of your .java files live in one directory (or are contained in
    > subdirectories of one directory), your life is much simplified. Examine
    > the -sourcepath option to javac.
     
    , Dec 1, 2005
    #7
  8. Adam Maass Guest

    "Roedy Green" <> wrote:
    >
    > Almost no one use a makefile anymore. They use ANT [instead].
    >


    Not true. My current project is a combination of Ant and make; Ant exec's
    make for the native portions of the project. There's good reason for
    makefiles if there's no portion of the project that's Java.

    If the project is pure Java (or nearly so), then I can see using Ant as
    *the* build executible. Otherwise, makefiles have a lot going for them.



    -- Adam Maass
     
    Adam Maass, Dec 2, 2005
    #8
  9. Adam Maass Guest

    <> wrote:

    > To be clearer, here is my compilation chain :


    > javac javah CC
    > X.java => X.class => X.h => X.o
    > X.cpp



    I suggested compiling all of the .java files to .class files at one go,
    using the -sourcepath argument to javac.


    > The Adam Maass will work but I will not gain any performance : each
    > time I will touch a Java source file, everything is recompiled. I work
    > on a very big project and recompiling anything each time will take
    > hours ...



    OP: try setting the dependency your X.h and X.cpp files not on X.class, but
    X.java instead. That should alleviate the "recompile the world when anything
    changes" problem.

    -- Adam Maass
     
    Adam Maass, Dec 2, 2005
    #9
  10. Guest

    Adam Maass a écrit :

    > OP: try setting the dependency your X.h and X.cpp files not on X.class, but
    > X.java instead. That should alleviate the "recompile the world when anything
    > changes" problem.



    Good idea. I am going to try it.
    Thx Adam.
     
    , Dec 2, 2005
    #10
  11. Guest

    Adam Maass a écrit :

    > OP: try setting the dependency your X.h and X.cpp files not on X.class, but
    > X.java instead. That should alleviate the "recompile the world when anything
    > changes" problem.


    I tried your issue but it leads to another problem.
    When I add the dependency "%.h : %.java", "make" must know how to make
    ".h" from ".java".

    To make ".h" from ".java", you must :

    1/ make the ".class" with "javac"
    2/ make the ".h" with "javah"

    As you said, the best way to do it is to compile all the world with
    "javac".
    With your issue, I have to recompile all the world with "javac" each
    time I have a ".h" to generate.


    I found an issue which works even if I am not self-satisfied !
    Anyway, I gained a lot of time doing like this :

    Instead of relying on built-in "make" processus based on modified file
    times, I implemented a CRC-based processus.
    I the Java class CRC change after a "javac" then I make the ".h"
    otherwise I do nothing.

    This works fine because I noticed that most of time "javac" regenerates
    the same classes binaries.

    "javac" is not really an optimized compiler !

    => Why recompiling "n" times the same classes ?
     
    , Dec 5, 2005
    #11
  12. On 5 Dec 2005 08:39:23 -0800, wrote:
    > I tried your issue but it leads to another problem. When I add the
    > dependency "%.h : %.java", "make" must know how to make ".h" from
    > ".java".


    But you don't make the header from the java source, you make it from
    the class. In fact that's what you say here:

    > To make ".h" from ".java", you must :
    >
    > 1/ make the ".class" with "javac"
    > 2/ make the ".h" with "javah"
    >


    The dependency rule should be: "%.h: %.class" instead.

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, Dec 6, 2005
    #12
  13. Guest

    Gordon Beaton a écrit :

    > On 5 Dec 2005 08:39:23 -0800, wrote:


    > The dependency rule should be: "%.h: %.class" instead.


    Yes, I know but my first problem was due to "javac" which recompiles
    any Java class even if the Java source did not change. Indeed, all
    headers are recompiled and nearly all my project was rebuilt.
    I work on a huge project so rebuilding everything take a while and the
    purpose of "make" was to avoid that kind of situation.

    That is why Adam suggested me to depend on ".java" instead of ".class"
    in order to avoid the full rebuild.

    I am not really happy with "javac" which works for nothing !
     
    , Dec 6, 2005
    #13
  14. wrote:
    > Yes, I know but my first problem was due to "javac" which recompiles
    > any Java class even if the Java source did not change.


    You should really check if you don't have a second version of (older)
    ..class files in the classpath.

    javac uses the directory as specified with -d to store compiled .class
    files, however, it does not automatically search in that directory when
    looking for type information. Instead, it uses the classpath, and, when
    defined, the sourcepath to search for classes and source code. If the
    directory specified with -d is not in the classpath (having it there
    first is a very good idea), then you are in trouble. Particular, because
    the default for the classpath is the current directory.

    So, if you happen to have old .class files in the classpath (current
    directory) first, javac thinks the corresponding source needs
    recompilation. But instead of replacing the old *.class files, it writes
    the compilation result to the -d directory. The old *.class files remain
    untouched in this case, and next time you compile javac once again
    thinks it needs to recompile them.

    In make parlance, using something like

    OUTPUT_DIR = classes
    CLASSPATH = $(OUTPUT_DIR):<rest of classpath>
    JFLAGS = -d $(OUTPUT_DIR) -cp $(CLASSPATH) <more options>

    etc. is a very good idea.


    > Indeed, all
    > headers are recompiled and nearly all my project was rebuilt.
    > I work on a huge project so rebuilding everything take a while and the
    > purpose of "make" was to avoid that kind of situation.


    make is still a fine tool, and it is very much possible to use it to
    build java applications. make's biggest problem with java is that most
    versions of make don't allow to specify cyclic dependencies.
    Unfortunately, these are common in java, where class A refers to class
    B, and class B to class A. If you have such a normal make, you need to
    group java source files into sets, where each set only has cyclic
    dependencies within the set, and each set is compiled with a single
    invocation of javac. The dependencies among sets should form a DAG,
    which is what make expects for dependencies.

    Some time ago I considered to write a special make for Java, which
    allows to specify cyclic dependencies, and does the grouping into sets
    of cyclic dependent source code internally. However, the atrocity called
    ant was already gaining rapid popularity, and I didn't see any change of
    widespread usage of such a special make.

    /Thomas
    --
    The comp.lang.java.gui FAQ:
    ftp://ftp.cs.uu.nl/pub/NEWS.ANSWERS/computer-lang/java/gui/faq
    http://www.uni-giessen.de/faq/archiv/computer-lang.java.gui.faq/
     
    Thomas Weidenfeller, Dec 6, 2005
    #14
  15. Guest

    Thomas Weidenfeller a écrit :

    First of all, thanks for all your detailed explanations.


    > In make parlance, using something like
    >
    > OUTPUT_DIR = classes
    > CLASSPATH = $(OUTPUT_DIR):<rest of classpath>
    > JFLAGS = -d $(OUTPUT_DIR) -cp $(CLASSPATH) <more options>
    >
    > etc. is a very good idea.


    In facts, that is what I do in my Makefile, I generate the ".class" in
    an output directory (specified by "-d") and include that dir into my
    classpath. I double-checked and I am sure that I have no older version
    of my ".class" in my directories.


    Here is an example of what happens to me (in the make context) :

    1/ I have the class A which depends on B, C, D

    2/ When A change, "make" will call "javac" which will recompile A
    Unfornutately, "javac" also recompile B, C, D.

    3/ Consequently, the JNI headers are rebuilt for A but also for B, C, D
    while it was not necessary.


    > make is still a fine tool, and it is very much possible to use it to
    > build java applications. make's biggest problem with java is that most
    > versions of make don't allow to specify cyclic dependencies.


    I did not know that. I think GNU make does not allow cyclic
    dependencies because I can see plenty of "Avoiding implicit rule
    recursion." in the verbose make logs.
    .... and I suppose that a cyclic dependency can be interpreted as a
    recursion by GNU make.


    > Some time ago I considered to write a special make for Java, which
    > allows to specify cyclic dependencies, and does the grouping into sets
    > of cyclic dependent source code internally. However, the atrocity called
    > ant was already gaining rapid popularity, and I didn't see any change of
    > widespread usage of such a special make.


    I was advised to use Ant by people on this forum but after a quick
    look, it does not seem as powerful as make (but I may be wrong)

    And I like "make" too much (especially "GNU make")
     
    , Dec 6, 2005
    #15
  16. wrote:
    > Thomas Weidenfeller a écrit :
    >
    > First of all, thanks for all your detailed explanations.
    >
    >
    >
    >>In make parlance, using something like
    >>
    >> OUTPUT_DIR = classes
    >> CLASSPATH = $(OUTPUT_DIR):<rest of classpath>
    >> JFLAGS = -d $(OUTPUT_DIR) -cp $(CLASSPATH) <more options>
    >>
    >>etc. is a very good idea.

    >
    >
    > In facts, that is what I do in my Makefile, I generate the ".class" in
    > an output directory (specified by "-d") and include that dir into my
    > classpath. I double-checked and I am sure that I have no older version
    > of my ".class" in my directories.
    >
    >
    > Here is an example of what happens to me (in the make context) :
    >
    > 1/ I have the class A which depends on B, C, D
    >
    > 2/ When A change, "make" will call "javac" which will recompile A
    > Unfornutately, "javac" also recompile B, C, D.


    It only should do this when the found class files are older than the
    source file, or it couldn't find the class files at all. Check your
    system time, check the time on the files, and you could also check which
    files javac evaluates by e.g. running javac with the -verbose option. If
    you don't trust javac's verbose output you could also run it under the
    control of truss ("truss -f -t open -t stat javac ..." should do) to see
    what it really does.

    > I did not know that. I think GNU make does not allow cyclic
    > dependencies because I can see plenty of "Avoiding implicit rule
    > recursion."


    Oh no, that indicates that something is wrong with the way you have set
    up your implicit rules. It does not indicate that you declared a
    circular dependency among source files. Something in your pattern rules,
    your suffix rules, or the .SUFFIXES list is broken, seriously broken.

    That would also be a possible explanation for your continuous
    recompilation: You simple provided make with a broken rule set.

    Circular dependencies in GNU make should give you a warning like
    "circular <some dependency> dropped". Other makes simply stop with an
    error message. But even if you use GNU make, you should fix the Makefile
    to get rid of the circular dependency specifications, and instead group
    your source in the previously mentioned sets.

    /Thomas
    --
    The comp.lang.java.gui FAQ:
    ftp://ftp.cs.uu.nl/pub/NEWS.ANSWERS/computer-lang/java/gui/faq
    http://www.uni-giessen.de/faq/archiv/computer-lang.java.gui.faq/
     
    Thomas Weidenfeller, Dec 6, 2005
    #16
  17. Guest

    Thomas Weidenfeller a écrit :

    > That would also be a possible explanation for your continuous
    > recompilation: You simple provided make with a broken rule set.


    You are probably right.
    I am going to look this way.

    Thanks a lot for all your usefull advices.
     
    , Dec 6, 2005
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. z3
    Replies:
    0
    Views:
    509
  2. Collin VanDyck
    Replies:
    2
    Views:
    662
    Collin VanDyck
    Nov 18, 2004
  3. Salil P
    Replies:
    2
    Views:
    1,706
    Andrew Thompson
    Jan 22, 2005
  4. drunken_wizard
    Replies:
    1
    Views:
    11,461
    Tris Orendorff
    Mar 23, 2006
  5. java
    Replies:
    6
    Views:
    6,722
    Juha Laiho
    Aug 19, 2007
Loading...

Share This Page