The program that would'nt Quit

Discussion in 'Java' started by Roedy Green, Jan 26, 2006.

  1. Roedy Green

    Roedy Green Guest

    I have I think discovered a strange bug in both JDK 1.5 and 1.6.

    the program ends like this:

    System.out.println( "done" );
    // System.exit( 0 );
    } // end main

    Without that System.exit, the program keeps on running endlessly using
    about 94% of my CPU time after printing "done".

    The odd thing is this is not a gui and it has but a single thread. I
    have no finalizers.

    It works fine in Jet.

    I don't know where to begin to track this down or how to reproduce it
    for Sun. The jar is 271K and the data files it works on are 14 MB.

    Any ideas on what could cause such behaviour?

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #1
    1. Advertising

  2. Roedy Green

    Roedy Green Guest

    On Thu, 26 Jan 2006 02:34:42 GMT, Roedy Green
    <> wrote, quoted or
    indirectly quoted someone who said :

    >the program ends like this:
    >
    > System.out.println( "done" );
    > // System.exit( 0 );
    > } // end main


    there in the class containing main. I don't see anything out of the
    ordinary about it.

    package com.mindprod.htmlmacros;
    import java.io.File;
    import java.io.FilenameFilter;
    import java.util.ArrayList;

    import com.mindprod.filter.ExtensionListFilter;

    /**
    * Feeds the entire Mindprod website to the Replacer macro mechanism
    refreshing
    * expansions of all macros. Expands HTML boilerplate refresher macros
    e.g. <!--
    * macro Include dogs.txt --> <!-- generated --> <img
    * src="images/animals/dog.jpg" height="410" width="511" alt="dog">
    <!--
    * /generated by Include --> Allows arbitrary expander macro code.
    Futures:
    * Error messages give offset/line number, nearby text check for
    generated by
    * macro not just generated to remove old text. Can embed a " in a
    parm by
    * encoding it as &quot;
    */
    public class ReplaceMindprod extends Replacer {

    /**
    * True if want debugging output, counts of files gathered from
    different
    * dirs.
    */
    private static final boolean DEBUGGING = false;

    /**
    * total count of files gathered
    */
    private static int gatherCount;

    /**
    * arrayList where we build list of files to process.
    */
    static ArrayList<String> filesToProcess;

    /**
    * Estimated files will process.
    */
    public static final int ESTIMATED_TOTAL_FILES = 4500;

    /**
    * Collect all files in the given dirName for processing.
    *
    * @param dirName
    * Name of fully qualified directory where to look for the
    html
    * files.
    * @param filter
    * filter to use to decide while files in that dir to
    process.
    */
    static void gather ( String dirName, FilenameFilter filter )
    {
    String[] filenames = new File( dirName ).list( filter );
    if ( filenames == null )
    {
    throw new IllegalArgumentException(
    "com.mindprod.htmlmacros.ReplaceMindprod: No such dir:
    "
    + dirName );
    }
    gatherCount += filenames.length;
    if ( DEBUGGING )
    {
    System.out.println( filenames.length
    + " files gathered from "
    + dirName );
    }
    for ( String filename : filenames )
    {
    filesToProcess.add( dirName + "/" + filename );
    }
    }

    /**
    * Get fully qualified name of all the files to process
    */
    public static void gatherFiles ()
    {

    // add *.html files in mindprod

    filesToProcess = new ArrayList<String>( ESTIMATED_TOTAL_FILES
    );
    FilenameFilter f = new ExtensionListFilter( new String[] {
    "html"
    } );

    gather( "e:/mindprod", f );
    gather( "e:/mindprod/animalrights", f );
    gather( "e:/mindprod/applets", f );
    gather( "e:/mindprod/applications", f );
    gather( "e:/mindprod/bgloss", f );
    // bookcovers
    gather( "e:/mindprod/contact", f );
    gather( "e:/mindprod/deepthoughts", f );
    // dvdcovers
    gather( "e:/mindprod/environment", f );
    gather( "e:/mindprod/esperanto", f );
    gather( "e:/mindprod/ethics", f );
    gather( "e:/mindprod/feedback", f );
    gather( "e:/mindprod/ggloss", f );
    gather( "e:/mindprod/humanrights", f );
    // icon
    // images
    // include won't work since macros in wrong dir
    gather( "e:/mindprod/jgloss", f );
    // jgloss/include
    // kjv, rarely changes, do manually when it needs it.
    gather( "e:/mindprod/livinglove", f );
    gather( "e:/mindprod/livinglove/methods", f );
    gather( "e:/mindprod/money", f );
    // pads
    gather( "e:/mindprod/phone", f );
    gather( "e:/mindprod/politics", f );
    gather( "e:/mindprod/politics/laser", f );
    // precis
    gather( "e:/mindprod/projects", f );
    gather( "e:/mindprod/quadra", f );

    // redir, only makes sense to expand them in place. Don't
    disturb copied
    // versions
    gather( "e:/mindprod/redir", f );
    gather( "e:/mindprod/religion", f );
    // renney
    // replicator
    gather( "e:/mindprod/reunion", f );
    // screenshots
    // snippets
    // sound
    gather( "e:/mindprod/utilities", f );
    gather( "e:/mindprod/webstarts", f );
    // zips

    // all files are ready to go.

    }

    /**
    * Expands macros written in Java in HTML files files.
    *
    * @param args
    * names of files to process, no wildcards.
    */
    public static void main ( String[] args )
    {
    System.out.println( "\nExpanding macros on entire Mindprod
    website" );

    Tools.setWebRoot( "e:\\mindprod" );
    gatherFiles();

    System.out.println( gatherCount
    + " files gathered for mindprod site macro expansion." );

    assert gatherCount <= ESTIMATED_TOTAL_FILES;
    for ( String fileToProcess : filesToProcess )
    {
    assert fileToProcess != null;
    assert new File( fileToProcess ).exists() : "file to
    process missing";
    try
    {
    expandMacrosInFile( new File( fileToProcess ), true /*
    quiet */);
    }
    catch ( Exception e )
    {
    System.err.println( e.getMessage()
    + " in file "
    + fileToProcess );
    System.err.println();
    e.printStackTrace();
    // we don't exit. We keep on processing files.
    }
    } // end for each file loop
    System.out.println( "done" );
    // don't remove the System.exit( 0 ). It bypasses a Sun bug.
    System.exit( 0 );
    } // end main

    } // end ReplaceMindProd

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #2
    1. Advertising

  3. Roedy Green

    Stefan Ram Guest

    You could use Thread.activeCount() to confirm that there is
    only one thread at the end.

    >Tools.setWebRoot( "e:\\mindprod" ); (...)
    >gatherFiles(); (...)
    >for ( String fileToProcess : filesToProcess ) (...)
    >expandMacrosInFile( new File( fileToProcess )


    Comment out (hide) the call of "expandMacrosInFile" and see
    whether the behavior still occurs.

    If yes, remove "gatherFiles" and see whether ...

    If no: The behavior might have been cause by
    something in "expandMacrosInFile": So now, edit
    this method, commenting out its final half or so.

    Recursively repeat this until the cause is found.

    (This approach is not always possible or might not
    always work.)
    Stefan Ram, Jan 26, 2006
    #3
  4. Roedy Green

    Guest

    Seems like taking a javacore after the "done" message is printed would
    a)
    tell you what thread is active and what it is doing and b)
    provide the information Sun would need to fix it, if it is truly a bug.

    -Andrew
    , Jan 26, 2006
    #4
  5. Roedy Green wrote:
    >
    > Without that System.exit, the program keeps on running endlessly using
    > about 94% of my CPU time after printing "done".
    >
    > The odd thing is this is not a gui and it has but a single thread. I
    > have no finalizers.
    >
    > It works fine in Jet.
    >
    > I don't know where to begin to track this down or how to reproduce it
    > for Sun. The jar is 271K and the data files it works on are 14 MB.
    >
    > Any ideas on what could cause such behaviour?


    Have you tried using jstack (or similar) to see what is continuing to run?

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
    Thomas Hawtin, Jan 26, 2006
    #5
  6. Good day Roedy,

    I empathize with this situation; I share an aversion for System.exit .
    Some ideas:

    There are situations where this occurs according to definition in
    non-gui programs. For example when using a JMF player one needs to
    stop(), deallocate(), and close() before graceful exit. Perhaps you
    are inadvartanly and indirectly creating some event loop?
    ---
    I know one program that exits gracefully after about ten seconds. Have
    you waited this long? The general rule is if there is a thread that
    only wakes up every X seconds then you might wait upto X seconds for it
    to return from its run().
    ---
    I believe Java spec says that JVM can choose to exit if only demon
    threads areleft, but does not have to. I doubt this is the case
    because Sun's jvm always seems to take the exit choice within a couple
    of seconds.
    ---
    Perhaps you should view running threads using JConsole. You will also
    see each thread's state. There is a negative in that you will see
    threads related to JConsole listed; this negative is not that large.

    Hopefully this email is of some service

    All the best,
    Opalinski

    http://www.geocities.com/opalpaweb/


    P.S. Jconsole takes a total of maybe five minutes to learn. Run your
    executable with -Dcom.sun.management.jmxremote and seperately execute
    jconsole found within java's bin directory. That's it

    Here is some documenation:
    http://java.sun.com/j2se/1.5.0/docs/guide/management/jconsole.html
    opalinski from opalpaweb, Jan 26, 2006
    #6
  7. Roedy Green

    Roedy Green Guest

    On 26 Jan 2006 12:02:00 -0800, " opalinski from
    opalpaweb" <> wrote, quoted or indirectly quoted
    someone who said :

    >There are situations where this occurs according to definition in
    >non-gui programs. For example when using a JMF player one needs to
    >stop(), deallocate(), and close() before graceful exit. Perhaps you
    >are inadvartanly and indirectly creating some event loop?


    This is weird. I am doing pure batch text processing. There are no
    events or listeners in the whole program. I don't spawn any threads --
    any of the usual suspects in refusal to shut down.

    Keep going. I hope I have overlooked something.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #7
  8. Roedy Green

    Roedy Green Guest

    On 26 Jan 2006 12:02:00 -0800, " opalinski from
    opalpaweb" <> wrote, quoted or indirectly quoted
    someone who said :

    >I know one program that exits gracefully after about ten seconds. Have
    >you waited this long? The general rule is if there is a thread that
    >only wakes up every X seconds then you might wait upto X seconds for it
    >to return from its run().


    There are no timers, execs or threads. About the only exotic feature
    of this program is using Class.forName to load most of its classes.

    Perhaps some sort of incomplete i/o? I do some auxiliary disk
    processing and perhaps a failure is getting lost, maybe an unclosed
    file somewhere. Unfortunately Java has no open method you can grep.
    Also I don't know of a way of testing if and what files are still
    open.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #8
  9. Roedy Green

    Roedy Green Guest

    On 26 Jan 2006 12:02:00 -0800, " opalinski from
    opalpaweb" <> wrote, quoted or indirectly quoted
    someone who said :

    >Perhaps you should view running threads using JConsole. You will also
    >see each thread's state. There is a negative in that you will see
    >threads related to JConsole listed; this negative is not that large.


    I'm running it now It turns out to be ten times easier to set up than
    it first looks. I have documented this, as is my wont, at
    http://mindprod.com/jgloss/jconsole.html

    Surprises for me includes:

    1. my "single thread' app had 18 threads.

    3. garbage collections are extremely frequent. There over a 1000 of
    them in the time it took to connect and start looking. However
    JConsole said that only a few seconds were used in GC. I had assumed
    they were much less frequent and far more painful.

    4. the different aged collections of objects truly do behave
    differently. the older collections are rock stable. The eden has a two
    frequency periodicity, like a sound with harmonics.

    5. My app uses about 15 MB of heap space by default. I had never done
    much thinking about ram usage. It is a bit of a shock. In the 1970s I
    think I would have pulled off an app like this in 200K including the
    run time.

    6. There is a quantum mechanical effect. Watching the apps with
    JConsole makes it work!!
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #9
  10. Roedy Green

    Roedy Green Guest

    On 26 Jan 2006 08:02:06 -0800, wrote, quoted or
    indirectly quoted someone who said :

    >taking a javacore


    What class/method is that?
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #10
  11. Roedy Green

    Roedy Green Guest

    On Thu, 26 Jan 2006 20:01:04 +0000, Thomas Hawtin
    <> wrote, quoted or indirectly quoted someone
    who said :

    >
    >Have you tried using jstack

    I had never heard of it till now. Will try that.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #11
  12. Roedy Green

    Roedy Green Guest

    On Thu, 26 Jan 2006 20:01:04 +0000, Thomas Hawtin
    <> wrote, quoted or indirectly quoted someone
    who said :

    >Have you tried using jstack (or similar) to see what is continuing to run?

    Jstack it turns out does not run on windows.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 26, 2006
    #12
  13. Monique Y. Mudama, Jan 27, 2006
    #13
  14. Roedy Green wrote:
    > On Thu, 26 Jan 2006 20:01:04 +0000, Thomas Hawtin
    > <> wrote, quoted or indirectly quoted someone
    > who said :
    >
    >> Have you tried using jstack (or similar) to see what is continuing to run?

    > Jstack it turns out does not run on windows.


    jstack, by default, does much the same as ctrl-\ (which I think is
    ctrl-break on Windows) only with less fiddling. There are some options,
    like -m which causes it to NPE lots.

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
    Thomas Hawtin, Jan 27, 2006
    #14
  15. Roedy Green

    Roedy Green Guest

    On Thu, 26 Jan 2006 17:24:31 -0700, "Monique Y. Mudama"
    <> wrote, quoted or indirectly quoted someone who
    said :

    >That certainly smells like a threading issue. Every time I've used
    >tools to inspect the code and the code started to work, it has always
    >turned out to be a threading problem.
    >
    >http://catb.org/~esr/jargon/html/H/heisenbug.html


    more weirdness. The bug has now gone altogether. What changed? I
    added some convenience methods to one of the base classes, but I don't
    yet use them anywhere.

    This is like the old days of FORTRAN when making tiny changes to code
    changed which code got clobbered when subscripts when out of range.

    I don't like this. I have been hit twice in a day with programs that
    work fine on one compiler but not another. It is a sign Java has
    grown past the manageable bounds of complexity for a group of humans.

    There is no JNI in my code.
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 27, 2006
    #15
  16. > This is weird. I am doing pure batch text processing.

    Opening files is preceeded by SecurityManager checks. That may involve
    threads. Stefan Ram suggests a gradual isolation which may be
    reasonable.

    > 6. There is a quantum mechanical effect. Watching the apps with
    > JConsole makes it work!!


    Maybe, a long shot, when watching you give extra time which you don't
    give when not watching and something time limited gets to complete.

    Opalinski

    http://www.geocities.com/opalpaweb/
    opalinski from opalpaweb, Jan 27, 2006
    #16
  17. opalinski from opalpaweb, Jan 27, 2006
    #17
  18. On 2006-01-27, Roedy Green penned:
    >
    >
    > I don't like this. I have been hit twice in a day with programs that
    > work fine on one compiler but not another. It is a sign Java has
    > grown past the manageable bounds of complexity for a group of
    > humans.


    Now, now. I happen to know for a fact that they've had problems like
    this all along. In my graphics class in college, we were using Java
    1.1.7. I always made sure to test my projects, which I developed on
    my windows box, on the lab boxes before submitting them. My professor
    laughed at me and told me that Java works everywhere.

    Well. I was running late for the final project of the semester, and I
    was running into some weird environment problem, so I turned in the
    project, a wireframe activated by keypresses, which worked flawlessly
    on my Windows box. I got a zero on the project because the keypress
    code didn't work the same between Windows and Solaris. I got a C for
    the course instead of an A.

    I know @!#! well that Java has *never* worked perfectly across all
    systems.

    --
    monique

    Ask smart questions, get good answers:
    http://www.catb.org/~esr/faqs/smart-questions.html
    Monique Y. Mudama, Jan 27, 2006
    #18
  19. Roedy Green

    Guest

    It's not an API, it's a "dump" of the JVM state that you can use to
    debug things. It is automatically created in certain error conditions
    (like a bad signal from JNI code), but you can request one to debug a
    problem. On Windows, type Ctrl-BREAK in the command window in which
    the Java.exe is active, on *NIX, issue a kill -3 to the java.exe
    process. In windows, it will dump to standard out (the console window
    -- usually, you cannot do this in an IDE) a list of all threads, the
    currently active thread, held locks and monitors, and a variety of
    other highly useful information. On *NIX it will write the same
    information to a file named javacoreTIMESTAMP.txt in the default
    directory. I use it all the time to determine why things don't stop
    and other problems. Some web searching should help you find some
    pointers on interpreting the contents, but the thread dump is pretty
    straightforward -- it lists all the threads and their current stack.


    -Andrew
    , Jan 27, 2006
    #19
  20. Roedy Green

    Roedy Green Guest

    On 27 Jan 2006 06:31:08 -0800, wrote, quoted or
    indirectly quoted someone who said :

    > On Windows, type Ctrl-BREAK in the command window in which
    >the Java.exe is active,


    All I have even seen is a abort. does it hide this dump somewhere
    without telling you?
    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Java custom programming, consulting and coaching.
    Roedy Green, Jan 27, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. M. Duijkers
    Replies:
    2
    Views:
    474
    Joe Smith
    Dec 12, 2004
  2. Craig

    Postback quit working

    Craig, Sep 27, 2004, in forum: ASP .Net
    Replies:
    10
    Views:
    661
    Kikoz
    Sep 27, 2004
  3. KB
    Replies:
    4
    Views:
    1,166
  4. Java and Swing
    Replies:
    4
    Views:
    658
    Oliver Wong
    Sep 11, 2006
  5. Boris
    Replies:
    2
    Views:
    371
    Eric Brunel
    Dec 4, 2007
Loading...

Share This Page