Process does not receive EOF when reading from Process.getOutputStream()

Discussion in 'Java' started by Reid Madsen, May 14, 2004.

  1. Reid Madsen

    Reid Madsen Guest

    I'm using Runtime.exec() to run a command via xargs so that I can pass
    a long list of arguments (too long to put on the command line). 99% of
    the time this works perfectly. In the 1% case that fails, the xargs process
    hangs reading from its input stream. It acts as if it has not detected that
    we have closed its input stream in the java application.

    I'm running the JDK 1.4.1_03 version on Solaris 2.9 with 24 CPU's.

    Here are the basic elements of this implementation:

    1) I have created a subprocess using:

    Process xargs = Runtime.exec("/bin/xargs cleartool lsview -long");

    BufferedReader stdOut =
    new BufferedReader(new InputStreamReader(xargs.getInputStream()));
    BufferedReader stdErr =
    new BufferedReader(new InputStreamReader(xargs.getErrorStream()));
    PrintStream stdIn =
    new PrintStream(new BufferedOutputStream(xargs.getOutputStream()), true);

    2) I have then created two threads, one to dispose of stderr from xargs,
    and another to feed input into xargs:

    new Thread() {
    public void run() {
    // Discard all output on stderr
    while (stdErr.readln() != null);
    }
    }.start();

    // The list of arguments to pass to the process via /bin/xargs
    String[] args= <Array of strings>;

    new Thread() {
    public void run() {
    // Feed arguments to process stdin
    for (int i=0; i<args.length; ++i) {
    stdIn.println(args);
    // close process stdin to signal end of input
    // process should then process arguments and exit
    stdIn.flush();
    stdIn.close();
    }
    }.start();

    3) Then I process the output from process stdout directly:

    String input;
    while ((input = stdOut.readln()) != null) {
    // process input
    }

    4) Finally I wait for the process to exit:

    int exitStatus = xargs.waitFor();

    This seems to be the correct way of doing this, and it works
    most of the time. Has anyone experienced this problem before?
    If so, what is the solution? Or, can you give me some ideas of where to look?
    Thank you in advance for any help you may offer.

    Reid Madsen
    i2 Technologies
     
    Reid Madsen, May 14, 2004
    #1
    1. Advertisements

  2. Reid Madsen

    Sudsy Guest

    Reid Madsen wrote:
    So where's the one which gobbles stdout?
     
    Sudsy, May 14, 2004
    #2
    1. Advertisements

  3. Reid Madsen

    Reid Madsen Guest

    Sudsy,
    The stdout from the process is read by the loop in #3.
    All three streams are being handled -- stdin and stderr by background
    threads, and stdin by the current thread.
    Reid
     
    Reid Madsen, May 15, 2004
    #3
  4. What is xargs doing when it is hung?

    Do a:
    truss -p <pid_of_xargs>

    What system call is it in? It it stuck in a read or write on
    filedescriptor 0/1/2?

    What does cleartool do? What does a truss of cleartool say its doing?

    -Misk





    --
    ========================================================================
    Missaka Wijekoon (a.k.a. Misk)
    Sr. Software Engineer http://www.villageEdocs.com
    VillageEdocs
    ========================================================================
     
    Missaka Wijekoon, May 15, 2004
    #4
  5. Reid Madsen

    Reid Madsen Guest

    Missaka,

    I should have included that earlier -- thanks for asking.
    Running truss on the xargs pid shows that it is hung in a read on file
    descriptor 0.
    The cleartool command isn't running, it has not been launched yet -- or it
    has
    been launched, finished, and now xargs is trying to read more arguments.

    Remember the parent has closed the xargs input stream -- xargs should have
    seen
    an EOF and exited.

    FYI, cleartool is part of Rational/IBM Clearcase -- a source code control
    system.
    I should also note that I've only seen this occur with xargs -- so far.
    Maybe it's a
    bug in the xargs implementation.

    Thanks again.
    Reid Madsen
    i2 Technologies

     
    Reid Madsen, May 15, 2004
    #5
  6. (pssst..)
    ...What you might have left out 'now'
    was (most of) the 95 lines of earlier
    post that was no longer needed.

    Otherwise you might upset Roedy.
    (shhhh..) ;-)
     
    Andrew Thompson, May 15, 2004
    #6
  7. Have you tried closing stdOut and stdErr at the end of their respective
    threads? The reason I do this is that I when I perform this operation
    in C, I do i/o multiplexing with poll()...and if you aren't careful
    about reading everthing and closing everything, somethimes the buffering
    gets in the way.

    -Misk




    --
    ========================================================================
    Missaka Wijekoon (a.k.a. Misk)
    Sr. Software Engineer http://www.villageEdocs.com
    VillageEdocs
    ========================================================================
     
    Missaka Wijekoon, May 15, 2004
    #7
  8. Reid Madsen

    Marc Slemko Guest

    I haven't taken the time to look at your example much, but have seen
    issues that sound similar before in 1.4.1_03. See this bug:

    http://developer.java.sun.com/developer/bugParade/bugs/4843136.html

    Synopsis (process) pipe file descriptor from Runtime.exec not being closed

    it was fixed in 1.4.1_05.
     
    Marc Slemko, May 15, 2004
    #8
  9. Reid Madsen

    Reid Madsen Guest

    Thank you Marc!!

    When I run into bugs like this I begin to question my design.
    Is the design flawed? Or, is the bug elsewhere.

    Your timely posting validates my initial premise -- that the closing of
    stdin was
    not being propagated all the way down to the process.

    Thanks again.

    Reid Madsen
    i2 Technologies

     
    Reid Madsen, May 16, 2004
    #9
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.