Process.waitFor() returns, but threads reading output/error streams are blocked on read

Discussion in 'Java' started by Michael Grove, Aug 19, 2005.

  1. java gurus,

    i'm seeing some odd behavior running a process via Runtime.exec on redhat
    linux (seen it on RHEL AS 2.1, 3, 4) with 1.4 and 1.5 JVMs (the only ones
    i've tried). if i start a specific process via Runtime.exec and set up
    threads to read in the out/err streams, i find that after Process.waitFor()
    returns (~5 seconds) the threads are still stuck in InputStream.read(). i
    would have expected them to return -1 (or fail with IOExceptions) almost
    immediately after process termination. i've tried closing the streams from
    the main thread and destroying the process, but that was not enough to break
    the threads out of read. i found that if i changed my reader threads to
    poll InputStream.available() they will find the streams closed after the
    main thread closes them, although available() will return 0 indefinitely if
    the main thread fails to close the streams manually.

    i'll include the java source below, but the problem seems extremely specific
    to the program i'm running in Runtime.exec(). i've been unable to duplicate
    the problem with another program. the program is a non-interactive command
    line tool (i'll call "cmd") that creates a socket connection to another
    process on the same machine and requests an action to be performed (in my
    case the action is "-start", passed in on the command line). the program
    takes about 5 seconds to complete. upon successful completion no data is
    written to standard out/err. the exit code is 0 on success, non-zero on
    failure. i can confirm in the process list that the program does terminate
    (which Process.waitFor() tells me anyway), and i can also confirm the
    program performed the action i expected. i discovered that when my java
    process is hung if i terminate the process the command line tool connects to
    that will break the threads out of read() with a -1 response. that leads me
    to believe that something was not cleaned up when the command line tool
    exited. i've also found that if i invoke the command line tool
    differently - e.g. i run "cmd -start" after a previous run of "cmd -start"
    (close to a no-op) or run "cmd -status" - the java process terminates
    normally. so something about the first call to "cmd -start" preventes
    InputStream.read() from returning.

    some questions:
    * after Process.waitFor() returns, what could cause
    Process.getInputStream().read() and Process.getErrorStream().read() to block
    indefinitely? shouldn't they return -1 or receive IOExceptions almost
    immediately?
    * any thoughts on what this command line tool might be doing (perhaps not
    closing its out/err streams gracefully when it terminates?) that would cause
    these reads to block?

    thanks.

    -mike

    import java.io.IOException;

    public class cmdstart {
    public static void main(String [] args) throws IOException,
    InterruptedException {
    String cmd = "cmd -start";
    Process process = Runtime.getRuntime().exec(cmd);
    MyReader outReader = new MyReader(process.getInputStream());
    Thread outThread = new Thread(outReader);
    outThread.start();
    MyReader errReader = new MyReader(process.getErrorStream());
    Thread errThread = new Thread(errReader);
    errThread.start();
    System.out.println("exit code is " + process.waitFor());
    //process.destroy();
    // process.getInputStream().close();
    // process.getErrorStream().close();
    outThread.join();
    errThread.join();
    System.out.println("joins completed");
    }
    }

    import java.io.*;

    public class MyReader implements Runnable {
    private final InputStream is;

    public MyReader(InputStream is) {
    this.is = is;
    }

    public void run() {
    // normally would read in while here, but these threads never get any
    data -
    // they are blocked on the first read() call forever.
    int length;
    try {
    length = is.read();
    if (length == -1) {
    System.out.println("-1");
    return;
    }
    if (length == 0) {
    System.out.println("0");
    return;
    }
    System.out.println("read something");
    } catch (Throwable t) {
    System.out.println(t);
    }
    }
    }
     
    Michael Grove, Aug 19, 2005
    #1
    1. Advertising

  2. Re: Process.waitFor() returns, but threads reading output/error streamsare blocked on read

    Michael Grove wrote:
    >
    > * after Process.waitFor() returns, what could cause
    > Process.getInputStream().read() and Process.getErrorStream().read() to block
    > indefinitely? shouldn't they return -1 or receive IOExceptions almost
    > immediately?
    > * any thoughts on what this command line tool might be doing (perhaps not
    > closing its out/err streams gracefully when it terminates?) that would cause
    > these reads to block?


    As a random guess forking (for instance executing another command)
    and/or reopening stdout/stderr.

    Apparently strace is the Linux command to use to see what a process is
    doing. Never tried it myself.

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
     
    Thomas Hawtin, Aug 19, 2005
    #2
    1. Advertising

  3. thanks for the response.

    you're right that something like fork occurs. i haven't fully tracked it
    down via strace, but i remembered that the first time "cmd -start" is run it
    actually bootstraps the other process it normally connects to if it finds it
    not running. that process ends up started with init (pid 1) as it's parent
    process. i didn't see exactly how this happened via strace, but it seemed
    to occur right around a clone() call, which i see is pretty similar to fork.
    i think the cloned process might actually be the one that bootstraps the
    other process before it terminates. the cloned process vanishes before i
    can get a chance to see what it's doing.

    that should at least help me duplicate this problem with a "cmd" program of
    my own creation. however i still wonder - is it correct behavior in java
    that after the process has terminated the streams are not closed? even if
    that answer is yes, i'm still confused why a polling InputStream.available()
    would fail with an IOException once i closed the Process streams manually
    yet InputStream.read() would remain blocked.

    i know Runtime.exec() is a bit of a crapshoot and the javadocs indicate it
    may not work well in a variety of cases. wondering though if this behavior
    is 1) expected/correct or 2) a Runtime.exec() limitation or 3) a JVM bug.

    thanks.

    -mike

    "Thomas Hawtin" <> wrote in message
    news:43051c2c$0$97138$...
    > Michael Grove wrote:
    > >
    >> * after Process.waitFor() returns, what could cause
    >> Process.getInputStream().read() and Process.getErrorStream().read() to
    >> block indefinitely? shouldn't they return -1 or receive IOExceptions
    >> almost immediately?
    >> * any thoughts on what this command line tool might be doing (perhaps not
    >> closing its out/err streams gracefully when it terminates?) that would
    >> cause these reads to block?

    >
    > As a random guess forking (for instance executing another command) and/or
    > reopening stdout/stderr.
    >
    > Apparently strace is the Linux command to use to see what a process is
    > doing. Never tried it myself.
    >
    > Tom Hawtin
    > --
    > Unemployed English Java programmer
    > http://jroller.com/page/tackline/
     
    Michael Grove, Aug 19, 2005
    #3
  4. Re: Process.waitFor() returns, but threads reading output/error streamsare blocked on read

    Michael Grove wrote:
    > you're right that something like fork occurs. i haven't fully tracked it
    > down via strace, but i remembered that the first time "cmd -start" is run it
    > actually bootstraps the other process it normally connects to if it finds it
    > not running. that process ends up started with init (pid 1) as it's parent
    > process. i didn't see exactly how this happened via strace, but it seemed
    > to occur right around a clone() call, which i see is pretty similar to fork.
    > i think the cloned process might actually be the one that bootstraps the
    > other process before it terminates. the cloned process vanishes before i
    > can get a chance to see what it's doing.
    >
    > that should at least help me duplicate this problem with a "cmd" program of
    > my own creation. however i still wonder - is it correct behavior in java
    > that after the process has terminated the streams are not closed? even if
    > that answer is yes, i'm still confused why a polling InputStream.available()
    > would fail with an IOException once i closed the Process streams manually
    > yet InputStream.read() would remain blocked.
    >
    > i know Runtime.exec() is a bit of a crapshoot and the javadocs indicate it
    > may not work well in a variety of cases. wondering though if this behavior
    > is 1) expected/correct or 2) a Runtime.exec() limitation or 3) a JVM bug.


    My guess would be that the observed behavior is more-or-less correct, in
    that the OS-level streams are not being closed, with the result that the
    Java-level streams are not closed either. This could be the case if the
    Process' stdin and stdout are inherited by the chain of other processes
    that are forked off, and not closed by those processes I suspect that's
    what is happening. If so, it's a minor bug in the bootstrapper,
    possibly duplicated in the other process, and it can probably be
    addressed in one or both of those places.

    --
    John Bollinger
     
    John C. Bollinger, Aug 20, 2005
    #4
  5. Michael Grove

    Roedy Green Guest

    On Sat, 20 Aug 2005 22:44:46 GMT, Roedy Green
    <> wrote or quoted :

    >
    >>that should at least help me duplicate this problem with a "cmd" program of
    >>my own creation. however i still wonder - is it correct behavior in java
    >>that after the process has terminated the streams are not closed? even if
    >>that answer is yes, i'm still confused why a polling InputStream.available()
    >>would fail with an IOException once i closed the Process streams manually
    >>yet InputStream.read() would remain blocked.

    >
    >try looking at java.util.prefs instead.


    I meant java.lang.ProcessBuilder. I was thinking of somenone else's
    question, and came back to this.
     
    Roedy Green, Aug 22, 2005
    #5
  6. thanks for clarifying - i couldn't imagine what preferences you had in mind
    ;-). program has to run on 1.4, so ProcessBuilder is not available.

    i've asked the authors of the "cmd" program to make certain streams are
    properly being closed, and will post any solution to the newsgroup.

    "Roedy Green" <> wrote in message
    news:...
    > On Sat, 20 Aug 2005 22:44:46 GMT, Roedy Green
    > <> wrote or quoted :
    >
    >>
    >>>that should at least help me duplicate this problem with a "cmd" program
    >>>of
    >>>my own creation. however i still wonder - is it correct behavior in java
    >>>that after the process has terminated the streams are not closed? even
    >>>if
    >>>that answer is yes, i'm still confused why a polling
    >>>InputStream.available()
    >>>would fail with an IOException once i closed the Process streams manually
    >>>yet InputStream.read() would remain blocked.

    >>
    >>try looking at java.util.prefs instead.

    >
    > I meant java.lang.ProcessBuilder. I was thinking of somenone else's
    > question, and came back to this.
    >
    >
     
    Michael Grove, Aug 23, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. googlepost
    Replies:
    0
    Views:
    1,583
    googlepost
    Jul 21, 2003
  2. Matthew Zimmer

    Process waitFor command

    Matthew Zimmer, Jan 6, 2004, in forum: Java
    Replies:
    4
    Views:
    4,586
    Matthew Zimmer
    Jan 7, 2004
  3. Jakir
    Replies:
    2
    Views:
    1,354
    Jakir
    Aug 26, 2004
  4. Replies:
    6
    Views:
    4,735
  5. Kevin
    Replies:
    2
    Views:
    24,587
    Kevin
    Mar 31, 2005
Loading...

Share This Page