Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does

Discussion in 'Java' started by Hal Vaughan, May 21, 2006.

  1. Hal Vaughan

    Hal Vaughan Guest

    I've included the method I'm having trouble with at the end of the post.

    I was using Runtime.getRuntime().exec(String) to execute programs and had no
    problem with it at all, but since I had to include an argument, I was
    worried about problems with spaces in program names, so I switched
    to .exec(String[]) instead of .exec(String). I figure this way when it
    turns out I have to run a program with a space in the filename, there
    should be no problem.

    When I was specifying the program with only a String, every program I would
    attempt would run. Now that I'm using String[], I have times where I run
    one program and it works, then run the same program, with the same
    arguments, a few seconds later, and waitFor() returns immediately and I get
    no data from the ErrorStream or InputStream.

    I tried, as an experiment, to put quotation marks around the filename in a
    string so it would look like this:

    "/usr/bin/myprogram" -myargument

    And that would not work -- the programs would not run at all.

    Also as an experiment, in the spot I marked with a comment, I tried a
    sleep() call, just in case waiting had any effect. When I had it wait 100
    milliseconds, every time I ran a program, I got data from the appropriate
    InputStream or ErrorStream, but not all the data I should have (it was
    lopped off at the beginning). I tried shorter times for sleep() and with
    10 milliseconds, no programs ran.

    So why is it that sometimes the program runs and I get input and sometimes
    it doesn't? My best guess is that the program always runs, but that there
    might be a timing issue that keeps me from reading the streams quickly
    enough, but that doesn't sound likely.

    Any help is appreciated.

    Thanks!

    Hal

    ------Code method-----
    public boolean runFile()
    {
    stopFlag = false;
    System.out.println("DEBUG: Starting to run file, Command: " + runLine);
    try
    {
    Process procRun = Runtime.getRuntime().exec(cmdInfo);
    iRead = new BufferedReader (new
    InputStreamReader(procRun.getInputStream()));
    eRead = new BufferedReader (new
    InputStreamReader(procRun.getErrorStream()));
    System.out.println("\tDEBUG: Buffers made");
    //Tried to use Thread.sleep() here with different timings
    //Thread to read Standard output
    //Only exception thrown by BufferedReader.readLine() is EOF, which
    //means we're at the end, so there's no need to do anything with it.
    new Thread(
    new Runnable() {
    public void run() {
    String sRead;
    try {
    while ((sRead = iRead.readLine()) != null) {
    System.out.println("DEBUG: Standard:" + sRead);
    outData = outData + sRead;
    if (stopFlag)
    break;
    }
    } catch (Exception eofE) {}
    }
    }
    ).start();

    //Thread to read Error output
    new Thread(
    new Runnable() {
    public void run() {
    String sRead;
    try {
    while ((sRead = eRead.readLine()) != null) {
    System.out.println("DEBUG: Error:" + sRead);
    outError = outError + sRead;
    if (stopFlag)
    break;
    }
    } catch (Exception eofE) {}
    }
    }
    ).start();

    procRun.waitFor();
    stopFlag = true;
    System.out.println("\tDEBUG: Run is complete");
    iStream.close();
    eStream.close();
    }
    catch (Exception e)
    {
    System.err.println("Could not execute IDFile: " + runLine);
    return false;
    }
    System.out.println("\tDEBUG: Done running file");
    return true;
    }
     
    Hal Vaughan, May 21, 2006
    #1
    1. Advertising

  2. Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String)Does

    Hal Vaughan wrote:
    > Also as an experiment, in the spot I marked with a comment, I tried a
    > sleep() call, just in case waiting had any effect. When I had it wait 100
    > milliseconds, every time I ran a program, I got data from the appropriate
    > InputStream or ErrorStream, but not all the data I should have (it was
    > lopped off at the beginning). I tried shorter times for sleep() and with
    > 10 milliseconds, no programs ran.


    The start of the data missing? That's very odd. It should drop anything
    from the streams.

    > procRun.waitFor();
    > stopFlag = true;


    You are instructing you child threads to stop before they have
    necessarily read all of the data. (I assume stopFlag is volatile.) They
    wont read the flag if there is no data to read and the streams are not
    closed. A better idea would be to jsut let them complete normally.

    > System.out.println("\tDEBUG: Run is complete");
    > iStream.close();
    > eStream.close();


    Now you are attempting to close the streams, before they are necessarily
    read.

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
     
    Thomas Hawtin, May 21, 2006
    #2
    1. Advertising

  3. Hal Vaughan

    Hal Vaughan Guest

    Thomas Hawtin wrote:

    > Hal Vaughan wrote:
    >> Also as an experiment, in the spot I marked with a comment, I tried a
    >> sleep() call, just in case waiting had any effect. When I had it wait
    >> 100 milliseconds, every time I ran a program, I got data from the
    >> appropriate InputStream or ErrorStream, but not all the data I should
    >> have (it was
    >> lopped off at the beginning). I tried shorter times for sleep() and with
    >> 10 milliseconds, no programs ran.

    >
    > The start of the data missing? That's very odd. It should drop anything
    > from the streams.


    I tried again, with a sleep() call. I may have gotten some output files
    mixed up, but it DID lose data from the streams. It may not have been from
    the beginning, but I'm pretty sure it was.

    >> procRun.waitFor();
    >> stopFlag = true;

    >
    > You are instructing you child threads to stop before they have
    > necessarily read all of the data. (I assume stopFlag is volatile.) They
    > wont read the flag if there is no data to read and the streams are not
    > closed. A better idea would be to jsut let them complete normally.


    So would I be better off just reading them continually until they throw an
    EOFException? It seems that would be the best if that is the case.

    > > System.out.println("\tDEBUG: Run is complete");
    > > iStream.close();
    > > eStream.close();

    >
    > Now you are attempting to close the streams, before they are necessarily
    > read.


    Is that so even if I'm closing them AFTER I'm using waitFor()?

    Thanks!

    Hal
     
    Hal Vaughan, May 21, 2006
    #3
  4. Hal Vaughan

    Hal Vaughan Guest

    Thomas Hawtin wrote:

    > Hal Vaughan wrote:
    >> Also as an experiment, in the spot I marked with a comment, I tried a
    >> sleep() call, just in case waiting had any effect.  When I had it wait
    >> 100 milliseconds, every time I ran a program, I got data from the
    >> appropriate InputStream or ErrorStream, but not all the data I should
    >> have (it was
    >> lopped off at the beginning).  I tried shorter times for sleep() and with
    >> 10 milliseconds, no programs ran.

    >
    > The start of the data missing? That's very odd. It should drop anything
    > from the streams.
    >
    >>                 procRun.waitFor();
    >>                 stopFlag = true;

    >
    > You are instructing you child threads to stop before they have
    > necessarily read all of the data. (I assume stopFlag is volatile.) They
    > wont read the flag if there is no data to read and the streams are not
    > closed. A better idea would be to jsut let them complete normally.
    >
    >  >                 System.out.println("\tDEBUG: Run is complete");
    >  >                 iStream.close();
    >  >                 eStream.close();
    >
    > Now you are attempting to close the streams, before they are necessarily
    > read.
    >
    > Tom Hawtin


    I made changes based on these suggestions and it works now,with only one
    issue that I fixed with a hack I'm not thrilled with.

    The streams are not closed at the end of the routine, but within the threads
    that read from the streams.  When reading from a stream throws an error
    (and the only documented error they throw is EOF), the stream is closed.

    I still use waitFor() at the end to determine when the program finishes
    running, but after that, I wait 100 milliseconds, to allow any extra data
    to be pulled from the streams.  Then I return.  That's the hack I'm worried
    about.  I can't wait too long, since I'm going through a number of files,
    but without waiting, I sometimes don't get any data from the streams.  I'd
    think there should be a better way to make sure all the output from the
    program has been received, but when the streams are done (i.e. all the
    program output has completed), they don't always throw an EOF.

    I tried setting flags when each stream threw an error and waiting until both
    streams had thrown an EOFException, but when I did that, it would freeze
    and at least one of the streams never closed.  If there is another way to
    handle this, other than waiting for some time to allow the streams to
    receive all the data, I'd really like to hear about it.

    Thanks for any help and suggestions.

    Hal
     
    Hal Vaughan, May 21, 2006
    #4
  5. Hal Vaughan

    Rhino Guest

    "Hal Vaughan" <> wrote in message
    news:...
    > Thomas Hawtin wrote:
    >
    >> Hal Vaughan wrote:
    >>> Also as an experiment, in the spot I marked with a comment, I tried a
    >>> sleep() call, just in case waiting had any effect. When I had it wait
    >>> 100 milliseconds, every time I ran a program, I got data from the
    >>> appropriate InputStream or ErrorStream, but not all the data I should
    >>> have (it was
    >>> lopped off at the beginning). I tried shorter times for sleep() and
    >>> with
    >>> 10 milliseconds, no programs ran.

    >>
    >> The start of the data missing? That's very odd. It should drop anything
    >> from the streams.

    >
    > I tried again, with a sleep() call. I may have gotten some output files
    > mixed up, but it DID lose data from the streams. It may not have been
    > from
    > the beginning, but I'm pretty sure it was.
    >
    >>> procRun.waitFor();
    >>> stopFlag = true;

    >>
    >> You are instructing you child threads to stop before they have
    >> necessarily read all of the data. (I assume stopFlag is volatile.) They
    >> wont read the flag if there is no data to read and the streams are not
    >> closed. A better idea would be to jsut let them complete normally.

    >
    > So would I be better off just reading them continually until they throw an
    > EOFException? It seems that would be the best if that is the case.
    >
    >> > System.out.println("\tDEBUG: Run is complete");
    >> > iStream.close();
    >> > eStream.close();

    >>
    >> Now you are attempting to close the streams, before they are necessarily
    >> read.

    >
    > Is that so even if I'm closing them AFTER I'm using waitFor()?
    >
    > Thanks!


    I found this article quite helpful when I was messing with Runtime.exec() a
    couple of years back. It may help you sort out your current problem or at
    least understand better how it works:
    http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps_p.html.

    --
    Rhino
     
    Rhino, May 21, 2006
    #5
  6. On Sun, 21 May 2006 15:49:27 -0400, Hal Vaughan wrote:
    > When I was specifying the program with only a String, every program
    > I would attempt would run. Now that I'm using String[], I have times
    > where I run one program and it works, then run the same program,
    > with the same arguments, a few seconds later, and waitFor() returns
    > immediately and I get no data from the ErrorStream or InputStream.
    >
    > I tried, as an experiment, to put quotation marks around the
    > filename in a string so it would look like this:
    >
    > "/usr/bin/myprogram" -myargument
    >
    > And that would not work -- the programs would not run at all.


    exec(String) simply tokenizes the command and invokes exec(String[]).
    So if exec(String) works but exec(String[]) doesn't, any problems with
    the latter are most likely due to your tokenization of the command
    line.

    exec(String[]) is, as you correctly observe, the right choice when
    your command contains *significant* whitespace, because the
    StringTokenizer used by exec(String) is not particularly smart.

    The rule is simple: each single component in the command line, i.e.
    the command name, an option flag, an argument, etc, is one String in
    the command array.

    Don't use any quotes or escape characters, because there is no shell
    to further interpret the command line. Any such characters will be
    treated literally by the system, so unless your program really is
    spelled with those characters it won't be found, or the arguments will
    be meaningless, etc.

    > So why is it that sometimes the program runs and I get input and
    > sometimes it doesn't? My best guess is that the program always runs,
    > but that there might be a timing issue that keeps me from reading
    > the streams quickly enough, but that doesn't sound likely.


    There should not be any timing issues. Data written to either of the
    streams stays there until you read it. And if you don't read it, the
    child will eventually hang waiting for you to do so, as you've already
    discovered. And you won't get EOF until you've emptied the streams and
    the child has exited (or closed the streams).

    I don't see BufferedReader.ready() or InputStream.available() in your
    code, but use of those could give the results you describe (avoid
    them).

    However if waitFor() returns, that is a string indication that your
    program has exited. What is returned by Process.exitValue()? What does
    your command line look like?

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, May 22, 2006
    #6
  7. On Sun, 21 May 2006 18:26:10 -0400, Hal Vaughan wrote:
    > I still use waitFor() at the end to determine when the program
    > finishes running, but after that, I wait 100 milliseconds, to allow
    > any extra data to be pulled from the streams.  Then I return.
    >  That's the hack I'm worried about.


    Simply read until readLine() returns null. That's BufferedReader's way
    of indicating EOF.

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, May 22, 2006
    #7
  8. Re: Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String)Does

    Hal Vaughan wrote:
    >
    > The streams are not closed at the end of the routine, but within the threads
    > that read from the streams. When reading from a stream throws an error
    > (and the only documented error they throw is EOF), the stream is closed.


    Rather than trying to sleep, I'd suggest using Thread.join. You actually
    only need one additional thread, as you are going to wait for the
    process anyway.

    I don't see any documentation that states that other types of
    IOException are not thrown.

    BTW: If your process returns many lines, the Java code will become
    increasingly slow. Repeated string concatenation is bad - use
    StringBuilder (or StringBuffer).

    Tom Hawtin
    --
    Unemployed English Java programmer
    http://jroller.com/page/tackline/
     
    Thomas Hawtin, May 22, 2006
    #8
  9. Hal Vaughan

    Hal Vaughan Guest

    Gordon Beaton wrote:

    > On Sun, 21 May 2006 18:26:10 -0400, Hal Vaughan wrote:
    >> I still use waitFor() at the end to determine when the program
    >> finishes running, but after that, I wait 100 milliseconds, to allow
    >> any extra data to be pulled from the streams.  Then I return.
    >> That's the hack I'm worried about.

    >
    > Simply read until readLine() returns null. That's BufferedReader's way
    > of indicating EOF.
    >
    > /gordon


    If the process is still running but hasn't printed anything to output for a
    while, what does readLine() return? An empty string instead of a null?

    Thanks for the pointer. I'll add that to the routine.

    An extra note: I did searches and found Sun's "official" method for reading
    from a process and it does not take a lot of what has come up in this
    thread into account.

    Hal
     
    Hal Vaughan, May 22, 2006
    #9
  10. Hal Vaughan

    Hal Vaughan Guest

    Thomas Hawtin wrote:

    > Hal Vaughan wrote:
    >>
    >> The streams are not closed at the end of the routine, but within the
    >> threads
    >> that read from the streams. When reading from a stream throws an error
    >> (and the only documented error they throw is EOF), the stream is closed.

    >
    > Rather than trying to sleep, I'd suggest using Thread.join. You actually
    > only need one additional thread, as you are going to wait for the
    > process anyway.


    I had never thought of that -- I'm not familiar with it and the API docs
    don't go into detail, so I'll read up on it.

    > I don't see any documentation that states that other types of
    > IOException are not thrown.


    Thanks. That confirms what I could find..

    > BTW: If your process returns many lines, the Java code will become
    > increasingly slow. Repeated string concatenation is bad - use
    > StringBuilder (or StringBuffer).
    >
    > Tom Hawtin


    I'm not too worried. Basically, this is part of a file finder. It looks
    for files to identify them. Then it either reads the file in to find
    identifying text, or runs it to check the output. That means usually the
    program is run with something like "--help" or "-version" or something
    similar and all I'm doing is reading in the output to compare it against a
    version string.

    Hal
     
    Hal Vaughan, May 22, 2006
    #10
  11. Hal Vaughan

    Hal Vaughan Guest

    Gordon Beaton wrote:

    > On Sun, 21 May 2006 15:49:27 -0400, Hal Vaughan wrote:
    >> When I was specifying the program with only a String, every program
    >> I would attempt would run. Now that I'm using String[], I have times
    >> where I run one program and it works, then run the same program,
    >> with the same arguments, a few seconds later, and waitFor() returns
    >> immediately and I get no data from the ErrorStream or InputStream.
    >>
    >> I tried, as an experiment, to put quotation marks around the
    >> filename in a string so it would look like this:
    >>
    >> "/usr/bin/myprogram" -myargument
    >>
    >> And that would not work -- the programs would not run at all.

    >
    > exec(String) simply tokenizes the command and invokes exec(String[]).
    > So if exec(String) works but exec(String[]) doesn't, any problems with
    > the latter are most likely due to your tokenization of the command
    > line.
    >
    > exec(String[]) is, as you correctly observe, the right choice when
    > your command contains *significant* whitespace, because the
    > StringTokenizer used by exec(String) is not particularly smart.
    >
    > The rule is simple: each single component in the command line, i.e.
    > the command name, an option flag, an argument, etc, is one String in
    > the command array.
    >
    > Don't use any quotes or escape characters, because there is no shell
    > to further interpret the command line. Any such characters will be
    > treated literally by the system, so unless your program really is
    > spelled with those characters it won't be found, or the arguments will
    > be meaningless, etc.


    Thanks for clearing that up. I had found out about the quotes the hard way,
    but wasn't sure of the why behind it and didn't realize the String was
    tokenized, but that also confirms my thought that spaces in the file name
    would be trouble. Thanks for the details and explicit background on this.

    >> So why is it that sometimes the program runs and I get input and
    >> sometimes it doesn't? My best guess is that the program always runs,
    >> but that there might be a timing issue that keeps me from reading
    >> the streams quickly enough, but that doesn't sound likely.

    >
    > There should not be any timing issues. Data written to either of the
    > streams stays there until you read it. And if you don't read it, the
    > child will eventually hang waiting for you to do so, as you've already
    > discovered. And you won't get EOF until you've emptied the streams and
    > the child has exited (or closed the streams).
    >
    > I don't see BufferedReader.ready() or InputStream.available() in your
    > code, but use of those could give the results you describe (avoid
    > them).


    That's what I've heard: avoid them like the plague because they aren't
    reliable and are just messy.

    > However if waitFor() returns, that is a string indication that your
    > program has exited. What is returned by Process.exitValue()? What does
    > your command line look like?
    >
    > /gordon


    Usually the command is just running the file (could be almost any file) to
    check for a version number, so it's not a long term situation. In most
    cases, the command will be run with an argument like "--help" or "/help" to
    get an output that will give me a unique string somewhere that identifies
    the specific version. I also know the help info is usually printed to
    STDERR, so I wasn't checking return codes from exitValue().

    It seems that even after waitFor() returns and the process is done, the
    buffers aren't completely "filled" with all the output yet. There's
    another post about Thread.join() that I'm looking into.

    Thanks for the help!

    Hal
     
    Hal Vaughan, May 22, 2006
    #11
  12. On Mon, 22 May 2006 10:32:45 -0400, Hal Vaughan wrote:
    > If the process is still running but hasn't printed anything to
    > output for a while, what does readLine() return? An empty string
    > instead of a null?


    It's much simpler than that: readLine() *doesn't* return until there
    *is* a line of text to return. At the end of the stream, it returns
    null.

    /gordon

    --
    [ do not email me copies of your followups ]
    g o r d o n + n e w s @ b a l d e r 1 3 . s e
     
    Gordon Beaton, May 22, 2006
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?QXJuZQ==?=

    valign=top doesn't always work

    =?Utf-8?B?QXJuZQ==?=, Dec 6, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    422
    =?Utf-8?B?QXJuZQ==?=
    Dec 6, 2004
  2. =?Utf-8?B?QXJuZQ==?=

    Valign=top doesn't always work

    =?Utf-8?B?QXJuZQ==?=, Dec 6, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    594
    Dave Fancher
    Dec 7, 2004
  3. Timo Nentwig
    Replies:
    5
    Views:
    1,734
    John C. Bollinger
    Dec 2, 2004
  4. =?Utf-8?B?am9uZWZlcg==?=

    responding to changed text doesn't always work

    =?Utf-8?B?am9uZWZlcg==?=, May 23, 2006, in forum: ASP .Net
    Replies:
    1
    Views:
    307
    Spam Catcher
    May 23, 2006
  5. Wayne Erfling

    .Selected = false doesn't always work in MenuControl

    Wayne Erfling, Oct 15, 2006, in forum: ASP .Net Web Controls
    Replies:
    0
    Views:
    140
    Wayne Erfling
    Oct 15, 2006
Loading...

Share This Page