Runtime.exec(String[]) Doesn't Always Work, bBut Runtime.exec(String) Does

H

Hal Vaughan

I've included the method I'm having trouble with at the end of the post.

I was using Runtime.getRuntime().exec(String) to execute programs and had no
problem with it at all, but since I had to include an argument, I was
worried about problems with spaces in program names, so I switched
to .exec(String[]) instead of .exec(String). I figure this way when it
turns out I have to run a program with a space in the filename, there
should be no problem.

When I was specifying the program with only a String, every program I would
attempt would run. Now that I'm using String[], I have times where I run
one program and it works, then run the same program, with the same
arguments, a few seconds later, and waitFor() returns immediately and I get
no data from the ErrorStream or InputStream.

I tried, as an experiment, to put quotation marks around the filename in a
string so it would look like this:

"/usr/bin/myprogram" -myargument

And that would not work -- the programs would not run at all.

Also as an experiment, in the spot I marked with a comment, I tried a
sleep() call, just in case waiting had any effect. When I had it wait 100
milliseconds, every time I ran a program, I got data from the appropriate
InputStream or ErrorStream, but not all the data I should have (it was
lopped off at the beginning). I tried shorter times for sleep() and with
10 milliseconds, no programs ran.

So why is it that sometimes the program runs and I get input and sometimes
it doesn't? My best guess is that the program always runs, but that there
might be a timing issue that keeps me from reading the streams quickly
enough, but that doesn't sound likely.

Any help is appreciated.

Thanks!

Hal

------Code method-----
public boolean runFile()
{
stopFlag = false;
System.out.println("DEBUG: Starting to run file, Command: " + runLine);
try
{
Process procRun = Runtime.getRuntime().exec(cmdInfo);
iRead = new BufferedReader (new
InputStreamReader(procRun.getInputStream()));
eRead = new BufferedReader (new
InputStreamReader(procRun.getErrorStream()));
System.out.println("\tDEBUG: Buffers made");
//Tried to use Thread.sleep() here with different timings
//Thread to read Standard output
//Only exception thrown by BufferedReader.readLine() is EOF, which
//means we're at the end, so there's no need to do anything with it.
new Thread(
new Runnable() {
public void run() {
String sRead;
try {
while ((sRead = iRead.readLine()) != null) {
System.out.println("DEBUG: Standard:" + sRead);
outData = outData + sRead;
if (stopFlag)
break;
}
} catch (Exception eofE) {}
}
}
).start();

//Thread to read Error output
new Thread(
new Runnable() {
public void run() {
String sRead;
try {
while ((sRead = eRead.readLine()) != null) {
System.out.println("DEBUG: Error:" + sRead);
outError = outError + sRead;
if (stopFlag)
break;
}
} catch (Exception eofE) {}
}
}
).start();

procRun.waitFor();
stopFlag = true;
System.out.println("\tDEBUG: Run is complete");
iStream.close();
eStream.close();
}
catch (Exception e)
{
System.err.println("Could not execute IDFile: " + runLine);
return false;
}
System.out.println("\tDEBUG: Done running file");
return true;
}
 
T

Thomas Hawtin

Hal said:
Also as an experiment, in the spot I marked with a comment, I tried a
sleep() call, just in case waiting had any effect. When I had it wait 100
milliseconds, every time I ran a program, I got data from the appropriate
InputStream or ErrorStream, but not all the data I should have (it was
lopped off at the beginning). I tried shorter times for sleep() and with
10 milliseconds, no programs ran.

The start of the data missing? That's very odd. It should drop anything
from the streams.
procRun.waitFor();
stopFlag = true;

You are instructing you child threads to stop before they have
necessarily read all of the data. (I assume stopFlag is volatile.) They
wont read the flag if there is no data to read and the streams are not
closed. A better idea would be to jsut let them complete normally.
System.out.println("\tDEBUG: Run is complete");
iStream.close();
eStream.close();

Now you are attempting to close the streams, before they are necessarily
read.

Tom Hawtin
 
H

Hal Vaughan

Thomas said:
The start of the data missing? That's very odd. It should drop anything
from the streams.

I tried again, with a sleep() call. I may have gotten some output files
mixed up, but it DID lose data from the streams. It may not have been from
the beginning, but I'm pretty sure it was.
You are instructing you child threads to stop before they have
necessarily read all of the data. (I assume stopFlag is volatile.) They
wont read the flag if there is no data to read and the streams are not
closed. A better idea would be to jsut let them complete normally.

So would I be better off just reading them continually until they throw an
EOFException? It seems that would be the best if that is the case.
Now you are attempting to close the streams, before they are necessarily
read.

Is that so even if I'm closing them AFTER I'm using waitFor()?

Thanks!

Hal
 
H

Hal Vaughan

Thomas said:
The start of the data missing? That's very odd. It should drop anything
from the streams.


You are instructing you child threads to stop before they have
necessarily read all of the data. (I assume stopFlag is volatile.) They
wont read the flag if there is no data to read and the streams are not
closed. A better idea would be to jsut let them complete normally.

 >                 System.out.println("\tDEBUG: Run is complete");
 >                 iStream.close();
 >                 eStream.close();

Now you are attempting to close the streams, before they are necessarily
read.

Tom Hawtin

I made changes based on these suggestions and it works now,with only one
issue that I fixed with a hack I'm not thrilled with.

The streams are not closed at the end of the routine, but within the threads
that read from the streams.  When reading from a stream throws an error
(and the only documented error they throw is EOF), the stream is closed.

I still use waitFor() at the end to determine when the program finishes
running, but after that, I wait 100 milliseconds, to allow any extra data
to be pulled from the streams.  Then I return.  That's the hack I'm worried
about.  I can't wait too long, since I'm going through a number of files,
but without waiting, I sometimes don't get any data from the streams.  I'd
think there should be a better way to make sure all the output from the
program has been received, but when the streams are done (i.e. all the
program output has completed), they don't always throw an EOF.

I tried setting flags when each stream threw an error and waiting until both
streams had thrown an EOFException, but when I did that, it would freeze
and at least one of the streams never closed.  If there is another way to
handle this, other than waiting for some time to allow the streams to
receive all the data, I'd really like to hear about it.

Thanks for any help and suggestions.

Hal
 
R

Rhino

Hal Vaughan said:
I tried again, with a sleep() call. I may have gotten some output files
mixed up, but it DID lose data from the streams. It may not have been
from
the beginning, but I'm pretty sure it was.


So would I be better off just reading them continually until they throw an
EOFException? It seems that would be the best if that is the case.


Is that so even if I'm closing them AFTER I'm using waitFor()?

Thanks!

I found this article quite helpful when I was messing with Runtime.exec() a
couple of years back. It may help you sort out your current problem or at
least understand better how it works:
http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps_p.html.
 
G

Gordon Beaton

When I was specifying the program with only a String, every program
I would attempt would run. Now that I'm using String[], I have times
where I run one program and it works, then run the same program,
with the same arguments, a few seconds later, and waitFor() returns
immediately and I get no data from the ErrorStream or InputStream.

I tried, as an experiment, to put quotation marks around the
filename in a string so it would look like this:

"/usr/bin/myprogram" -myargument

And that would not work -- the programs would not run at all.

exec(String) simply tokenizes the command and invokes exec(String[]).
So if exec(String) works but exec(String[]) doesn't, any problems with
the latter are most likely due to your tokenization of the command
line.

exec(String[]) is, as you correctly observe, the right choice when
your command contains *significant* whitespace, because the
StringTokenizer used by exec(String) is not particularly smart.

The rule is simple: each single component in the command line, i.e.
the command name, an option flag, an argument, etc, is one String in
the command array.

Don't use any quotes or escape characters, because there is no shell
to further interpret the command line. Any such characters will be
treated literally by the system, so unless your program really is
spelled with those characters it won't be found, or the arguments will
be meaningless, etc.
So why is it that sometimes the program runs and I get input and
sometimes it doesn't? My best guess is that the program always runs,
but that there might be a timing issue that keeps me from reading
the streams quickly enough, but that doesn't sound likely.

There should not be any timing issues. Data written to either of the
streams stays there until you read it. And if you don't read it, the
child will eventually hang waiting for you to do so, as you've already
discovered. And you won't get EOF until you've emptied the streams and
the child has exited (or closed the streams).

I don't see BufferedReader.ready() or InputStream.available() in your
code, but use of those could give the results you describe (avoid
them).

However if waitFor() returns, that is a string indication that your
program has exited. What is returned by Process.exitValue()? What does
your command line look like?

/gordon
 
G

Gordon Beaton

I still use waitFor() at the end to determine when the program
finishes running, but after that, I wait 100 milliseconds, to allow
any extra data to be pulled from the streams.  Then I return.
 That's the hack I'm worried about.

Simply read until readLine() returns null. That's BufferedReader's way
of indicating EOF.

/gordon
 
T

Thomas Hawtin

Hal said:
The streams are not closed at the end of the routine, but within the threads
that read from the streams. When reading from a stream throws an error
(and the only documented error they throw is EOF), the stream is closed.

Rather than trying to sleep, I'd suggest using Thread.join. You actually
only need one additional thread, as you are going to wait for the
process anyway.

I don't see any documentation that states that other types of
IOException are not thrown.

BTW: If your process returns many lines, the Java code will become
increasingly slow. Repeated string concatenation is bad - use
StringBuilder (or StringBuffer).

Tom Hawtin
 
H

Hal Vaughan

Gordon said:
Simply read until readLine() returns null. That's BufferedReader's way
of indicating EOF.

/gordon

If the process is still running but hasn't printed anything to output for a
while, what does readLine() return? An empty string instead of a null?

Thanks for the pointer. I'll add that to the routine.

An extra note: I did searches and found Sun's "official" method for reading
from a process and it does not take a lot of what has come up in this
thread into account.

Hal
 
H

Hal Vaughan

Thomas said:
Rather than trying to sleep, I'd suggest using Thread.join. You actually
only need one additional thread, as you are going to wait for the
process anyway.

I had never thought of that -- I'm not familiar with it and the API docs
don't go into detail, so I'll read up on it.
I don't see any documentation that states that other types of
IOException are not thrown.

Thanks. That confirms what I could find..
BTW: If your process returns many lines, the Java code will become
increasingly slow. Repeated string concatenation is bad - use
StringBuilder (or StringBuffer).

Tom Hawtin

I'm not too worried. Basically, this is part of a file finder. It looks
for files to identify them. Then it either reads the file in to find
identifying text, or runs it to check the output. That means usually the
program is run with something like "--help" or "-version" or something
similar and all I'm doing is reading in the output to compare it against a
version string.

Hal
 
H

Hal Vaughan

Gordon said:
When I was specifying the program with only a String, every program
I would attempt would run. Now that I'm using String[], I have times
where I run one program and it works, then run the same program,
with the same arguments, a few seconds later, and waitFor() returns
immediately and I get no data from the ErrorStream or InputStream.

I tried, as an experiment, to put quotation marks around the
filename in a string so it would look like this:

"/usr/bin/myprogram" -myargument

And that would not work -- the programs would not run at all.

exec(String) simply tokenizes the command and invokes exec(String[]).
So if exec(String) works but exec(String[]) doesn't, any problems with
the latter are most likely due to your tokenization of the command
line.

exec(String[]) is, as you correctly observe, the right choice when
your command contains *significant* whitespace, because the
StringTokenizer used by exec(String) is not particularly smart.

The rule is simple: each single component in the command line, i.e.
the command name, an option flag, an argument, etc, is one String in
the command array.

Don't use any quotes or escape characters, because there is no shell
to further interpret the command line. Any such characters will be
treated literally by the system, so unless your program really is
spelled with those characters it won't be found, or the arguments will
be meaningless, etc.

Thanks for clearing that up. I had found out about the quotes the hard way,
but wasn't sure of the why behind it and didn't realize the String was
tokenized, but that also confirms my thought that spaces in the file name
would be trouble. Thanks for the details and explicit background on this.
There should not be any timing issues. Data written to either of the
streams stays there until you read it. And if you don't read it, the
child will eventually hang waiting for you to do so, as you've already
discovered. And you won't get EOF until you've emptied the streams and
the child has exited (or closed the streams).

I don't see BufferedReader.ready() or InputStream.available() in your
code, but use of those could give the results you describe (avoid
them).

That's what I've heard: avoid them like the plague because they aren't
reliable and are just messy.
However if waitFor() returns, that is a string indication that your
program has exited. What is returned by Process.exitValue()? What does
your command line look like?

/gordon

Usually the command is just running the file (could be almost any file) to
check for a version number, so it's not a long term situation. In most
cases, the command will be run with an argument like "--help" or "/help" to
get an output that will give me a unique string somewhere that identifies
the specific version. I also know the help info is usually printed to
STDERR, so I wasn't checking return codes from exitValue().

It seems that even after waitFor() returns and the process is done, the
buffers aren't completely "filled" with all the output yet. There's
another post about Thread.join() that I'm looking into.

Thanks for the help!

Hal
 
G

Gordon Beaton

If the process is still running but hasn't printed anything to
output for a while, what does readLine() return? An empty string
instead of a null?

It's much simpler than that: readLine() *doesn't* return until there
*is* a line of text to return. At the end of the stream, it returns
null.

/gordon
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,733
Messages
2,569,440
Members
44,831
Latest member
HealthSmartketoReviews

Latest Threads

Top