Java Runtime.exec problem

S

suru

HI,
I am using Runtime.exec to excute my unix (wrapper) commands.
These commands are fired consecutively.
But somtimes i get following exception,
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:
145)
at java.io.BufferedInputStream.read(BufferedInputStream.java:304)
at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:
411)
at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:
453)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
at java.io.InputStreamReader.read(InputStreamReader.java:167)
at java.io.BufferedReader.fill(BufferedReader.java:136)
at java.io.BufferedReader.readLine(BufferedReader.java:299)
at java.io.BufferedReader.readLine(BufferedReader.java:362)
at com.vistaar.web.util.StreamGobbler.run(ToolkitAgent.java:711)

Following is the code snippet:
//method from which exec is invoked
public static String processCommandWithNewLine(String
commandToExecute) {
Runtime systemShell = Runtime.getRuntime();
Process outputProcess = null;
String toolkitOutput = "";
try {
LogHelper.debug(lg, commandToExecute);
outputProcess = systemShell.exec(commandToExecute);

StreamGobbler errorGobbler = new StreamGobbler
(outputProcess.getErrorStream(), true, lg);
StreamGobbler outputGobbler = new StreamGobbler
(outputProcess.getInputStream(), true, lg);
errorGobbler.start();
outputGobbler.start();
outputProcess.waitFor();

toolkitOutput = toolkitOutput + outputGobbler.getOutput();
toolkitOutput = toolkitOutput + errorGobbler.getOutput();
LogHelper.debug(lg, toolkitOutput);
} catch (Throwable ex) {
LogHelper.printErrorTrace(lg, ex);
} finally {
closeProcessStreams(outputProcess);
}
return toolkitOutput.trim();
}

// steam gobbler implmentaion to read output of the command
class StreamGobbler extends Thread {

InputStream is;
String output = "";
boolean insertNewLine;
Logger lg;

StreamGobbler(InputStream is, boolean insertNewLine, Logger lg) {
this.is = is;
this.insertNewLine = insertNewLine;
this.lg = lg;
}

public void run() {
try {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
if (insertNewLine) {
while ((line = br.readLine()) != null) {
output = output + line + "\n";
}
} else {
while ((line = br.readLine()) != null) {
output = output + line;
}
}
} catch (IOException ioe) {
LogHelper.printErrorTrace(lg, ioe);
}
}

public String getOutput() {
return output;
}
}


1. i m using thread to read output & error stream simultaneously
2. the above problem doesn't come consistently, but comes frequently

eventhough the program says "stream closed",
when i fire same command from shell it gives me proper output

any idea what is causing the problem?
or der there any better way to read output given by exec process?

thnx,
suraj
 
S

suru

Okay something closed your stream, but your snippet doesn't show anything
explicitly doing that.  However...








Does this last line close the streams that your gobblers are gobbling
on?  If so, they might still be gobbling after the process has finished
(i.e. after waitFor() returns).  Before closing the streams, try waiting
for the gobblers to complete (Thread#join()?), or get the gobblers to
close their respective streams themselves.


really good opinion :)
wil try and let u know
 
L

Lew

suru said:
i [sic] tried with Thread.join approach,
for some calls it just hangs after join call :(

Did the thread it was joining terminate? It sounds like it didn't.
The join() won't return until the thread it's joining terminates.

Also, join() is a blunt tool.

It's really not possible to fully answer your question without seeing
the code, though.

http://sscce.org/
 
L

Lew

Where did you put the join()s?  Which one is hanging?  Can you determine
whether the gobblers have completed?  What does your code look like
now?  (Take note of others' comments about SSCCEs too.)

From your original post:



I think you're going to have to do some sort of waiting (e.g. join())
between waitFor() and getOutput() anyway, as you need to make sure that
your gobblers have finished by then.  You could make getOutput() block
until the stream is finished, instead of using join().

Get the gobblers (instead of anything else) to close the streams too.
There's no reason for anything else to be dealing with the streams (is
there?).

Also you might consider using an Executor and/or a Future
(FutureTask).

Thread coordination is a fairly advanced topic. After decades of
working with concurrent programming, I still keep researching,
studying and repeating that research and study, and I still feel that
I need to keep doing so.

I recommend the book /Java Concurrency in Practice/ by Brian Goetz, et
al. There are articles by Mr. Goetz available on the IBM
DeveloperWorks site that will help a lot. The Sun tutorials are a
decent starting point.
 
A

alexandre_paterson

Hi Suraj,

any idea what is causing the problem?

I've long stopped caring about such problems. Java's
runtime support as always been flakey and tricky (documented
bugs regarding sometimes magically disappearing slave processes
when using the Un*x pipe '|' symbol comes to mind).

Correctly running external processes was already a "Java Pitfall"
ten years ago, and it still is. Your post is a case in point.

The sheer number of posts in here with people having
trouble executing external processes is unusually high.

The number of framework and "helper" classes out there trying
to give a saner abstraction is also madenning (and directly
related to the immediate pain most people encounter when trying
to launch external processes from Java).

I've probably launched more external processes, on more
platforms/OSes/differents versions of these OSes, than
anyone in this group and I can give you the "trick" I'm
using since years.

or der there any better way to read output given by exec process?

In my opinion yes, and it's trivially simple...

"free your mind"

Seen that correctly "Gobbling" or consuming stdout/stderr
from Java is tricky, I went on with a radical, ultra-simple,
solution : I wrap all my scripts in another wrapper script
that suppresses every stdout/stderr output and redirects them
to temporary files. In other words, I let the OS take care
of the correct gobbling/consuming of the script's output
by redirecting these to temporary files.

Then I simply parse these temporary files.

This is both radical and elegant and it works on Windows, OS X,
Linux, and other Un*x flavors.

The Un*x shell script (Bash in this case but YMMV) may
look like this (out of memory, to give an idea [both stdout
and stderr are redirected here to the same temp file]):

#!/bin/bash
TEMP_REDIR_FILE=$1
shift;
$@ > $TEMP_REDIR_FILE 2>&1

Brutally simple.

The equivalent for the various flavor of Windows is
easy too.

In case you need to specifically output to a non-tty
and be immune to hangups, then 'nohup' is useful
(man nohup and check your shell's nohup doc).

My actual scripts are a tad bit more advanced for in
the case of terminating external processes (most do
terminate, some are 'non-stop') I also append some
"magic" string to know, when parsing my file, that
the command is done, and exited without errors, etc.

I abstracted all this in a clean interface and handle
the result (ie the parsed stdout/stderr text) using a
callback. I implemented timeout facilities, watchdog
process kill -9'ing the watched process should it block
(a correctly implement Un*x OS makes guarantees regarding
kill -9's behavior [on Windows this was more complicated,
I had to resort to pskill.exe]), I also implemented retries,
etc.

It's just an opinion but I find this more advanced than what
you can do in Java (for example regarding the "kill -9"
guarantees [longstanding Java bugs where Process's destroy()
would not really release all the ressources], yadda yadda!) but
yet I find my abstraction much more simple to
use than the 'external processes Java SNAFU' (pardon, I meant
than the Java API dealing with external processes :)

I wrote this several years ago and it never failed on me,

Alex
 
S

suru

really nice opinion Alex,

i hv few queries for understanding,

1. do u know why exactly java hangs up? is der any posts/link
explaining the details?
Also, i have observed that sometimes the program hangs at
"br.readLine()" call

2. in ur suggestion, u have mentioned that, u redirect ur process
output to a temporary file,
is der simple way to keep temporary file name unique? since multiple
users may execute same commands (from java) hence output for each user
should be in different files.

thnx,
suraj
 
A

Andrew Thompson

really nice opinion Alex,

i hv .. u ... der .. ur ...

Do you expect anyone to read that rubbish?

How about you show your appreciation for the
help you've received by showing the extra effort
needed to type words properly (and in the process,
you might save yourself from seeming like a lazy
bozo).
 
S

suru

apologies for using short words (got habit due to chat) :)


Following is the proper version,

Really nice opinion Alex,

I have few queries for understanding,


1. Do you know why exactly java hangs up? is there any posts/link
explaining the details?
Also, i have observed that sometimes the program hangs at
"br.readLine()" call


2. In your suggestion, you have mentioned that, you redirect your
process
output to a temporary file,
is there simple way to keep temporary file name unique? since
multiple
users may execute same commands (from java) hence output for each
user
should be in different files.


Thanks,
Suraj
 
R

Roedy Green


If you want your temp file created in a particular directory so it can
be easily renamed to the original file, use the createTempFile as part
of the hunkio package. See http://mindprod.com/products1.htmlHUNKIO
--
Roedy Green Canadian Mind Products
http://mindprod.com

"The industrial civilisation is based on the consumption of energy resources that are inherently limited in quantity, and that are about to become scarce. When they do, competition for what remains will trigger dramatic economic and geopolitical events; in the end, it may be impossible for even a single nation to sustain industrialism as we have know it in the twentieth century."
~ Richard Heinberg, The Party’s Over: Oil, War, and the Fate of Industrial Societies
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top