Terminating a process tree

[Wayne Marrison]

(posted yesterday in comp.lang.java.help, but not sure if that was the right
place and had no response, so posting in here today, sorry for the
additional bandwidth to those that subscribe to both groups)

Hi group,

I'm new here, so apologies if I ask this in the wrong way.

I have been programming in java for about 2 months, and am currently trying
to control an external application that I start using:

Process proc;
Runtime rt;

proc = rt.exec("java.exe -cp ... etc.. something");

Then I capture the input/output & error streams and thread them off.

The problem I have, is that I need to architect for the eventuality of the
external process locking up and not responding to requests. I have the
whole capturing of streams in a timeout, so I can tell when the external
program has not responded within a reasonable time, however when I issue the
proc.destroy() method, the java.exe process takes up almost 100% cpu and
never stops. I have to use operating system tools to perform a manual kill
of the process.

I can only assume that the proc.destroy() doesnt kill the entire process
tree, and because the command line starts off java.exe, which runs the
something.class file, a process is orphaned somewhere.

Any help would be much appreciated.

Thanks

Wayne

 

[John C. Bollinger]

I have been programming in java for about 2 months, and am currently trying
to control an external application that I start using:

Process proc;
Runtime rt;

proc = rt.exec("java.exe -cp ... etc.. something");

Then I capture the input/output & error streams and thread them off.

No detail there, but it sounds like a correct approach. Do you really
need to spawn a new VM for this, though? Perhaps there are reasons why
you do need to do (or at least, reasons why it's convenient), but in
many cases it would work as well or better to just start up a new thread
in the current VM for an "external" Java process.

The problem I have, is that I need to architect for the eventuality of the
external process locking up and not responding to requests. I have the
whole capturing of streams in a timeout, so I can tell when the external
program has not responded within a reasonable time, however when I issue the
proc.destroy() method, the java.exe process takes up almost 100% cpu and

_Which_ java.exe? You have (at least) two. Not that it makes much
difference.

never stops. I have to use operating system tools to perform a manual kill
of the process.

If the external process is unable to respond to requests then it may be
in a state where it is unable to respond to the OS' normal signals to
shut down, either.

I can only assume that the proc.destroy() doesnt kill the entire process
tree, and because the command line starts off java.exe, which runs the
something.class file, a process is orphaned somewhere.

You are inappropriately mixing scope / terminology. From the OS' point
of view, the program running is java.exe. Period. That particular
program will typically be multithreaded, but the OS has no sense or
specific knowledge of the particular class files from which the VM is
drawing code to execute. It may be in your case that some thread of
that process is refusing to die, but that would be an OS- and state-
dependent property of the Java program, with some dependency on the
OS-dependent implementation of Process.destroy(). It is outside the
scope of the specification of Process.destroy().

My recommendation would be to fix the external program, if that is
possible, and to avoid using Process.destroy(). You may be able to
simply abandon a deadlocked external program, although that's not very
clean. A cleaner approach might be to spawn the "external" program
_inside_ the host VM (in its own thread), probably with the use of an
appropriate ClassLoader so as to isolate the external program from the
host. Make the external app's thread a daemon thread so that it will
(in principle) not prevent the host from shutting down (and shutting it
down) if it locks up. The one problem this cannot solve is that of
forcing the external app to relinquish resources it has locked, such as
TCP ports.


John Bollinger
(e-mail address removed)

 

[Wayne Marrison]

No detail there, but it sounds like a correct approach. Do you really
need to spawn a new VM for this, though? Perhaps there are reasons why
you do need to do (or at least, reasons why it's convenient), but in many
cases it would work as well or better to just start up a new thread in the
current VM for an "external" Java process.

If you can point me in the right direction for an example on how to do this,
it would be appreciated. I'm still very new to this, but have deadlines to
attempt to meet.

_Which_ java.exe? You have (at least) two. Not that it makes much
difference.


If the external process is unable to respond to requests then it may be in
a state where it is unable to respond to the OS' normal signals to shut
down, either.

Using O/S tools works fine to kill the remaining java.exe.

You are inappropriately mixing scope / terminology. From the OS' point of
view, the program running is java.exe. Period. That particular program
will typically be multithreaded, but the OS has no sense or specific
knowledge of the particular class files from which the VM is drawing code
to execute. It may be in your case that some thread of that process is
refusing to die, but that would be an OS- and state- dependent property of
the Java program, with some dependency on the OS-dependent implementation
of Process.destroy(). It is outside the scope of the specification of
Process.destroy().

My recommendation would be to fix the external program, if that is
possible, and to avoid using Process.destroy(). You may be able to simply
abandon a deadlocked external program, although that's not very clean. A
cleaner approach might be to spawn the "external" program _inside_ the
host VM (in its own thread), probably with the use of an appropriate
ClassLoader so as to isolate the external program from the host. Make the
external app's thread a daemon thread so that it will (in principle) not
prevent the host from shutting down (and shutting it down) if it locks up.
The one problem this cannot solve is that of forcing the external app to
relinquish resources it has locked, such as TCP ports.

I cannot fix the external application, and strongly suspect that its the way
I am trying to control it and close it that is causing the problem. The
external app is a 3rd party app and as such is beyond my adjustment.

John Bollinger
(e-mail address removed)

Thanks for your help sofar.

Wayne

 

[John C. Bollinger]

If you can point me in the right direction for an example on how to do this,
it would be appreciated. I'm still very new to this, but have deadlines to
attempt to meet.

All you need to do to start a Java program is invoke the static main()
method of its main class. You can do that from any Java code:

void startExternalApp(String[] args) {
ExternalApp.main(args);
}

If you want to make the external app use its own ClassLoader (so as to
segregate its classes from your launcher app's own, and also from those
of any other instance your launcher starts, then you would need to get a
bit fancier. You would instantiate the ClassLoader (probably an
instance of URLClassLoader; make sure it points to a suitable parent
ClassLoader) use it to load the correct Class, then look up the correct
main() Method in the Class object and invoke it. The Class and Method
classes are part of the Java "Reflection" API -- you should generally
avoid them unless you need them, but this is a case where you would need
them.

You can take one more step and assign the external app to its own
thread. The easiest way to do this would be something like:

Thread startExternalAppInThread(final String[] args) {
ClassLoader cl = new URLClassLoader(/* your args here*/);
Class appClass = cl.findClass("fully.qualified.name");
final Method mainMethod =
appClass.getMethod("main", new Class[] {args.getClass()});

return new Thread() {
public void run() {
mainMethod.invoke(null, new Object[] {args});
}
}
}

Note that I'm typing quickly, so don't assume that that's all kosher,
but it should give you an idea.

Using O/S tools works fine to kill the remaining java.exe.

The OS' tools may include both a "soft" kill and a hard "kill". Java
implementations are unlikely to perform the latter because it may not be
safe. The former, however, may not work if the process is badly hung.


John Bollinger
(e-mail address removed)

 

[marcus]

Hey Wayne,
Since you agree this isn't the correct way to do it and you need a hack,
I can suggest one that has worked for me in FreeBSD (should work in any
linux type environment). Instead of launching the second java program
directly I used a control script (stolen from my apache setup, I think)
that records the PID in a dedicated text file. When I needed to kill it
I invoked the control script, which issued the OS kill command on the
recorded PID. Kills it dead as far as I know.

 

[Wayne Marrison]

If you can point me in the right direction for an example on how to do
this, it would be appreciated. I'm still very new to this, but have
deadlines to attempt to meet.

All you need to do to start a Java program is invoke the static main()
method of its main class. You can do that from any Java code:

void startExternalApp(String[] args) {
ExternalApp.main(args);
}

ExternalApp is imported somehow?

If you want to make the external app use its own ClassLoader (so as to
segregate its classes from your launcher app's own, and also from those of
any other instance your launcher starts, then you would need to get a bit
fancier. You would instantiate the ClassLoader (probably an instance of
URLClassLoader; make sure it points to a suitable parent ClassLoader) use
it to load the correct Class, then look up the correct main() Method in
the Class object and invoke it. The Class and Method classes are part of
the Java "Reflection" API -- you should generally avoid them unless you
need them, but this is a case where you would need them.

You can take one more step and assign the external app to its own thread.
The easiest way to do this would be something like:

Thread startExternalAppInThread(final String[] args) {
ClassLoader cl = new URLClassLoader(/* your args here*/);
Class appClass = cl.findClass("fully.qualified.name");
final Method mainMethod =
appClass.getMethod("main", new Class[] {args.getClass()});

return new Thread() {
public void run() {
mainMethod.invoke(null, new Object[] {args});
}
}
}

It would appear the idea of ClassLoading is a good one (still lots of docs
to read to find out how to make my situation fit), however my primary
questions are:

The external application is a console based java app, and is started from
this command line:
"java -cp C:/xxxxxx/lib/xxxxxxxxxx23.jar com.xxxxxxxx.remote.xxx XXX" (names
changed to protect the innocent). Does this still fit the ClassLoader
model?

Can I still perform the capture and redirection of the relevant i/o streams
in order to control the application?

If so, would how would the following code change to reflect it?
(ErrorStreamReader,StreamReader and StreamWriter are all threads for the
relevant streams).
============================================
// any error message?
error = new ErrorStreamReader(proc.getErrorStream(), "ERROR", running); //
this should be picked up by the log reader

// any output?
pin = new PipedInputStream(); // open input pipe
pout = new PipedOutputStream(pin); // open output pipe and bind to input
pipe

dis = new DataInputStream(pin);

reader = new StreamReader(proc.getInputStream(), pout , "OUTPUT", running);
// capture output stream, and send to pout

// prepare writer
writer = new StreamWriter(proc.getOutputStream(),running);
==========================================

Note that I'm typing quickly, so don't assume that that's all kosher, but
it should give you an idea.


The OS' tools may include both a "soft" kill and a hard "kill". Java
implementations are unlikely to perform the latter because it may not be
safe. The former, however, may not work if the process is badly hung.


John Bollinger
(e-mail address removed)

Is there not a 3rd party class that perform this activity for me as there
seems to be a lot to learn and not a lot of time to learn it in? Perhaps
the hurry that I am in is half my problem ..

Many thanks for your time.

 

[Gordon Beaton]

I'm not sure how I would achieve this under Windows. I have looked
at calling the "kill.exe" or the "taskkill.exe" directly, but then
fall into the trap of trying to get the PID of the command I have
just started. I know there are suggestions on using JNI to achieve
that, but the learning curve is too great for me at this point.

I dont know how under windows to get the PID of an application using
batch/script commands .. maybe I'd best rethink this as the
complexities are starting to outweigh the benefits.

How does the child process behave when its stdin is closed?

A simple way to "kill" the process is to close the corresponding
stream in the parent process, effectively signalling EOF to the child.

The child could have a main loop that does nothing but this:

while ((in = System.in.read()) > 0); // empty loop body
System.exit(0);

Put the rest of the child's logic in a separate thread.

If the child actually needs to handle input from stdin, add logic to
the loop that feeds commands through a queue of objects to the other
thread, so that any hang won't prevent the loop from exiting when
stdin closes.

/gordon

 

[Wayne Marrison]

Hey Wayne,
Since you agree this isn't the correct way to do it and you need a hack, I
can suggest one that has worked for me in FreeBSD (should work in any
linux type environment). Instead of launching the second java program
directly I used a control script (stolen from my apache setup, I think)
that records the PID in a dedicated text file. When I needed to kill it I
invoked the control script, which issued the OS kill command on the
recorded PID. Kills it dead as far as I know.

Hi Marcus,

I'm not sure how I would achieve this under Windows. I have looked at
calling the "kill.exe" or the "taskkill.exe" directly, but then fall into
the trap of trying to get the PID of the command I have just started. I
know there are suggestions on using JNI to achieve that, but the learning
curve is too great for me at this point.

I dont know how under windows to get the PID of an application using
batch/script commands .. maybe I'd best rethink this as the complexities are
starting to outweigh the benefits.

Thanks for your help.

Wayne

 

[Wayne Marrison]

How does the child process behave when its stdin is closed?

A simple way to "kill" the process is to close the corresponding
stream in the parent process, effectively signalling EOF to the child.

The child could have a main loop that does nothing but this:

while ((in = System.in.read()) > 0); // empty loop body
System.exit(0);

Put the rest of the child's logic in a separate thread.

If the child actually needs to handle input from stdin, add logic to
the loop that feeds commands through a queue of objects to the other
thread, so that any hang won't prevent the loop from exiting when
stdin closes.

/gordon

The problem here is that the command I'm trying to control is not written by
myself - its seperate. I have my other threads (the ones that are
controlling the input/output streams to this external app) operating as far
as I can see, correctly. They are closed before I ask the external app to
close (tried it the other way round too, just to make sure) but the act of,
or attempting to forcibly destroy the hung application is not working.

I think the art of closing a working application is quite simple, as you
say, closing the input stream usually works. But if the application is in
an unknown, perhaps hung state, what I thought would terminate it
(proc.destroy()) isnt.

Wayne

 

[John C. Bollinger]

[...]

All you need to do to start a Java program is invoke the static main()
method of its main class. You can do that from any Java code:

void startExternalApp(String[] args) {
ExternalApp.main(args);
}

ExternalApp is imported somehow?

In that example, yes. The external app's jar would need to be in the
launcher's classpath, and you would need to import the
package.name.ExternalApp class into the launcher. Do note that that is
a compile-time name resolution thing: it will end up putting the
fully-qualified name of the ExternalApp class into your launcher class,
but it does not bring in any of its code.

Note also that ClassLoader-based solution is completely different in
this regard: in that case you want to _avoid_ having the external app's
classes accessible via your launcher app's own ClassLoader; they should
only be accessible via the ClassLoader(s) that you configure for the
external app.

If you want to make the external app use its own ClassLoader (so as to
segregate its classes from your launcher app's own, and also from those of
any other instance your launcher starts, then you would need to get a bit
fancier. You would instantiate the ClassLoader (probably an instance of
URLClassLoader; make sure it points to a suitable parent ClassLoader) use
it to load the correct Class, then look up the correct main() Method in
the Class object and invoke it. The Class and Method classes are part of
the Java "Reflection" API -- you should generally avoid them unless you
need them, but this is a case where you would need them.

You can take one more step and assign the external app to its own thread.
The easiest way to do this would be something like:

Thread startExternalAppInThread(final String[] args) {
ClassLoader cl = new URLClassLoader(/* your args here*/);
Class appClass = cl.findClass("fully.qualified.name");
final Method mainMethod =
appClass.getMethod("main", new Class[] {args.getClass()});

return new Thread() {
public void run() {
mainMethod.invoke(null, new Object[] {args});
}
}
}

It would appear the idea of ClassLoading is a good one (still lots of docs
to read to find out how to make my situation fit), however my primary
questions are:

The external application is a console based java app, and is started from
this command line:
"java -cp C:/xxxxxx/lib/xxxxxxxxxx23.jar com.xxxxxxxx.remote.xxx XXX" (names
changed to protect the innocent). Does this still fit the ClassLoader
model?

Java uses a ClassLoader to load classes no matter what. In many cases
you don't have to do anything special about it to get what you want;
Java sets up a suitable ClassLoader for you and uses it to load your
classes. We're discussing using ClassLoaders to achieve somewhat
fancier effects that would require explicit use of a ClassLoader in your
code. The short answer is that every working Java application
necessarily fits the ClassLoader model because that's the _only_
Java-compliant way to run it.

Can I still perform the capture and redirection of the relevant i/o streams
in order to control the application?

Yes (see java.lang.System.setIn(), etc.). However, a JVM has only one
standard input stream, one standard output stream, and one standard
error stream, so if that's the only way to talk to the application then
your own application must forgo using those streams while the external
app is using them. (And you wouldn't be able to simultaneously run
multiple instances of the external app.)

If so, would how would the following code change to reflect it?
(ErrorStreamReader,StreamReader and StreamWriter are all threads for the
relevant streams).

Keep in mind that bringing the external app into your own app, you
reverse the sense of its streams. I'm guessing a bit on your
StreamReader, etc. classes, but here goes:

============================================
// any error message?
error = new ErrorStreamReader(proc.getErrorStream(), "ERROR", running); //

errorOut = new PipedOutputStream();
errorOutSink = new PipedInputStream(errorOut);
System.setErr(new PrintStream(errorOut));
error = new ErrorStreamReader(errorOutSink, "ERROR", running);

this should be picked up by the log reader

// any output?
pin = new PipedInputStream(); // open input pipe
pout = new PipedOutputStream(pin); // open output pipe and bind to input
pipe

dis = new DataInputStream(pin);

reader = new StreamReader(proc.getInputStream(), pout , "OUTPUT", running);

procOut = new PipedOutputStream();
procOutSink = new PipedInputStream(procOut);
System.setOut(new PrintStream(procOut));
dis = new DataInputStream(procOutSink);

/*
Then read the external app's output via "dis".
Comment: Do you really need a DataInputStream here? Is the external app
really writing raw Java primitives?
*/

// capture output stream, and send to pout

// prepare writer
writer = new StreamWriter(proc.getOutputStream(),running);

I'm not certain of the details of StreamWriter's operation, but you
probably want to do something based on this:

procIn = new PipedInputStream();
procInSource = new PipedOutputStream(procIn);
System.setIn(procIn);

/*
(Then write to procInSource to send data to the external app's standard
input.)
*/

==========================================

Note that in that example it may not be safe to write to "procIn" and
read from "dis" in the same thread. That is not materially different
from the situation in your own code, however, where it is not safe to
write to "writer" and read from "dis" in the same thread. Even though
you are using separate StreamWriters and StreamReaders, you are sending
the external proc's output (read from the Process' InputStream) through
a pipe, which can block if the pipe's internal buffer fills. The whole
StreamReader bit is therefore kind of pointless for that part -- it
introduces an additional thread that doesn't materially benefit you.
The thread that is handling each stream needs to do the appropriate
thing with it, more or less on its own. It must not be possible for any
of them to block on action by any of the others.

Is there not a 3rd party class that perform this activity for me as there
seems to be a lot to learn and not a lot of time to learn it in? Perhaps
the hurry that I am in is half my problem ..

The ClassLoading bit? I wouldn't be surprised to hear that there is
one, but I'm not specifically aware of any. An external process
manager? I am fairly certain that you could find such a thing, perhaps
even for free, but again, I don't have a link to give you.


John Bollinger
(e-mail address removed)