Shell out without so many threads?

[bugbear]

Long ago I carefully implemented a Shell out
command, following the guidance from this:

http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html

Summary: use threads to gather stdout and stderr from
your exec, otherwise it might deadlock.

I even added (yet another) thread to implement timeouts
(to kill a shell script gone infinite).

Time moves on, and I now, under Tomcat, have a secondary
problem.

My threads are thrashing.

I'm using 4 threads (main, stdout_gather, stderr_gather, timeout)
per shell, and it's just too many.

Does anybody have any advice, experience, or code
to reduce this number.

My (hazy) thoughts involve using non-blocking IO to pick
up multiple stdout/stderr streams in a single thread (but I can't get a
Selectable from a Stream) and using a
single thread with a "next timeout" model to
use a single timout thread for all shell outs.

All input gratefully received.

BugBear

 

[Knute Johnson]

Long ago I carefully implemented a Shell out
command, following the guidance from this:

http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html

Summary: use threads to gather stdout and stderr from
your exec, otherwise it might deadlock.

I even added (yet another) thread to implement timeouts
(to kill a shell script gone infinite).

Time moves on, and I now, under Tomcat, have a secondary
problem.

My threads are thrashing.

I'm using 4 threads (main, stdout_gather, stderr_gather, timeout)
per shell, and it's just too many.

Does anybody have any advice, experience, or code
to reduce this number.

My (hazy) thoughts involve using non-blocking IO to pick
up multiple stdout/stderr streams in a single thread (but I can't get a
Selectable from a Stream) and using a
single thread with a "next timeout" model to
use a single timout thread for all shell outs.

All input gratefully received.

BugBear

If you use ProcessBuilder you can merge stdout and stderr, that will
save you 25% of your threads. See
ProcessBuilder.redirectErrorStream(boolean redirect).

 

[Kevin McMurtrie]

Long ago I carefully implemented a Shell out
command, following the guidance from this:

http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html

Summary: use threads to gather stdout and stderr from
your exec, otherwise it might deadlock.

I even added (yet another) thread to implement timeouts
(to kill a shell script gone infinite).

Time moves on, and I now, under Tomcat, have a secondary
problem.

My threads are thrashing.

I'm using 4 threads (main, stdout_gather, stderr_gather, timeout)
per shell, and it's just too many.

Does anybody have any advice, experience, or code
to reduce this number.

My (hazy) thoughts involve using non-blocking IO to pick
up multiple stdout/stderr streams in a single thread (but I can't get a
Selectable from a Stream) and using a
single thread with a "next timeout" model to
use a single timout thread for all shell outs.

All input gratefully received.

BugBear

Define crashing.

Usually the JVM can deal with many thousands of threads. The JVM gets
away with this by not having a 1:1 mapping to native threads on systems
where they're expensive. The primary limiting factor is how much memory
you're holding on to when the thread blocks. You may need to null
unused local variables before blocking to get past a few hundred threads.

Check the thread group when you're running under Tomcat. Some thread
groups shouldn't be interfered with.

 

[bugbear]

Define crashing.

"thrashing"; high kernel load on Solaris (as indicated by Solaris
performance monitoring tools) traced to process swapping
in the JVM.

Usually the JVM can deal with many thousands of threads.

Interesting; the "standard" config file for tomcat has a "max" of 150.

> The JVM gets
away with this by not having a 1:1 mapping to native threads on systems
where they're expensive. The primary limiting factor is how much memory
you're holding on to when the thread blocks. You may need to null
unused local variables before blocking to get past a few hundred threads.

Check the thread group when you're running under Tomcat. Some thread
groups shouldn't be interfered with.

Pardon my ignorance; that's a "thread group"

BugBear

 

[bugbear]

Pardon my ignorance; that's a "thread group"

Bad typo!

what's a "thread group" ?

BugBear

 

[Joshua Cranmer]

Bad typo!

what's a "thread group" ?

BugBear

A group of threads, as implemented by java.lang.ThreadGroup. They'll
have some higher-order cohesiveness, e.g., grouping a series of worker
threads together.

 

[Kevin McMurtrie]

Bad typo!

what's a "thread group" ?

BugBear

The Tomcat thread group is a ThreadGroup instance. It makes it easier
to act on many threads at once.

The other thing to check is the native options:

-Xss<size>
Stack size per thread.


-XX:+UseBoundThreads
Forces 1:1 thread ratio. Probably not what you want.

-XX:-UseLWPSynchronization

 

[Roedy Green]

Does anybody have any advice, experience, or code
to reduce this number.

some possibilities:

1. write the shelled out code in Java and just call it directly.

2. tell the shelled out code to use files for in/out.. Prepare the
input file ahead of time and read the output file after termination.

3. tell the shelled out code to communicate via transactions that look
just like requests coming in over the web.
see http://mindprod.com/jgloss/http.html

4. create a JNI hook to the shelled out code. see
http://mindprod.com/jgloss/jni.html

5. create a queue to a thread pool so that you don't spawn too many
shells at once. see http://mindprod.com/jgloss/queue.html

 

[Arne Vajhøj]

Usually the JVM can deal with many thousands of threads. The JVM gets
away with this by not having a 1:1 mapping to native threads on systems
where they're expensive.

Are any JVM's besides older JRockit's with -Xthinthreads actually
doing that ?

Arne