Java 1.5 thread tuning question

[cswanson]

I recently ported a high-transaction application I wrote in Java 1.3
to 1.5 and discovered a possible thread issue. The evidence suggests
that high thread generation such as 800 threads in three seconds will
cause the JVM to throw the following error:

java.lang.OutOfMemoryError: unable to create new native thread

This does not happen on Java 1.3.

Therefore, I anticipate there must be some tuning values I can send to
the JVM that I am unaware of.

I invoke the JVM with the following flags:

-Xmx512m -Xms64m

Everything else would be default.

I run this application on Linux and Solaris. The thread issue has been
observed only on Linux with jdk-1_5_0_06-linux-i586.bin from Sun as I
haven't executed the application using 1.5 on Solaris.

Any help would be appreciated.

Chris

 

[Daniel Dyer]

I recently ported a high-transaction application I wrote in Java 1.3
to 1.5 and discovered a possible thread issue. The evidence suggests
that high thread generation such as 800 threads in three seconds will
cause the JVM to throw the following error:

java.lang.OutOfMemoryError: unable to create new native thread

This does not happen on Java 1.3.

Therefore, I anticipate there must be some tuning values I can send to
the JVM that I am unaware of.

I invoke the JVM with the following flags:

-Xmx512m -Xms64m

Everything else would be default.

I run this application on Linux and Solaris. The thread issue has been
observed only on Linux with jdk-1_5_0_06-linux-i586.bin from Sun as I
haven't executed the application using 1.5 on Solaris.

This is most likely an operating system issue. Did you run the 1.3
version on the same machine as the 1.5 version?

Take a look at the Linux "ulimit" command.

Dan.

 

[cswanson]

This is most likely an operating system issue. Did you run the 1.3
version on the same machine as the 1.5 version?

Take a look at the Linux "ulimit" command.

Dan.

Thanks for the input.

Yes, I did run the 1.3 version on the same machine. As for the ulimit,
the same shell environment invoked both Java virtual machines. In fact
the only real difference between the versions was the URLDecoder and
URLEncoder classes as they both want encoding Strings where in version
1.3 the classes didn't. Otherwise, the code, environment, operating
system, and hardware are identical.

Chris

 

[Patricia Shanahan]

Thanks for the input.

Yes, I did run the 1.3 version on the same machine. As for the ulimit,
the same shell environment invoked both Java virtual machines. In fact
the only real difference between the versions was the URLDecoder and
URLEncoder classes as they both want encoding Strings where in version
1.3 the classes didn't. Otherwise, the code, environment, operating
system, and hardware are identical.

Chris

Have you tried adjusting the number of threads to measure the extent of
the problem?

If slightly more threads on the earlier version breaks, and slightly
fewer threads on the later version works, it may be simply the general
tendency of software to grow with age.

Patricia

 

[Daniel Dyer]

Thanks for the input.

Yes, I did run the 1.3 version on the same machine. As for the ulimit,
the same shell environment invoked both Java virtual machines. In fact
the only real difference between the versions was the URLDecoder and
URLEncoder classes as they both want encoding Strings where in version
1.3 the classes didn't. Otherwise, the code, environment, operating
system, and hardware are identical.

I would speculate that 1.5 has a different strategy for mapping Java
threads to native threads than 1.3. I think there have been at least 3
different implementations used by Sun's JVMs. Prior to version 1.3, the
Linux JVM used "green threads". I don't recall how things have changed
since then.

What does "ulimit -a" say? I'm betting it's less than 800, so if there is
a difference in threading implementation you could run up against this
limit.

Alternatively, if the error really is what it says, i.e. a lack of memory,
you could try dropping the thread stack size with the -Xss switch. I'm
not convinced this will make much difference since 800 threads would only
use 200mb for the thread stacks. You shouldn't start getting this kind of
error until the sum of thread stacks and heap space exceeds 2 gigabytes
(on a 32-bit machine).

Dan.

 

[Robert Klemme]

Have you tried adjusting the number of threads to measure the extent of
the problem?

If slightly more threads on the earlier version breaks, and slightly
fewer threads on the later version works, it may be simply the general
tendency of software to grow with age.

Good point! Another idea: the newer JVM could be more optimized thusly
yielding a higher thread creation rate. If those threads are short
lived this could lead to earlier resource exhaustion.

Another difference might be that the newer JVM allocates more resources
for threads, especially stacks. You could try to explicitly set the
same thread stack size via -Xss<size> as Daniel suggested and see how
things go.

Kind regards

robert