Java 5 threads in linux

[Sunny]

Hi,

I am using the following java version in Linux

java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Server VM (build 1.5.0_06-b05, mixed mode)

Threads are created using the source code:
private static ExecutorService thPool =
Executors.newFixedThreadPool(5);

My question is how threads are created in linux using the mentioned
runtime.
1. Does the code line above create 5 linux native threads (process)?
2. Does it create java threads? Don't know if java and native threads
are same.
3. If answer of the question 1 is "NO" then does one native linux
threads for java runtime do all the concurrent execution?

Please explain. Any documents on java threads in Linux for java 5?

Thanks in advance

Sunny

 

[Martin Gregorie]

Threads are created using the source code:
private static ExecutorService thPool =
Executors.newFixedThreadPool(5);

My question is how threads are created in linux using the mentioned
runtime.
1. Does the code line above create 5 linux native threads (process)?
2. Does it create java threads? Don't know if java and native threads
are same.
3. If answer of the question 1 is "NO" then does one native linux
threads for java runtime do all the concurrent execution?

If you'd started that code (or ANY Java program) and then run ps from
another window you'd already know that the answer is (1).

 

[Tom Hawtin]

(Why update 6? What are we on, update 11?)

If you'd started that code (or ANY Java program) and then run ps from
another window you'd already know that the answer is (1).

The output from ps would change depending upon whether the machine was
running a 2.4 or 2.6 series kernel.

The suck it and see approach will tell you one possible behaviour, but
it doesn't tell you what might happen in different circumstance. For
instance, some JVMs might appear to start off with a 1-1 mapping between
native and Java threads, but then latter start sharing native threads.

Tom Hawtin

 

[Martin Gregorie]

(Why update 6? What are we on, update 11?)


The output from ps would change depending upon whether the machine was
running a 2.4 or 2.6 series kernel.

The suck it and see approach will tell you one possible behaviour, but
it doesn't tell you what might happen in different circumstance. For
instance, some JVMs might appear to start off with a 1-1 mapping between
native and Java threads, but then latter start sharing native threads.

The output from ps might be different but the mapping of threads to
processes doesn't appear to change between 2.4 and 2.6.

IMO the OP should have done some investigation and then asked if he
couldn't understand what he saw.

 

[Sunny]

(Why update 6? What are we on, update 11?)

Yes, update 6. I am still with the old one.

instance, some JVMs might appear to start off with a 1-1 mapping between
native and Java threads, but then latter start sharing native threads.

I am relatively new in java and know little about the history of JVM.
In a previous document SUN has provided the following information for
java version "1.3.0" in Linux platform (from
http://java.sun.com/developer/technicalArticles/Programming/linux/ ):

"In Java 2 Release 1.3, the Hotspot virtual machine uses system
threads to implement Java threads. Because Linux threads are
implemented as a cloned process, each Java thread shows up in the
process table if you run the ps command. This is normal behavior on
Linux."

As I have started with the version 1.5, don't know if the native
threads support is still threre in this version. Is there any doc from
SUN or, other to describe the threads behaviour in new version of JVM
in Linux platform?

Surely I can use 'ps' command to see if the code line creates
processes. The problem is elsewhere. While I was using the
ExecutorService high level API to manage threads, I got an error which
states "unable to create new native thread". The exact message is
"java.lang.OutOfMemoryError: unable to create new native thread".
Native thread must be linux system threads (may be POSIX). But the
'ps' command is not showing multiple JVM threads.

Either the error message is wrong when JVM is running in Linux system
or, 'ps' command is not showing up the java native threads correctly.

Please share your experience on it.

Sunny

 

[Lew]

I am relatively new in java and know little about the history of JVM.
In a previous document SUN has provided the following information for
java version "1.3.0" in Linux platform (from
http://java.sun.com/developer/technicalArticles/Programming/linux/ ):

Java 1.3 is obsolete.

Nothing about the Java 1.3 JVM can be used to draw conclusions about the
current Sun implementation, much less anyone else's, wrt thread-to-process
mapping.

As I have started with the version 1.5, don't know if the native
threads support is still threre in this version.

"Still" there? The point is that Java 1.3 did not have native thread support,
but mapped threads to processes in Linux.

Is there any doc from SUN or, other to describe the threads behaviour in new version of JVM
in Linux platform?

Surely I can use 'ps' command to see if the code line creates
processes. The problem is elsewhere. While I was using the
ExecutorService high level API to manage threads, I got an error which
states "unable to create new native thread". The exact message is
"java.lang.OutOfMemoryError: unable to create new native thread".
Native thread must be linux system threads (may be POSIX). But the
'ps' command is not showing multiple JVM threads.

With what options did you invoke 'ps'?

Either the error message is wrong when JVM is running in Linux system
or, 'ps' command is not showing up the java native threads correctly.

You haven't told us what you regard as "correctly" nor how you ran "ps", nor
precisely what the output was, nor precisely how it differed from what you
want. That information would be useful for anyone wishing to comment on your
situation.

Please share your experience on it.

I tried this on my Linux system:

$ ps -C java -o "pid,lwp,comm"
PID LWP COMMAND
4848 4848 java
5987 5987 java

Then this:

$ ps -C java -Lo "pid,lwp,comm"
PID LWP COMMAND
4848 4848 java
4848 4849 java
4848 4850 java
4848 4851 java
4848 4852 java
4848 4853 java
4848 4854 java
4848 4855 java
4848 4856 java
4848 4857 java
4848 4858 java
4848 4859 java
4848 4860 java
4848 4861 java
4848 4863 java
...

Is this anything like what you experienced?

 

[alexandre_paterson]

java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Server VM (build 1.5.0_06-b05, mixed mode)

Sun VM, so, yup, one native Linux thread per Java thread. But they
are "hidden".

At one point if I'm not mistaken IBM had a very creative Java VM for
Linux where things were different. Then when "NPTL" (Native Posix
Thread Library") came along for kernel 2.4 (not all kernel 2.4 have
NPTL),
IBM dropped that very creative VM.

Anyway, Linux threads got faster and faster. From kernel 2.4 to 2.6
we
went from mutex to futex to "robust futex" (for, hey, basic futex were
actually buggy but then they were 'robust' but weren't that fast
anymore (the 'f' in "futex" stands for fast) so now we have...
"robust lightweight futexes".

I'm not kidding

And processes (and especially processes creation) are hyper-efficient
under Linux: this is something Windows-centric programmer usually
have a hard-time understanding.

On a modern Linux box (i.e. a 2.4 with NPTL or later kernel) you
won't see at first all the processes corresponding to Java threads:
they are "hidden".

You can find them using the "-L" switch to the 'ps' command (as
shown by Lew), but also in /proc/{pid}, if you look carefully.
The PIDs for the various threads are hidden, yet they exist.

You've got to look in /proc/{pid}/task/ to find the number of all
the hidden PIDs.

For example :

[alex@saturne 6.7G ~/] # ps aux | grep java
alex 31103 1.9 7.2 541984 263172 pts/7 Sl+ Apr21 1:27 /home/
alex/jdk1.6.0/bin/java -Xms192m -Xmx256m -XX:MaxPermSize=99m -ea -
server -Dsun.awt.keepWorkingSetOnMinimize=true -Xbootclasspath/p:../
lib/boot.jar: com.intellij.idea.Main
PID TTY TIME CMD
31103 pts/7 00:01:27 java

[alex@saturne 6.7G ~/] $ ls /proc/31103/
attr cmdline cwd exe loginuid mem oom_adj root
smaps statm task
auxv cpuset environ fd maps mounts oom_score seccomp
stat status wchan

[alex@saturne 6.7G ~/] $ ls /proc/31103/task/
31103 31110 31113 31116 31120 31124 31128 31135 31154 31161
31165 31171 31174 31184
31108 31111 31114 31117 31121 31125 31129 31136 31157 31163
31169 31172 31175
31109 31112 31115 31118 31122 31126 31134 31137 31160 31164
31170 31173 31181

The following one is interesting: by looking into /proc/ you won't
find, say, 31108...
But you know it exists from looking into the main java process's /proc/
{pid}/task/ subdir.

[alex@saturne 6.7G ~/] $ ls /proc/31108/
attr cmdline cwd exe loginuid mem oom_adj root
smaps statm task
auxv cpuset environ fd maps mounts oom_score seccomp
stat status wchan


Alex

 

[alexandre_paterson]

I forgot to add that you can easily see if you're
using NPTL or not and you can force a process
to use the old LinuxThreads (you'll be killing
the perfs if you do so) if you want to:

....$ uname -a
Linux saturne 2.6.16.33 #1 SMP Fri Mar 30 04:13:58 BST 2007 i686 GNU/
Linux
....$ getconf GNU_LIBPTHREAD_VERSION
NPTL 2.3.6
....$ export LD_ASSUME_KERNEL=2.4.1
....$ getconf GNU_LIBPTHREAD_VERSION
linuxthreads-0.10

If you're not familiar with Unix systems: don't worry about
the "export" : the changes won't be permanent.

 

[Sunny]

...$ uname -a
Linux saturne 2.6.16.33 #1 SMP Fri Mar 30 04:13:58 BST 2007 i686 GNU/
Linux

my linux:
$ uname -a
Linux leo 2.4.21-37.ELsmp #1 SMP Wed Sep 7 13:28:55 EDT 2005 i686 i686
i386 GNU/Linux

...$ getconf GNU_LIBPTHREAD_VERSION
NPTL 2.3.6

$ getconf GNU_LIBPTHREAD_VERSION
NPTL 2.3.4

Many thanks Alex making it clear to me. I am getting those now using
ps -C java -mo "pid,ppid,cmd"

With what options did you invoke 'ps'?

ps aux | grep java
ps -ef | grep java
ps -AH | grep java

Thanks all for sharing your knowledge.

Sunny