Hox to match Linux PID to tomcat thread

[tkonrath]

Hi.

On our production server (Suse Linux) we run a Tomcat server. From
time to time, our users tell us, that our web application is running
slow.

Starting the command "top" I see that one Tomcat-Process is running
consuming a lot of process time from the server. So I know the process
ID of the thread, but how can I match the PID to a Tomcat Thread?

I have also tried to do a Java dump but I was not able to find the
according Tomcat Thread.

Is there a way to get the right Tomcat Thread from the process ID?
Or is there a way from within a Tomcat Thread to get its process ID?

I have searched in Google but I didn't found any answer to this
problem ...

Please Help, Thanks.

Thomas Konrath

 

[Thomas Fritsch]

Hi.

On our production server (Suse Linux) we run a Tomcat server. From
time to time, our users tell us, that our web application is running
slow.

Starting the command "top" I see that one Tomcat-Process is running
consuming a lot of process time from the server. So I know the process
ID of the thread, but how can I match the PID to a Tomcat Thread?

I have also tried to do a Java dump but I was not able to find the
according Tomcat Thread.

Is there a way to get the right Tomcat Thread from the process ID?
Or is there a way from within a Tomcat Thread to get its process ID?

I have searched in Google but I didn't found any answer to this
problem ...

I found "How to generate a Thread Dump"
<http://qa.netbeans.org/bugzilla/generating-thread-dumps.html>

I tested the StackTrace tool (the JNLP link near the bottom of that page),
and successfully grabbed a Java ThreadDump of a large Java-server-process
running on my Linux.
(Remember that your Tomcat-process is probably running with user-id root.
You therefore need to run your browser from root, so that the launched
JavaWS-application will run from root too, and hence has permission for
dumping the Tomcat-process.)
You should get the stack traces of all Tomcat threads. Watch out for busy
threads, i.e. the ones not idling in method Object.wait() or
Thread.sleep() .

 

[alexandre_paterson]

I tested the StackTrace tool

Ah, thanks for the link : it does a little bit more than simply
dumping the threads. It may be handy.

(Remember that your Tomcat-process is probably running with user-id root.
You therefore need to run your browser from root, so that the launched
JavaWS-application will run from root too, and hence has permission for
dumping the Tomcat-process.)

No no no! A production server running Tomcat as root is a
very bad idea.

On Linux not only can you run Tomcat as a non-root
user but you can also install the whole JDK as a non-root
user (on Windows last time I checked you needed to
have admin privs to install Java).

I said "can" though I really meant "should".

It is trivial on a Linux system to have a non-privileged process
listen to a port > 1024 (a non-privileged can't listen on port <
1024)
and then have the stateful firewall transparently redirect requests
to the non-privileged port.

If a production Tomcat on a Linux server is "probably running
as root" then something is very wrong :-/

Not a single security conscious Unix admin would ever
allow this for a production server.

Talk to you soon on c.l.j.p.,

Alex

 

[alexandre_paterson]

On Apr 19, 9:32 am, (e-mail address removed) wrote:
....

On our production server (Suse Linux) we run a Tomcat server. From
time to time, our users tell us, that our web application is running
slow.

How many users? How slow? Do you think you're hitting some
kind of I/O bottleneck (network / disk) or do you think it's more of
a performance problem in the way the Webapp is programmed?

Starting the command "top" I see that one Tomcat-Process is running
consuming a lot of process time from the server. So I know the process
ID of the thread,

No, you don't know the process ID of the thread. You know the process
ID of the Tomcat process.

but how can I match the PID to a Tomcat Thread?

If you're running a non-antique Linux version there's no
one-to-one mapping between threads and processes. It
used to be like this in kernel 2.4 non-NPTL and older...

But it's not like that anymore.

I have also tried to do a Java dump but I was not able to find the
according Tomcat Thread.

Is it always slow or only occasionally? Did you allocate a lot of
memory to Tomcat? (if so the GC kicking in may wreak havoc
and you'd be better to reconfigure the type of GC used)

Depending on why you're experiencing slowdown, you may
be better with a Java Webserver using multiplexing and NIO.

Many people are saying good things about Resin.

Of course the best Java Web server in the world won't
do much good if the slowness is due to a problem in
the way the Webapp itself is programmed.

If it was to me I'd first try to reproduce the problem
in a similar environment: install same Linux + JDK +
Tomcat + Webapp then do some load-testing to try
to see what's going on.

The best, non-intrusive, way to do real-time profiling
on a production server would be to run DTrace on
(Open)Solaris... But you'll have to wait some more
before that tool (and its Java probes) are ported to
Linux.

Good luck on finding the bottleneck and don't
hesitate to report once you solve the problem,

Alex


P.S: you may want to play with the memory settings
of the JVM and Tomcat's server.xml and see if it helps.

 

[Brandon McCombs]

Hi.

On our production server (Suse Linux) we run a Tomcat server. From
time to time, our users tell us, that our web application is running
slow.

Starting the command "top" I see that one Tomcat-Process is running
consuming a lot of process time from the server. So I know the process
ID of the thread, but how can I match the PID to a Tomcat Thread?

Processes are not the same thing as threads. By using top you have the
process ID of the process that contains multiple threads for Tomcat. If
you kill that process all the threads go away. You may need to look into
how OSes work when you have some spare time after fixing this problem.
Without the tool mentioned by Thomas you won't be able to determine the
individual Tomcat thread identifiers (if there are any to begin with)
unless you wanted to hack the kernel to provide that information.

I have also tried to do a Java dump but I was not able to find the
according Tomcat Thread.

Is there a way to get the right Tomcat Thread from the process ID?

No because you don't have further information to go on to determine
which thread within that process is the one you want. The process
contains multiple threads and the process ID is just a number that only
matches up with the process, hence the name.

 

[Thomas Fritsch]

No no no! A production server running Tomcat as root is a
very bad idea.

Aha, good to know!
But even then it might be necessary to run the StackTrace tool from the
same user-id as the tomcat process (in order to have the permission for
dumping the process).

 

[ajX]

Here is a shell script to take and save thread dump in a text file. I have written a shell script to take the thread dump automatically.

visit : http://www.technotechi.com/2012/12/shell-script-to-perform-thread-dump.html

It might help you.