Multithreaded Throughput Degradation in Java/Oracle Application

[Robert Brown]

I have a multithreaded Java application which a part of a standard
three tier architecture. There are some users in the database and each
user can request his account details (such as the items they've
bought). Each user request is executing on a separate thread borrows a
db connection from the db pool, retrieves the data (via a select),
performs some computations and returns the results to the user. I am
using PreparedStatements, reusing connections etc. The db pool is
sized to allow each thread to have its own connection so that at no
time do threads have to wait for a connection to become available. The
JVM is using native threads (jdk1.3).

In these conditions when running a benchmark test on a 4CPU box I
would expect the throughput (total ops per minute) to grow linearly as
go from 1 concurrent thread to 2,3,4 concurrent threads.

With 1 thread I can call retrieveAccountInfo() about 500 times a
minute. I would expect with 4 threads to be able to execute a total of
2000 calls a minute across all threads since each thread can run on a
different CPU and the application is completely parallel (that is, no
synchronization points and no contention for the db connections).
However with 4 threads I am only able to execute about 850 calls a
minute which is a really disappointing scalability.

Given the scenario described above would you expect linear throughput
increase as long as there are as many CPUs as there threads (no
contention for the CPU)?

If not why not?

If all the conditions described above hold, what other factors may be
inhibiting scalability? How can I find out? That is, what OS commands
can I run to see what the choke point is?

Thanks so much!

- robert

 

[Bob Kranson]

You can look at the Java and Oracle vendor sites for some tuning
information. These are some suggestions.

Did you enable the JVM for concurrent garbage collection?
Add any JVM parameters to allow the maximum heap for your RAM install?
Is the JVM set with a '-server' configuration?

Maybe an analysis tool can help?
http://www.compuware.com/products/devpartner/java.htm

Regards,
BobK

 

[Robert Brown]

So if the application I described were ideally tuned would you expect
to have linear throughput improvement when you increase the number of
threads from 1 to 4 on a 4-CPU box?

 

[Bob Kranson]

I would first ask which operating system you are using....Windows, Unix,
Mainframe, AS400, VMS? Each has its own implementation of the JVM and its
own multi-processor logic. Each of these OS's have system-kernal settings
which can 'ulimit' files, processes, etc. These are usually hidden
adversaries to the greatest performance intentions!@

 

[Roedy Green]

Each of these OS's have system-kernal settings
which can 'ulimit' files, processes, etc.

Ulimit = user limit? In other words, unbeknownst to you, you are
queuiing up to access a common file?

Is there any I/O going on OUTSIDE SQL? You could find out by
temporarily moving the SQL server to another machine and monitoring
i/o.

 

[Harald Hein]

So if the application I described were ideally tuned would you expect
to have linear throughput improvement when you increase the number of
threads from 1 to 4 on a 4-CPU box?

Never. Already the hardware architecture will prevent this.
Communication overhead will prevent this, I/O will prevent this.