JSP/Servlet performance related question

[itreflects]

Hello,
I have JSP performance related concern.

Here is the situation: My Database table has 10,000 records, I fetch
these records in a servlet and do all the necessary calculation on each
record and come up with a set of say 10 parameters for each record. All
these 10 parameters per record are displayed in a JSP and 10,000 are
iterated in JSP and shown in HTML table.
In servlet class I am storing all 10 parameters in a Vector object and
adding this Vector object to an ArrayList object. This ArrayList object
I am putting in session (HttpSession) to make it available to JSP page.
All is well, page works fine.Concern is that it takes around 3 minutes
to load onto browser.
My questions are:

How can time to load the JSP page be reduced?
Is storing such a large ArrayList object in session a good practice?
what are the alternatives to make huge data available to JSP?


Any comments are welcome.

Note: I am Using server with 2GB RAM and 2.4 GHz processor, app server
is Tomcat 5.5.Don't think increasing RAM would help.

 

[Eric Sosman]

Hello,
I have JSP performance related concern.

Here is the situation: My Database table has 10,000 records, I fetch
these records in a servlet and do all the necessary calculation on each
record and come up with a set of say 10 parameters for each record. All
these 10 parameters per record are displayed in a JSP and 10,000 are
iterated in JSP and shown in HTML table.
In servlet class I am storing all 10 parameters in a Vector object and
adding this Vector object to an ArrayList object. This ArrayList object
I am putting in session (HttpSession) to make it available to JSP page.
All is well, page works fine.Concern is that it takes around 3 minutes
to load onto browser.
My questions are:

How can time to load the JSP page be reduced?
Is storing such a large ArrayList object in session a good practice?
what are the alternatives to make huge data available to JSP?


Any comments are welcome.

The first step I'd suggest is to locate the bottleneck.
You seem to be concentrating on the server, but it's not out
of the question that a browser might take quite a while to
render a table of 100,000 cells. A simple test would be to
save the generated HTML to the browser's local disk, shut
down and restart the browser (to dodge cache effects), and
then time how long it takes to do an "Open File..." on the
saved page. If it's three minutes, you need to think about
writing better HTML (or making a smaller table). If it's
three seconds, then you might start looking at the network
speed and at the server.

But first things first: Build the wolf fence.

 

[Manish Pandit]

You could consider paginating the application, and giving a search
functionality on those 10 parameters for random access.

This way you will not have to push a gigantic arraylist with 10,000
vectors of 10 elements each. This will save your database fetching
time, data manipulation time and HTML rendering time.

I do not know how usable a UI with 10,000 rows would be. You can check
with your users if they would be okay with pagination.

-cheers,
Manish

 

[Wesley Hall]

How can time to load the JSP page be reduced?

First, by finding the problem. I would suspect the database if you are
doing anything other than a simple, "Select * from Table". Put some log
statements (or simple System.out.println calls) at strategic parts of
your application and narrow down the areas that are taking the most
time. Then you wont have to waste your time trying to optimize parts of
the code that already have satisfactory performance.

Is storing such a large ArrayList object in session a good practice?

No, not really. Mostly because it will make any future clustering effort
impractical. I dont understand why you are using the session anyway, the
data would be request scope wouldn't it?

If you are storing it in the session for caching purposes, it would
probably be better to use a proper cache in the middle-tier. There are
many open source caches available.

what are the alternatives to make huge data available to JSP?

You should probably consider paginating the data (like google search
results). It is unlikely that your users can consume 10'000 pieces of
data at once.

 

[itreflects]

Paginating is one option I am considering. No database fectching is
quick (approx.100 ms). And data is not request specific, it needs to be
available throughout the session.My resultset from database is
disconnected one (javax.sql.rowset.CacheRowSet) i.e I make select query
on database only once and close the connection. Hence,even if I
paginate result across multiple pages, I do need to keep the entire
result either in session or in a persistent data storage such as file
(i.e serialize ArrayList object into temp file), just because the whole
result need to be given as a file download in csv format.
It is JSP page that is taking longer to load onto browser.

 

[Wesley Hall]

Paginating is one option I am considering. No database fectching is
quick (approx.100 ms). And data is not request specific, it needs to be
available throughout the session.My resultset from database is
disconnected one (javax.sql.rowset.CacheRowSet) i.e I make select query
on database only once and close the connection. Hence,even if I
paginate result across multiple pages, I do need to keep the entire
result either in session or in a persistent data storage such as file
(i.e serialize ArrayList object into temp file), just because the whole
result need to be given as a file download in csv format.
It is JSP page that is taking longer to load onto browser.

I recommend against top-posting. It is generally frowned upon.

On second thoughts, I guess it is the download time that is taking so
long. I didn't think it through properly before but I guess your
generated HTML is several MB with 10'000 results.

Aside from the delay the first time you access a JSP, there should be no
major delay in running the JSP.

Paginating will probably solve your issue.

 

[itreflects]

I recommend against top-posting. It is generally frowned upon.

On second thoughts, I guess it is the download time that is taking so
long. I didn't think it through properly before but I guess your
generated HTML is several MB with 10'000 results.

Aside from the delay the first time you access a JSP, there should be no
major delay in running the JSP.
Paginating will probably solve your issue.

fetch 10k results. So, basically how do I make 10K results available @
100 results/page. So, concern boils down to, how can a large data be
made available across multiple page requests
with only one DB call (i.e very first time user requests results I make
a DB call after that for 2nd page of 101-200 th rows I want to fetch
from the cached result set).

 

[Lew]

Paginating is one option I am considering. No database fectching is
quick (approx.100 ms). And data is not request specific, it needs to be
available throughout the session.My resultset from database is
disconnected one (javax.sql.rowset.CacheRowSet) i.e I make select query
on database only once and close the connection. Hence,even if I
paginate result across multiple pages, I do need to keep the entire
result either in session or in a persistent data storage such as file
(i.e serialize ArrayList object into temp file), just because the whole
result need to be given as a file download in csv format.

What is the database itself but "persistent data storage"?

Doesn't it seem just a bit redundant to pull data out of one persistent store
and hold it in a different (slower) persistent store for later use?

Perhaps you should reconsider creation of queries that work on fragments of
your overall result, so that whatever you are displaying at a given moment is
found in the database section by section. It could actually be cheaper overall
to repeate queries, or nearly repeat them for stripes of the overall result,
than it is to keep a huge magilla in the session.

- Lew

 

[Manish Pandit]

on database only once and close the connection. Hence,even if I

paginate result across multiple pages, I do need to keep the entire
result either in session or in a persistent data storage such as file

No, you do not need to keep entire result in the session. By pagination
I meant paginating at the query level. You can have a pagination strip
which shows the number of pages and an ability to navigate by clicking
on the page numbers, or next/previous links ( totally up to you - how
you want to design the UI). These links translate to 2 parameters in
the query, which control the number of rows fetched. In MySQL, the
query can have a limit clause (limit x,y) where x= start row and y =
number of records to be returned. This way even though you'd hit the db
n/m times, where n = total records and m = records per page, your
sessions will be much lightweight and you will not generate as much
network traffic to render the markup.

-cheers,
Manish

 

[itreflects]

No, you do not need to keep entire result in the session. By pagination

I meant paginating at the query level. You can have a pagination strip
which shows the number of pages and an ability to navigate by clicking
on the page numbers, or next/previous links ( totally up to you - how
you want to design the UI). These links translate to 2 parameters in
the query, which control the number of rows fetched. In MySQL, the
query can have a limit clause (limit x,y) where x= start row and y =
number of records to be returned. This way even though you'd hit the db
n/m times, where n = total records and m = records per page, your
sessions will be much lightweight and you will not generate as much
network traffic to render the markup.

-cheers,
Manish

This approach sounds good to me.Largely we can conclude that any and
every data that needs to be made available to JSP page can be broken
into smaller data so that we can have lightweight session.
One curious question, it is possible to read the HTML table contents in
Javascript by using table id, is it possible to create a temporary
file using JavaScript and give that file for download to browser?
probably I should post this question in Javascript group. But anyway
thanks for comments.