java.lang.OutOfMemory Error

[puneet.bansal]

I had used StringBuffer, as with 'out.println()' I will have to use +
operator for all the concatenation which I thought won't be that
efficient as using the StringBuffer.

 

[Ben_]

There is no need for you to concatenate. See my dummy sample has no "+" or
".append()".

 

[Ben_]

I don't know what your application does and I didn't react on this before,
but honestly, I can't imagine someone will ever spend time reading (and even
waiting for) a page showing 50,000 records... The page will be MB's large,
and it would take ages to render. From a usability point of view, a
scrollable list is preferred.

Depending on the RDBMS, you can make queries to select the "n items at
position y" (like the 10 records from 50 to 60). I'm not an SQL specialist,
but I believe these are extensions to plain SQL, but I think it's a fair
trade-off to break database portability for usability.

I was reading that PDF incidentally (http://www.nyoug.org/javasets.pdf)
while looking at how to implement efficiently a JDBC query with a large
ResultSet. Interestingly, it says that the ScrollableResultSet will cache
the data as it is being read (slide 23). It goes about Oracle, but I'd
suppose all implementations do it that way. This means that as your loop is
looping you are caching more and more data. Not sure, but it could be
that you end up caching the 50,000 rows in your servlet...

 

[puneet.bansal]

The page does not take much time to load. I don't show it as an HTML
page even though I generate an HTML response. I change the header to
XLS. So the user gets it as an Excel report.
I clearly asked the users, whether they will be willing to see such a
huge report and they said yes and they are happy with the report that
gets generated.

 

[puneet.bansal]

I fixed the problem. I was using a scrollable resultset which means
that I can move back and forth among the rows and hence the driver has
to keep in memory all the rows. I changed my printing logic to make use
of the previous row rather than the next row in if blocks. Now, I don't
need a scrollable resultset and just a FORWARD_ONLY resultset is
enough. For the same report now the server memory goes up from 90MB to
just 98MB !!

Puneet

 

[Ben_]

Mmh, this was a good programming lesson, wasn't it ?