heap memory

[Andrea]

Hi all,
I've a jsp-form which inquiry a database with a sql-query, and obtain a
recordset placed in a type-Vector variable; all working fine when I expose
this result in a html-page.
Now I need to expose same result in a Excel worksheet, so now my jsp contain
this code string:
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-disposition",
"attachment;filename=report_wiz.xls");
All working fine until (I suppose) Vector extracted is "light", but if it's
"heavy" in its dimension then I have an error page instead my xls sheet with
data inside.
I can expand my java-container with this parameter:
-Xms512m -Xmx512m
but could be not sufficient!
So, there is a way to intercept Vector-dimension? If yes, I can check this
before pass it to Excel-page, and if it's too big I can throw an alert like
"too much fields or too much rows extracted" to my users, instead a bad
error-page.
Thanks in advance
JFM

 

[Daniel Pitts]

Hi all,
I've a jsp-form which inquiry a database with a sql-query, and obtain a
recordset placed in a type-Vector variable; all working fine when I expose
this result in a html-page.
Now I need to expose same result in a Excel worksheet, so now my jsp contain
this code string:
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-disposition",
"attachment;filename=report_wiz.xls");
All working fine until (I suppose) Vector extracted is "light", but if it's
"heavy" in its dimension then I have an error page instead my xls sheet with
data inside.
I can expand my java-container with this parameter:
-Xms512m -Xmx512m
but could be not sufficient!
So, there is a way to intercept Vector-dimension? If yes, I can check this
before pass it to Excel-page, and if it's too big I can throw an alert like
"too much fields or too much rows extracted" to my users, instead a bad
error-page.
Thanks in advance
JFM

First, you don't tell us what database interface you are using, JDBC?
iBATIS? Hibernate?

Some data access frameworks allow you to use lazy loading, and/or row
handling. This allows you to only load into memory what you need.

Alternatively, you can add a new SQL query that is like your original
query, but returns the number of rows.

Original: SELECT a, b, c FROM table_a WHERE a=b+1 <etc...>
Counting: SELECT COUNT(*) FROM table_a WHERE a=b+1 <etc...>

 

[Andrea]

First, you don't tell us what database interface you are using, JDBC?
iBATIS? Hibernate?

Some data access frameworks allow you to use lazy loading, and/or row
handling. This allows you to only load into memory what you need.

Alternatively, you can add a new SQL query that is like your original
query, but returns the number of rows.

Original: SELECT a, b, c FROM table_a WHERE a=b+1 <etc...>
Counting: SELECT COUNT(*) FROM table_a WHERE a=b+1 <etc...>

Hi Daniel,
sorry, I have a JDBC interface to an Oracle database.
And about second suggestion, number and type of field selected can be
changed at each inqury, so a "select count(*)" not solve my trouble. Two
example: I can exctract 50000 rows with only a numeric field (and this works
fine), or 20000 rows composed by twenty fields (some numeric, some
varchar)... and is exactly this last event which cause my trouble.
Two seconds ago, I find this method:
vector.capacity()
now I go to search documentation about in order to see if it can help me; do
you know if this method could be ad-hoc for my problem?
Thanks again
JFM

 

[Daniel Pitts]

"Daniel Pitts" <[email protected]> ha scritto nel messaggio







Hi Daniel,
sorry, I have a JDBC interface to an Oracle database.
And about second suggestion, number and type of field selected can be
changed at each inqury, so a "select count(*)" not solve my trouble. Two
example: I can exctract 50000 rows with only a numeric field (and this works
fine), or 20000 rows composed by twenty fields (some numeric, some
varchar)... and is exactly this last event which cause my trouble.
Two seconds ago, I find this method:
vector.capacity()
now I go to search documentation about in order to see if it can help me; do
you know if this method could be ad-hoc for my problem?
Thanks again
JFM

If you have control of the code that adds the data into the Vector,
change it to instead process it as a stream of rows! Many people find
it easier conceptually to deal with a Vector, however, often times you
can minimize memory requirements by changing your algorithms to
stream.

Well, hope this helps.
Daniel.

 

[Andrea]

If you have control of the code that adds the data into the Vector,
change it to instead process it as a stream of rows! Many people find
it easier conceptually to deal with a Vector, however, often times you
can minimize memory requirements by changing your algorithms to
stream.

Well, hope this helps.
Daniel.

Hi Daniel,
thanks for your suggestion, very interesting!
Please, have you an example about this method? Or can you tell me an url
where I can find documentation about it?
Thanks again
JFM

 

[Daniel Pitts]

Hi Daniel,
thanks for your suggestion, very interesting!
Please, have you an example about this method? Or can you tell me an url
where I can find documentation about it?
Thanks again
JFM

http://www.google.com

<End of line>

 

[Patricia Shanahan]

"Daniel Pitts" <[email protected]> ha scritto nel messaggio


Hi Daniel,
thanks for your suggestion, very interesting!
Please, have you an example about this method? Or can you tell me an url
where I can find documentation about it?
Thanks again
JFM

Here's a snippet from one of my programs:

Connection con = Connect.getConnection(dbName);
con.setAutoCommit(false);
Statement st = con.createStatement();
st.setFetchSize(1000);
ResultSet result = st.executeQuery(getData);
while (result.next()) {
count(result.getInt(1), result.getDouble(2), result.getBoolean(3));
}

(Don't assume this is good style - I'm just learning combining Java and
database. It works with PostgreSQL.)

The main issue seems to be to design the query so that the results are
in an order in which you can process them. If you are not currently
processing the Vector in index order, you may need to add/change an
"ORDER BY" clause in the query.

Patricia